Image processing device, image printing system, and image processing program

Abstract

This technology facilitates the placement of images onto printable media with three-dimensional shapes and allows for a preview of the printed media after placement. [Solution] Image data of an image to be printed on a printing medium having a three-dimensional shape is acquired, a specification for changing the shape of at least a part of the printing medium is accepted, and the shape of the printing medium and the printable area on the printing medium are changed according to the specified change. Then, the acquired image is placed in the changed printable area, and the printing medium after the shape change and the image data placed in the printable area are rendered.

Description

Technical Field

[0001] The present disclosure relates to image processing technology when placing an image on a printing medium and previewing it.

Background Art

[0002] Conventionally, before printing using a printer or a printing machine, a preview of the printing medium has been displayed. For example, in Patent Document 1 below, an image is placed according to the layout pattern set for the printed matter and previewed. In Patent Document 1, the size of the printing medium can be changed, and when the size is changed, the size of the print data is automatically made to follow and previewed.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in Patent Document 1, the size of the image data in the preview image is only determined according to the size of the printing medium. That is, when the printing medium has a three-dimensional shape, how to handle and preview the image as the shape of the printing medium changes has not been sufficiently studied.

Means for Solving the Problems

[0005] The present disclosure can be realized in the following forms or application examples.

[0006] (1) The present disclosure can be implemented as an image processing apparatus. This image processing apparatus includes: an image acquisition unit that acquires image data of an image to be printed on a printing medium having a three-dimensional shape; a change designation unit that designates a change to the shape of at least a part of the printing medium; a change implementation unit that changes the shape of the printing medium and the printable area on the printing medium in accordance with the designated change; an image placement unit that places the image in the printable area after the change; and a rendering unit that renders the printing medium after the shape has been changed and the image data placed in the printable area.

[0007] (2) The present disclosure can also be implemented as an image printing system. This image printing system includes: an image acquisition unit that acquires image data of an image to be printed on a printing medium having a three-dimensional shape; a change designation unit that specifies a change to the shape of at least a part of the printing medium; a change implementation unit that changes the shape of the printing medium and the printable area on the printing medium in accordance with the specified change; an image placement unit that places the image in the printable area after the change; a rendering unit that renders the printing medium after the shape has been changed and the image data placed in the printable area; a display unit that displays the result of the rendering; and a printing unit that prints the image on the printing medium using the image data.

[0008] (3) The Disclosure can also be implemented as an image processing program that generates a rendered image of a printed medium on which an image is printed. This image processing program generates a rendered image of a printed medium on which an image is printed, and includes a function to acquire image data of an image to be printed on a printed medium having a three-dimensional shape; a function to accept a specification for changing the shape of at least a part of the printed medium; a function to change the shape of the printed medium and the printable area on the printed medium in accordance with the specified change; a function to place the image in the modified printable area; and a function to render the printed medium after the shape has been changed and the image data placed in the printable area, all of which are implemented by a computer. [Brief explanation of the drawing]

[0009] [Figure 1] A schematic diagram showing an image processing apparatus of an embodiment. [Figure 2] A schematic diagram of the image editing department. [Figure 3] A flowchart illustrating an example of an image processing control routine. [Figure 4] An explanatory diagram illustrating a part of the selection and editing process for print media with a three-dimensional shape. [Figure 5] An explanatory diagram showing an example of deformation processing for printed materials. [Figure 6] An explanatory diagram illustrating the printable area of ​​a printing medium having a three-dimensional shape. [Figure 7] An explanatory diagram illustrating an example of the process of editing print media and its images. [Figure 8] An explanatory diagram showing the editing process. [Figure 9] A schematic diagram of the rendering execution unit. [Figure 10] A schematic diagram illustrating an example of how a rendered image can be displayed. [Figure 11] An explanatory diagram showing the relationship between the light source, viewpoint, and the angles of the surfaces of a 3D object. [Figure 12] An explanatory diagram showing how the printable area and other aspects change as the printing medium expands. [Figure 13] An explanatory diagram showing how the printable area and image placement area change when the print medium is enlarged or reduced. [Figure 14] An explanatory diagram illustrating other ways in which the printable area and image placement area change when printing media are enlarged or reduced. [Figure 15] An explanatory diagram showing how to set printable and unprintable sizes for multiple print media of different sizes. [Figure 16] A schematic diagram of the image printing system. [Modes for carrying out the invention]

[0010] A. First Embodiment: (A1) Overall configuration of the image processing device 100: The overall configuration of the image processing apparatus 100 according to the first embodiment is shown in FIG. 1. This image processing apparatus 100 performs image processing for previewing how an image is printed on a predetermined printing medium. As shown in the figure, this image processing apparatus 100 includes an image editing unit 111, a rendering execution unit 121, a memory 135, a communication unit 141, and a rendering image display unit 151 inside. Although the configuration of each unit will be described later, a change specifying unit 60, an editing operation unit 70, an edited image display unit 80, etc. are connected to the image editing unit 111. Further, the memory 135 is configured such that first data FD and second data SD are input from the outside, and the rendering execution unit 121 is configured to perform rendering using these data. Further, the communication unit 141 is provided for connecting the image processing apparatus 100 to an external network NW, and the image processing apparatus 100 acquires a part of the second data SD from an external site 190 as necessary.

[0011] The image processing apparatus 100 inputs image data ORG from the outside and arranges an image on a pre-prepared printing medium. These operations are executed by the image editing unit 111. The image editing unit 111 includes a color management system 20, an image arranging unit 30, a change implementing unit 40, a printing medium database 50, etc., and outputs data in which an image is arranged on a printing medium MT having a three-dimensional shape to the rendering execution unit 121. The rendering execution unit 121 receives this and performs rendering processing, and displays the rendered image on the rendering image display unit 151. Note that the state of arranging the image data ORG on the printing medium MT etc. may be displayed on the edited image display unit 80 without performing rendering. Of course, rendering may be performed constantly, and the user may perform an editing operation while viewing the image on the rendering image display unit 151.

[0012] The image processing apparatus 100 not only performs image processing but also displays the processing result as a preview image. As shown in the figure, the image processing apparatus 100 mainly includes a color management system 20 that performs color conversion, a rendering execution unit 121 that executes rendering of a printing medium, a memory 135 including a first storage unit 131 and a second storage unit 132, a communication unit 141 that exchanges data with an external site 190 via a network NW such as the Internet, and an image display unit 151 that displays a preview image. Note that programs for performing each of the processes described later are stored in the memory 135 etc. of the image processing apparatus 100, and each function of the image processing apparatus 100 is realized when the CPU or GPU executes the programs stored in the memory.

[0013] The memory 135 takes in and stores the first data FD in the first storage unit 131 and takes in and stores the second data SD in the second storage unit 132. The first data FD and the second data SD are parameters necessary for physically-based rendering and displaying a printing medium for printing an image as a 3D object. In particular, the first data FD is data related to the form of the printing medium under a light source in a virtual space, and includes 3D object information of the printing medium, camera information such as the position where the printing medium is being viewed, lighting information such as the position and color tone of the lighting, and further background information indicating the background where the printing medium is placed. Also, the second data SD is data related to image formation on the surface of the printing medium, and includes, for example, data representing the texture of the surface of the printing medium. These first data FD and second data SD are used during rendering in the rendering execution unit 121.

[0014] For the first data FD and second data SD, representative data that is used more than a predetermined frequency can be stored non-volatile in the first storage unit 131 and second storage unit 132 in advance, and can be selected as needed and referenced by the rendering execution unit 121. For printing media that are not used frequently or not normally used, such as texture data for special materials such as cloth, cans, and plastic sheets, the data can be stored at an external site 190 and retrieved via the communication unit 141 when needed. For the first data FD, such as lighting information, the user may specify it individually during rendering, but representative camera angles and light sources can be stored in the first storage unit 131 in advance and used. The camera angle is the position and direction from which the target printing medium is viewed, and corresponds to the position and direction of the virtual viewpoint of the user viewing the virtual space. For this reason, the camera is sometimes described as the viewpoint or direction of the line of sight, and is referred to as the "viewpoint" or "view".

[0015] The image display unit 151 displays the image of the print medium rendered by the rendering execution unit 121, along with the background and other elements. The image display unit 151 reads the image data for display from the frame memory FM provided in the rendering execution unit 121 and performs the display. The image display unit 151 may be provided in the image processing device 100, or it may be provided separately from the image processing device 100. The image processing device 100 may be implemented as a dedicated machine, or it may be implemented by running an application program on a computer. Of course, the computer includes terminals such as tablets and mobile phones. Since the processing of the rendering execution unit 121 requires considerable resources and computing power, the rendering execution unit 121 alone may be run by a CPU capable of high-speed processing or a dedicated GPU, and the image processing device 100 may be configured with the rendering execution unit 121 located at a separate site on the network.

[0016] (A2) Structure and functions of the image editing unit 111: The image editing unit 111 performs editing tasks such as placing the input image data (ORG) onto the selected print medium (MT) from the print medium database 50, and outputs this to the rendering execution unit 121. The detailed processing will be described later, but the change specification unit 60 instructs changes to the shape of the print medium (MT), and the change implementation unit 40 of the image editing unit 111 changes the shape of the print medium (MT) according to the instructions. The editing operation unit 70 accepts editing operations such as where to place the image on the print medium (MT) and the size of the image, and outputs these operations to the image placement unit 30 of the image editing unit 111. The image placement unit 30 sets the image placement and size according to the received editing instructions.

[0017] Figure 2 shows the detailed configuration of the image editing unit 111 that performs these processes. The functions of each part of the image editing unit 111, such as the color management system 20, will be explained sequentially below. The color management system 20 will be abbreviated as CMS below for simplicity. The CMS 20 can acquire image data ORG representing the image to be printed (hereinafter referred to as the original image). The image data ORG may be received via wired or wireless communication from the image forming apparatus that created the image data ORG, or it may be read from a memory card that saves the image data ORG in file format. It may also be acquired via a network. Alternatively, the image data ORG may be created within this image processing device 100.

[0018] CMS20 converts the original image, which is the target of the print preview, to the object colors that will be represented on the print medium. The converted image data is called managed image data (MGP). Details of the CMS processing will be described later. Managed image data (MGP) is set as the texture of the print medium, which has a three-dimensional shape. The fact that the print medium (MT) has a three-dimensional shape is also referred to below as the print medium being a 3D object. CMS20 receives input profiles such as input profile IP, media profile MP, and common color space profile CP. The input profile IP converts the input color system, which is dependent on the device, from the device-dependent color system L* a * b * It is used to perform conversions to formats such as (hereinafter simply abbreviated as Lab). A media profile MP is a profile that represents the color reproduction when printed on a specific printing device, such as a printer, on a specific printing medium, under specific printing conditions such as printing resolution. It is a profile that converts color values ​​between a device-independent color system and a device-dependent color system. A media profile MP also includes information other than the printing medium, such as the printing settings of the printing device. Therefore, if we try to cover all combinations of printing device (printer) × printing medium × printing settings, the number of media profile MPs will increase. So, when the dependency on printing conditions is small, or when we do not want to increase the number of profiles, the media profile MP is configured as a combination of printing device (printer) × printing medium. Furthermore, multiple printing media with similar color characteristics may be grouped together and the same media profile MP may be applied to them. In this case, it is desirable to group printing media that not only have similar color characteristics but also the same or similar duty cycle limits according to the ink receiving capacity of the printing medium, and use a common media profile MP. Thus, since the color of an image on a print medium is influenced by both the characteristics of the printing device and the characteristics of the printing medium itself, the media profile MP is sometimes referred to as the print profile MP below.

[0019] Applying the input profile IP to the image data ORG, and then applying the print profile MP, yields color values ​​that would appear under specific printing conditions, i.e., color values ​​dependent on the printing device and medium. Applying the print profile MP to these image color values ​​to convert them from a device-dependent color system to a device-independent color system, and then applying the common color space profile CP, converts them to a representation in the second color space used during rendering (in this case, the sRGB color space). Because the image data ORG has been converted once using the print profile MP to color values ​​dependent on the characteristics of the printing device and medium, the image data ORG is color-converted to a range of actually printable color values. The common color space profile CP is used to convert the image data to color values ​​in the color space used during rendering. The sRGB color space is a typical common color space, but AdobeRGB, Display-P3, etc., may also be used.

[0020] As described above, CMS20 uses each profile to convert image data ORG, which is expressed in the first color space, a device-dependent color system, into image data (managed image data) MGP, which is expressed in the sRGB color space, the second color space used during rendering. Here, the converted image data is not limited to the color values ​​of the sRGB color space, but can be expressed in any color space that the rendering execution unit 121 can handle. For example, if the rendering execution unit 121 employs a configuration that allows rendering using color values ​​of Lab or XYZ color spaces, the image data can be converted to the color values ​​used when displaying it on the image display unit 151 during the lighting processing (described later) performed within the rendering execution unit 121, or in the post-processing unit (described later) located after the rendering execution unit 121.

[0021] While the CMS20 handles images placed on the print medium MT, the image placement unit 30 and modification implementation unit 40 in the image editing unit 111 set the shape of the print medium MT and perform processing such as determining where the image will be placed on the print medium MT. The print medium database 50 stores data for multiple print mediums MT, and the user selects a print medium MT with a three-dimensional shape from this database to print an image. The print medium database 50 stores data for multiple types of print mediums MT. There are no particular restrictions on the print medium MT that can be printed, such as ceramics like mugs, plastic bottles, plastic smartphone cases, fabric pen cases, bags, and clothing; as long as it is printable. The shape of the selected print medium MT is modified by the modification implementation unit 40 based on a specification from the modification specification unit 60.

[0022] The image processed by the CMS20 is placed on the modified print medium MT. This placement process is performed by the image placement unit 30. The position and size of the image on the print medium MT are set by editing operations from the editing operation unit 70. The result is displayed on the edited image display unit 80. The change specification unit 60 and editing operation unit 70 used here are specifically implemented by pointing devices such as a mouse or input devices such as a keyboard. Change specifications and editing operations may also be input using voice recognition or the like.

[0023] The CMS20 in the image editing unit 111 takes image data (ORG) as input and outputs managed image data (MGP). The change execution unit 40 outputs print media data (TOI), which is data such as the shape and surface condition of the print medium (MT) on which the image is printed. Furthermore, the image placement unit 30 outputs image placement data (GLD), which includes the placement position of the image on the print medium (MT), as well as the image magnification and rotation angle. All of this data together is called the processing target data (RS). The processing target data (RS) is output to the rendering execution unit 121 for rendering.

[0024] Next, the image processing control process performed by the image editing unit 111 using the CMS 20, image placement unit 30, and change implementation unit 40 will be explained with reference to Figure 3. When the image editing unit 111 starts this process, it first performs the process of selecting and editing the print medium MT (step S71). The print medium MT is registered in advance in the print medium database 50, and the user selects the print medium MT on which to print the image by looking at a list or the like. Alternatively, the system may print to a predetermined print medium MT.

[0025] Figure 4 is an explanatory diagram illustrating a part of the print media selection and editing process. The print media MTs registered in the print media database 50 have their initial values, such as shape, size, surface texture, and color, pre-registered. The user selects one of the registered MTs. The selection process involves displaying thumbnails of the print media MTs on the editing image display unit 80, and selecting one of them using the pointing device of the change specification unit 60. The selected print media MT is then rendered by the rendering execution unit 121 and displayed on the image display unit 151. This process is shown in the middle section of the figure.

[0026] Here, mug OJ1 is selected as the printing medium MT. This mug OJ1 comprises a cup body cp with a three-dimensional shape and a handle hn. The cup body cp and the handle hn are registered as separate objects, with their shapes and sizes defined, and the handle hn is configured to be joined to the side surface of the cup body cp. The cup body cp of mug OJ1 is registered as having a cylindrical shape with a bottom and a predetermined thickness. Immediately after selection, mug OJ1 is displayed as if it were placed on a horizontal surface. In this case, the horizontal surface is the XZ plane in a Cartesian coordinate system represented using XYZ, as shown in the figure. The height direction of mug OJ1 coincides with the Y axis. That is, the axial direction of the cylinder coincides with the height direction of mug OJ1, and this is defined as the Y direction. The two directions that are perpendicular to this Y direction and are also perpendicular to each other are the X direction and the Z direction.

[0027] The diameter of the cup body cp is determined by the dimensions in the X and Z directions. If the dimensions in the X and Z directions are the same, a perfect circle is formed. The dimensions of the X, Y, and Z directions of an object may be set individually and freely, but for each object, for example, the dimensions in the X and Z directions of the cup body cp may be restricted to be linked. Hereafter, the dimensions and scales of mug cup OJ1 and other objects will be expressed along the XYZ axes that constitute this Cartesian coordinate system, and each axis direction is shown as appropriate in other drawings. In addition, although a Cartesian coordinate system is used here, a coordinate system that specifies by height, diameter, and thickness may also be used.

[0028] In this embodiment, the shape of the mug OJ1 can be freely set, but if the mug used as the printing medium is made of ceramic, it is not easy to mold it to any size. One reason for changing the shape of such a printing medium MT is that in an environment where a printing medium MT with a three-dimensional shape can be freely manufactured using a 3D printer, it is conceivable that the shape of the printing medium MT can be changed and printing performed on it. Alternatively, the shape of the printing medium MT may be changed prior to the manufacture of a printing medium MT with a three-dimensional shape, for the purpose of determining what shape and size of printing medium MT is preferable from a design perspective or in terms of customer preference. In fact, if multiple types of MT to be printed are available, for example, if multiple mugs are available ranging from the smallest to those with diameters and heights increasing by 10 mm increments, and the user selects one of them to print, it is sufficient to limit the degree of change in each axial direction to be specified in 10 mm increments. The same applies to printing medium MTs other than mugs, and 10 mm is merely an example.

[0029] In this embodiment, the selected print medium MT is rendered and then displayed on the image display unit 151. However, thumbnails used during the selection process of the print medium MT, and dialog boxes for numerical input during the editing process described later, are displayed on the edited image display unit 80. These may also be displayed on the image display unit 151 together with the rendered image.

[0030] The printable medium MT, a mug OJ1, has a printable area GA defined as the area where an image can be printed. In this example, the printable area GA is set on the outer surface of the cup body cp. Although the printable area GA is located on the outer surface of the three-dimensional cup body cp, it is treated as a rectangular area obtained by unfolding the outer surface of the cup body cp into a plane, as will be described later. The printable area GA will be described later.

[0031] In step S71, the selected print medium MT is edited. This editing includes changing the shape of the print medium MT as specified by the user, and editing the possible placement positions of images, which will be described later. An example of editing, specifically changing the external shape of the print medium MT, a mug OJ1, is shown at the bottom of the figure. Here, the dialogue box DA1 and display box DD1 displayed on the edited image display unit 80 are shown. Dialogue box DA1 shows the magnification in each axis direction when changing the shape of the print medium MT, and display box DD1 shows the dimensions in each axis direction of the print medium MT at the magnification shown in dialogue box DA1. When mug OJ1 is selected, the magnification in each axis direction is 1.0, the outer diameter of the cup body cp is 70 mm, and the height is 150 mm. Alternatively, display box DD1 may be a dialogue box similar to dialogue box DA1, allowing direct input of the lengths of each axis.

[0032] In the editing image display unit 80 shown at the bottom of the diagram, a button BM1 labeled "Execute" is displayed along with a dialogue box DA1 and a display box DD1. This button BM1 shown in the editing image display unit 80 is used by the user to give instructions to the image placement unit 30, etc. of the image editing unit 111, using the change specification unit 60, etc. To distinguish these buttons, the instruction name written on the button is used, and below it will be described as "Execute" button BM1. The usage of the "Execute" button BM1 will be explained later. In addition, the editing image display unit 80 also displays various other buttons used in normal editing work, such as the "Back" button that returns to the previous state, as needed, but the illustration and detailed explanation are omitted.

[0033] In step S71, the selection and editing process of the printing medium allows you to change the size of the mug OJ1. This is shown in Figure 5. As shown in the upper part of the figure, the rendered image of the mug OJ1 before the shape change is displayed in the image display unit 151. In the middle part, the dialogue box DA1 and display box DD1 displayed in the edited image display unit 80 are shown. In the illustrated example, the X-axis scaling ratio of the dialogue box DA1 is set to a value of 1.3, and accordingly, the dimensions of the mug OJ1 in the X-axis and Z-axis directions are changed from the original "70mm" (see bottom of Figure 4) to "91mm", which is 1.3 times that value.

[0034] In this state, clicking the "Execute" button BM1 displayed on the edited image display unit 80 modifies the shape of the mug OJ1 to the dimensions in each axis direction determined from the initial value and the scaling ratio. The rendered image of the mug OJ1 with an enlarged diameter is displayed on the image display unit 151, as shown at the bottom of the figure. After selecting the mug OJ1 and editing its shape (step S71), the process of placing and editing an image on the mug OJ1 is performed (step S81). In preparation for this process, a printable area GA is provided for each print medium MT. In the figure, this area is shown by a dashed line. The printable area GA is predetermined for each print medium MT and stored in the print medium database 50.

[0035] In the mug OJ1, the printable area GA is located on the outer surface of the cup body cp. In the mug OJ1, the connection point of the handle hn and the surrounding connection area are excluded from the printable area GA. This is because irregularly shaped areas may not be suitable for printing. The arrangement of the printable area GA, excluding these specific parts, is explained using Figure 6. The top row of the figure shows the rendered image of the mug OJ1 displayed on the image display unit 151 after being magnified to a predetermined magnification, in this case 1.3 times in the X and Z directions. At this time, the editing image display unit 80 displays a dialogue box DA2 that shows the magnification / reduction ratio in each axis direction, and a display box DD2 that shows the size of the printable area. The dialogue box DA2 displays the current magnification / reduction ratio, and the display box DD2 displays the dimensions in each axis direction before and after magnification. The editing image display unit 80 also displays the "Unfolded View Display" button BM2 and the "Image Processing" button B3.

[0036] When the "Display Unfolded Diagram" button BM2 is clicked, the editable image display unit 80 displays an unfolded diagram of the printable area GA, as shown in the lower part of the figure. The figure shows both the unfolded diagram before and after enlargement, but only the unfolded diagram after enlargement is also acceptable. In order to make it easier to understand the size of the printable area GA in the three-dimensional shape, the unfolded diagram here shows the outer surface of the cup body cp unfolded into a plane. The unfolding of the outer surface of the three-dimensional shape can be done at any position as the end, but it is desirable to show the printable area GA as a continuous area, as shown in the figure. In this case, as shown in the figure, handles hn may be drawn at both ends, or they may be displayed at only one end. Alternatively, they may not be displayed at all. In this case, the printable area GA covers most of the outer surface of the unfolded cup body cp, but its size and arrangement are arbitrary. The initial value may be made small, for example, half of the circumferential direction. In this example, the printable area GA is defined as the outer surface of the cup body cp of the mug OJ1. However, if you intend to print a small mark or logo, you could also set the printable area GA on the handle hn. Of course, setting it on both sides is also acceptable.

[0037] In the diagram, the areas hatched with thick diagonal lines sloping downwards to the right are the printable areas GA0 and GA1. The outer areas not included in these areas, i.e., the areas hatched with thin diagonal lines sloping downwards to the left, are the excluded areas Gin. These excluded areas Gin are sometimes called non-printable areas because printing cannot be performed on them. Inside the printable areas GA0 and GA1, an image placement area GH is set. The image placement area GH is an area pre-set as the area where the image will be placed in the processing in step S81 described later, and it has a default position and size. The position and size of this image placement area GH can be arbitrarily changed by the user within the printable areas GA0 and GA1, but in this embodiment, in the enlarged printable area GA1, the position of the image placement area GH is set to be shifted parallel to the direction away from the handle hn in order to balance the image position. In the case of reduction, it is moved closer to the handle hn.

[0038] As already explained, the "Image Processing" button BM3 is displayed in the editable image display unit 80, and clicking it moves the process to image processing (step S81). In this process, an image is selected and edited. Editing in this case includes the position, size, and rotation angle of the image. The image is selected by the image editing unit 111 specifying the image data ORG to be imported from an external source. The imported image data ORG is first placed in the image placement area GH within the pre-prepared printable area GA. The placement is shown in Figure 7. The upper part of the figure shows the image placement when the shape of the mug OJ1 is at its initial value, and the lower part of the figure shows the image placement when the shape of the mug OJ1 is scaled up by 1.3 times in the X and Z directions.

[0039] In the diagram, the magnification is shown in dialogue boxes DA3 and DA4, and the sizes of the enlarged printable area GA and image placement area GH are shown in display boxes DD3 and DD4, respectively. In this example, as shown in the diagram, changing the dimensions of the mug OJ1 in each axial direction changes the dimensions of the printable area GA and image placement area GH accordingly. The image IM is initially positioned approximately in the center of the image placement area GH. As an example shown in Figure 8, when an image data ORG is selected, the image IM corresponding to the image data ORG is displayed at any position on the editing image display unit 80, and the user can move it using the pointing device on the editing operation unit 70 to position it in the desired location within the image placement area GH. Alternatively, the image IM can be dragged to any position within the image placement area GH, where it is automatically positioned in the center, and then its position within the image placement area GH can be fine-tuned.

[0040] After performing the print media selection and editing process (step S71) and the image selection and editing process (step S81) as described above, it is determined whether the editing is finalized (step S91). If not, the process is repeated from step S71. The completion of the editing process can be determined by whether or not the "Editing Complete" button (not shown) displayed in step S81 is clicked.

[0041] Once various editing processes are complete, the next step is color conversion by CMS20 (step S100). This process converts the original image data ORG into color data in a common color space for rendering using CMS20. Step S100 includes the processes described below (steps S110 to S160). When the color conversion process starts, the original image data ORG and input profile IP are first input, and the original image data ORG, which is represented in a device-dependent color system (e.g., RGB color system), is converted into color data in a device-independent color system (e.g., Lab or XYZ color system) (step S110). Next, it is determined whether a media profile MP is available (step S120). If a media profile MP exists, it is applied, and the color conversion is performed to the range of colors that can be expressed by printing, taking into account the combination of printing device (printer) and printing medium as printing conditions (step S130). If there is no media profile MP, the process in step S130 is not performed. Subsequently, the color values ​​are converted to the color space of the common color space, which is the second color space used during rendering, using the common color space profile CP (step S150). In this embodiment, sRGB is used as the common color space. The managed image data MGP obtained in this way is set to the albedo color, which is the texture of the 3D object (step S160), and this processing routine is terminated.

[0042] In step S130, setting the rendering intent for color conversion in the media profile to absolute allows the color of the print medium itself (background color) to be reflected. Note that if the color value of the image targeted for color conversion in step S150 is outside the sRGB color space gamut, it may be approximated to a value within the sRGB color space, or it may be treated in a way that allows it to take values ​​outside the sRGB color space. Generally, RGB values ​​of image data are stored as 8 bits per color, i.e., integers between 0 and 255, but this is not limited to this; any method that allows specifying a wider color range is acceptable. For example, by adopting signed integers or increasing the number of bits representing RGB values, negative values ​​and values ​​exceeding 255, i.e., values ​​outside the sRGB color space, can be easily handled. Alternatively, pixel values ​​can be represented as floating-point numbers between 0.0 and 1.0, and values ​​outside the sRGB color space can be handled as negative values ​​or values ​​exceeding 1.0.

[0043] The color conversion by CMS20 is not limited to the illustrated configuration; other methods are also possible. For example, after color conversion using media profile MP (step S130), color conversion processing using display device correction data DPD may be performed. Alternatively, composite correction data SPD may be prepared by pre-combining such display device correction data DPD and common color space profile CP, and color conversion using composite correction data SPD may be performed instead of color conversion using common color space profile CP (step S150). Alternatively, a device link profile may be prepared by combining input profile IP and media profile MP, and instead of applying input profile IP and media profile MP individually, the device link profile may be applied to perform the conversion all at once. Note that correction for the shift in display color of the image display unit 151 may be performed in the post-processing unit PST after the render backend, as shown in Figure 9, which will be described later, instead of being performed by CMS20.

[0044] (A3) Rendering process: The rendering execution unit 121 places the managed image data MGP, which is color-converted and output by the CMS 20, onto each print medium MT and renders it, thereby displaying on the image display unit 151 how the print medium MT, on which the original image data ORG is printed, looks in the virtual space. An example configuration of the rendering execution unit 121 is shown in Figure 9. This rendering execution unit 121 shows a typical configuration for physically based rendering processing, and other configurations can also be adopted. The rendering execution unit 121 of this embodiment employs a pipeline configuration that includes a vertex pipeline VPL and a pixel pipeline PPL, and performs physically based rendering at high speed. The vertex pipeline VPL includes a vertex shader VS and a geometry shader GS. A configuration without using the geometry shader GS is also possible.

[0045] The vertex shader VS converts the coordinates of the vertices of the 3D object, the print medium MT, from the print medium's coordinates to the 3D space being rendered. The coordinate transformation comprehensively includes coordinate transformations such as the coordinates of the model (in this case, the print medium) → world coordinates → view (camera) coordinates → clip coordinates, but the transformation to view coordinates is performed by the geometry shader GS. In addition, the vertex shader VS also performs shading and calculates texture coordinates (UV). In performing these processes, the vertex shader VS and geometry shader GS refer to various data received from the image editing unit 111, such as the 3D object data, print medium data TOI, image placement data GLD, camera information CMR, lighting information LGT, and background information BGD.

[0046] The print media data TOI is information regarding the shape of the print media as a 3D object. The print media dealt with in this embodiment has a three-dimensional shape, and its surface is not flat. Therefore, the print media MT is treated as a shape that has a three-dimensional shape and minute irregularities on its surface. Basically, it is treated as a collection of minute polygons. If the surface of the print media is represented by minute polygons, the number of polygons becomes enormous. For this reason, it is also practical to handle the surface of the print media using textures such as normal maps and height maps. Textures such as normal maps and height maps are provided as texture parameters, which will be described later. Camera information CMR is virtual information about the position and direction in which the camera is placed relative to the print media. Lighting information LGT includes at least one piece of virtual information such as the position, angle, intensity, and color temperature of the light source in the virtual space where the print media is placed. Note that multiple light sources can be set, in which case the effects of the multiple light sources can be calculated separately and superimposed on the 3D object.

[0047] Background information (BGD) is optional, but it contains information about the background on which the print medium, as a 3D object, is placed in the virtual space. Background information (BGD) includes information about objects such as walls, floors, and furniture placed in the virtual space, and these objects are rendered in the rendering execution unit 121 just like the print medium. In addition, since lighting hits these background objects and illuminates the print medium, it is also treated as part of the lighting information. By performing rendering using this various information, a three-dimensional preview becomes possible. The vertex information calculated by the vertex shader VS is passed to the geometry shader GS.

[0048] The geometry shader GS is used to manipulate the set of vertices within an object. The geometry shader GS allows for increasing or decreasing the number of vertices at runtime, and changing the types of primitives that make up a 3D object. One example of increasing or decreasing the number of vertices is culling, which excludes vertices that are not visible to the camera based on the camera's position and orientation. The geometry shader GS also generates new primitives from existing primitives such as points, lines, and triangles. The geometry shader GS receives primitives from the vertex shader VS, either the entire primitive or a primitive containing information about adjacent primitives. The geometry shader GS processes the input primitives and outputs a rasterized primitive.

[0049] The output of the vertex pipeline (VPL), specifically the primitives processed by the geometry shader (GS), is rasterized by the rasterizer (RRZ) to obtain pixel-level data, which is then passed to the pixel pipeline (PPL). In this embodiment, the pixel pipeline (PPL) comprises a pixel shader (PS) and a render backend (RBE).

[0050] The pixel shader PS manipulates rasterized pixels, and simply put, calculates the color of each pixel. Based on information input from the vertex shader VS and geometry shader GS, it performs processes such as compositing textures and applying surface colors. The pixel shader PS maps managed image data MGP, which is obtained by converting image data ORG using CMS20 based on various profiles, onto the print medium as a 3D object. At this time, the lighting processing function provided in the pixel shader PS performs lighting processing and mapping the managed image data MGP based on the object's light reflection model, the illumination information LGT mentioned above, and the texture parameter TXT, which is one of the second data SD stored in the second storage unit 132. The texture parameter TXT is also specified for the surface of the print medium MT, but it is important in the areas where the image IM is printed on the print medium MT. If printing on the print medium MT is done by a transfer method, the parameter used represents the texture when the ink printed on the transfer paper is transferred to the surface of the print medium MT. When printing is performed by directly ejecting UV ink onto the printing medium (MT), a parameter is used that represents the texture after the ink has been cured by ultraviolet light. The reflection model used in the lighting process is one of the mathematical models used to simulate illumination phenomena in the real world. The reflection model used in this embodiment will be explained in detail later.

[0051] Pixel manipulation processes become computationally intensive and time-consuming when the number of rasterized pixels increases, such as when the output resolution is high. Therefore, compared to vertex-level processing, it can take longer and result in inefficient pipeline processing. In this embodiment, the pixel shader PS processing program is optimized for execution on a GPU with high parallel processing capabilities, enabling advanced effects, including texture rendering, to be achieved in a short time.

[0052] The pixel information obtained through the processing of the pixel shader PS is further evaluated by the render backend (RBE) to determine whether or not to write it to the frame memory (FM) for display. Only when the render backend (RBE) determines that it is safe to write the pixel data to the frame memory (FM) is the pixel data saved as something to be rendered. Well-known tests used to determine whether to write include the "alpha test," "depth test," and "stencil test." The render backend (RBE) executes the configured test from among these tests and writes the pixel data to the frame memory (FM).

[0053] With the above processing completes the rendering pipeline, the post-processing unit (PST) then performs processing on the data stored in the frame memory (FM) to improve its appearance. Examples of such processing include anti-aliasing, which smooths out unwanted edges in the image. Other processes include ambient occlusion, screen-space reflection, and depth of field. The post-processing unit (PST) should be configured to perform the necessary post-processing.

[0054] The rendering execution unit 121 completes the above processing, and the rendering is finished. The result is output as the render result RRD. In practice, the data written to the frame memory FM is read out in accordance with the display cycle of the image display unit 151 and displayed as the render result RRD. An example of the render result RRD is shown in Figure 10. In this example, the image display unit 151 displays a mug OJ1, which is one of the printing media as a 3D object placed in the virtual space, a light source LG, and a background object Bob, which is part of a table and exists as one of the backgrounds.

[0055] Figure 11 illustrates the relationship between the print medium MT (mug cup OJ1) placed in the virtual space, the light source LG, and the viewpoint (camera) VP. The relationship between the light source LG, viewpoint VP, and the print medium MT is three-dimensional within the virtual space VSP, but the figure shows the virtual space VSP as the xz plane. x is the coordinate of the point where each of the vectors described below converges. The figure illustrates the positional relationship between the light source LG and viewpoint VP that illuminate the print medium MT, on which the image IM, the object of rendering, is printed, at a given coordinate x. The figure shows the light source direction vector ωl from coordinate x to the light source LG, the viewpoint direction vector ωv from coordinate x to the viewpoint VP, and the half vector HV of both. Furthermore, the symbol Np represents the normal vector assuming that the surface of the print medium MT is a virtual plane PLp when viewed microscopically, and the symbol Nb represents the normal vector at coordinate x of the actual plane PLb of the actual print medium MT, which is not a plane when viewed microscopically.

[0056] In the image processing device 100 of this embodiment, the position and angle of the print medium in the virtual space can be freely changed, and the appearance of the print medium and the image on the print medium can be checked. As shown in Figure 10, this can be achieved by a series of processes in which, when the position and angle of the three-dimensional print medium MT, which is a mug OJ1, are changed by operating a pointing device on the image displayed on the image display unit 151 and dragging the object with the pointer PTD displayed on the screen by the pointing device, the rendering execution unit 121 performs rendering processing each time, and the processing result is displayed on the image display unit 151. The pointing device may be a 3D mouse or a tracking ball, or it may be a type that is operated with a finger or stylus on a multi-touch panel provided on the image display unit 151. For example, if a multi-touch panel is provided on the surface of the image display unit 151, the print medium MT and light source LG may be moved directly with a finger, or two fingers may be used to rotate the print medium MT or change the distance between the light source LG and the actual plane PLb of the print medium in three dimensions.

[0057] When the position and angle of the print medium MT or light source LG in this virtual space are changed, the rendering execution unit 121 performs rendering processing each time, and the rendered result RRD is displayed on the image display unit 151. When the position and angle of the print medium MT or light source LG in the virtual space are changed, the print medium on which the image is printed is physically rendered each time, and the actual print medium on which the image is printed is shown in a manner close to how it would appear in real space.

[0058] In particular, in this embodiment, in addition to converting the colors of the image to be printed on the print medium to the actual colors of the printed image using a color management system (CMS), during the lighting process during rendering, [1] The printing medium on which the image is printed is being treated as a 3D object. [2] The texture parameter TXT is used to consider the texture of the surface of the print medium and the ink printed on the surface. Therefore, the reproducibility of the printed medium displayed on the image display unit 151 is extremely high. The following describes the processes of [1] and [2].

[0059] Regarding [1]: How a 3D object appears in a virtual space can be represented using the bidirectional reflectance distribution function (BRDF) and luminance of the reflected light at each part of the object. The bidirectional reflectance distribution function (BRDF) indicates the angular distribution characteristics of the reflected light when light is incident from a specific angle. Luminance is the brightness of the object. Together, these are called the illumination model. An example of the reflection model adopted in this embodiment is shown below. The BRDF can be expressed as a function f(x,ωl,ωv), and the luminance as a function L(x,ωv), as shown in the following equations (1) and (2). f(x,ωl,ωv)=kD / π+kS*(F*D*V) …(1) L(x,ωv)=f(x,ωl,ωv)*E⊥(x)*n·ωl …(2) x: coordinate within the plane, ωv: viewpoint direction vector, ωl: light source direction vector kD: Diffuse Albedo, kS: Specular Albedo F: Fresnel term, D: Normal distribution function, V: Geometric decay term E⊥(x): Illuminance incident perpendicular to coordinate x, n: normal vector

[0060] The first term of the BRDF, kD / π, is the diffuse reflection component and is based on the Lambert model. The second term is the specular reflection component and is based on the Cook-Torrance model. In equation (1), kd / π is sometimes called the diffuse reflection term, and kS*(F*D*V) is sometimes called the specular reflection term. The Fresnel term F, the normal distribution function D, and the geometric decay term V are based on well-known models and calculation methods, so their explanation is omitted. As for the BRDF, any function appropriate to the reflection characteristics of the 3D object surface and the purpose of rendering may be used. For example, the Disney Principled BRDF may be used. In this embodiment, the BRDF is used as the function representing light reflection, but the Bidirectional Scattering Surface Reflectance Distribution Function (BSSRDF) may also be used as the function representing light reflection.

[0061] As can be seen from equations (1) and (2) above, the calculation of the above reflection model requires the normal vector n, the light source direction vector ωl, and the viewpoint direction vector ωv. Printed media are treated as 3D objects composed of multiple minute polygons for rendering purposes, but the normal vector n, which reflects the minute irregularities on the surface of the printed media, is calculated from the polygon normal Np and the normal map described later. Therefore, the vertex pipeline VPL calculates the polygon normal Np and the UV coordinates that determine the reference position of the normal map, and inputs these, along with the light source direction vector ωl and the viewpoint direction vector ωv, into the pixel pipeline PPL. In the pixel pipeline PPL, the pixel shader PS references the normal map, which is given as one of the texture parameters, using the UV coordinates, and calculates the normal vector n from the referenced normal map value and the polygon normal Np.

[0062] In this embodiment, as described above, the printing medium MT on which the image IM is printed is a 3D object with a three-dimensional shape, and physically based rendering is performed according to the above equations (1) and (2). The light source direction vector ωl and the viewpoint direction vector ωv are calculated each time the user changes the position and angle of the real plane PLb of the printing medium or the light source LG in the virtual space using a pointing device, as shown in Figure 10.

[0063] Regarding [2]: In this embodiment, the texture of the surface of the printing medium and the texture of the printed ink are considered using the texture parameter TXT. The texture parameter TXT may include the following, but it is not necessary to consider all of them; at least one of the parameters listed below, for example, smoothness, should be considered. • Smoothness (S) or Roughness (R): This parameter indicates the smoothness of the surface of a 3D object. Smoothness S is generally specified within the range of 0.0 to 1.0. Smoothness S affects the normal distribution function D and the geometric attenuation term V in equation (1) BRDF mentioned above. A larger value results in stronger specular reflection and a glossy appearance. Roughness R can be used instead of smoothness S. The two are convertible using the formula S = 1.0 - R. Note that smoothness is sometimes referred to as "smoothness," and roughness as "roughness."

[0064] ·Metallic M (metallic): This indicates the degree to which a 3D object's surface is metallic. A higher metallicity (M) value indicates a higher surface metallicity. When metallicity (M) is high, the object's surface reflects ambient light more easily, reflecting the surrounding scenery and obscuring the object's own color. Metallicity (M) affects the Fresnel term (F). The Fresnel term F can be expressed as equation (3) below using the Schlick approximation. F(ωl,h)=F0+(1-F0)(1-ωl·h) 5 …(3) Here, h is the half-vector of the viewpoint direction vector ωv and the light source direction vector ωl, and F0 is the specular reflectance when perpendicularly incident. The specular reflectance F0 can be directly specified as the specular color of the specularly reflected light, or it can be given by linear interpolation (referred to here as the lerp function) using the metallicity M in equation (4). F0 = lerp(0.04, tC, M) …(4) Here, tC is the texture color (albedoColor) of the 3D object. The value 0.04 in equation (4) is a representative value for each RGB component, showing typical values ​​for nonmetals. The same applies to the texture color tC. Although we refer to this as metallicity, materials with high reflectivity and gloss, such as porcelain surfaces, are treated as having high metallicity M even if they are not metals.

[0065] • Normal Map: The normal map represents the normal vectors of the minute surface irregularities on the printed material. By associating (applying) the normal map to a 3D object, the normal vectors of the minute surface irregularities on the printed material can be assigned to the 3D object. The normal map can influence the Fresnel term F, normal distribution function D, and geometric decay term V of the BRDF.

[0066] Other texture parameters: Other parameters that can function as texture parameters include specular color, and clear coat layer parameters that indicate the presence or absence of a clear coat layer on the surface of the printing medium, its thickness, or its transparency.

[0067] As explained above, [1] Treating the printing medium on which the image is printed as a 3D object, [2] The texture parameter TXT is used to consider the texture of the surface of the printing medium and the surface of the printed ink. As a result, the image processing device 100 of this embodiment can display the appearance of the printed medium MT on which the image IM is printed on the image display unit 151 with a high degree of freedom and high reproducibility.

[0068] (A4) Effects of the first embodiment: According to the image processing apparatus 100 of the first embodiment described above, image data ORG of an image to be printed on a printing medium MT having a three-dimensional shape is acquired, and the change in the shape of the cup body cp portion of a mug OJ1, which is an example of the printing medium MT, is specified by a numerical value representing a magnification. In accordance with this specification, the shape of the mug OJ1, which is the printing medium, and the printable area GA set on the outer surface of the cup body cp are changed. Then, an image IM corresponding to the image data ORG is placed in the changed printable area GA, and the mug OJ1 after the shape change and the image IM placed in the printable area GA are rendered. Therefore, the user can change at least a part of the shape of the printing medium MT, place an image IM in the printable area GA that is changed in accordance with the shape change, and immediately check how it looks in that case by rendering it.

[0069] Furthermore, in this embodiment, the shape of the three-dimensional print medium MT can be changed by specifying a scaling factor, and this can be specified individually for each axis in the Cartesian coordinate system. Therefore, the shape of the print medium MT can be flexibly deformed. Also, in the case of an MT consisting of multiple objects, such as the mug OJ1 with a cup body cp and a handle hn, the shape changes of each object can be specified individually. For this reason, it is easy to adjust the balance, such as when the cup body cp is enlarged while the handle hn remains the same size, as in the case of the mug OJ1. Of course, the scaling of multiple objects can be linked.

[0070] In this embodiment, the shape of the print media MT prepared in the print media database 50 is changed by the shape modification unit 40, and then the image is placed. This makes it easy to change the shape, and eliminates the need to prepare a 3D model for each print media with a different shape. Even if the print media MT were prepared in stages as multiple models with, for example, a 10mm difference in size, the effort required to create these multiple 3D models would increase, and the data capacity would also increase as the number of polygons prepared for each model increases. In contrast, in this embodiment, the problem is addressed by changing the shape of one or a few print media MTs that have a three-dimensional shape, thus avoiding these issues, namely the increased effort required to prepare 3D models, the increased data capacity when the shape is defined in detail using polygons, and consequently the strain on the storage capacity of the storage device that stores the print media database 50.

[0071] Furthermore, in this embodiment, the shape of the image placement area GH, which is the area within the printable area GA where the image IM is preferentially placed, can be changed or enlarged / reductiond in accordance with the change in the printable area GA. Therefore, it is easy to make the shape of the image placed on the print medium MT conform to the changes in the print medium MT and the printable area GA. Of course, this also includes the option of not changing the image placement area GH in response to changes in the shape of the print medium MT or the printable area GA.

[0072] B. Second Embodiment: The image processing apparatus 100 of the second embodiment will now be described. The hardware configuration of the image processing apparatus 100 of the second embodiment is the same as that of the first embodiment, but the handling of the printable area GA and the image placement area GH differs. Figure 12 shows the appearance of each area after enlargement of the print medium MT, which is a mug OJ1. As shown, in the unfolded view of the mug OJ1 before enlargement, the width W1 of the excluded area Gin adjacent to the handle hn can remain as width W1 without enlargement, as shown in enlargement 1, when the mug OJ1 is enlarged 1.3 times in the X and Z directions. However, as shown in enlargement 2, it is enlarged to width W2 according to the enlargement ratio. Here, W2 = a·W1, where a = 1.3 However, the scaling factor a of the excluded area Gin may be the same as the scaling factor of the mug OJ1 itself (here, 1.3), or it may not be the same. In this embodiment, the excluded area Gin is not set in the Y direction of the mug OJ1, but it may be set. Even if an excluded area Gin is set at the Y-direction end of the mug OJ1, the Y-direction width of the excluded area Gin will not change unless expansion occurs in the Y direction. If expansion occurs in the Y direction, the excluded area Gin in the Y direction may be expanded in accordance with the scaling factor.

[0073] In the second embodiment, the image placement area GH maintains the same shape regardless of the change in the shape of the mug OJ1, but its placement position relative to the printable area GA0 is changed. This is because if the image placement area GH is kept at the same distance from the edge of the printable area GA0 before enlargement, it would give the impression that the image placement area GH is too close to the handle hn when viewed from the perspective of the entire mug OJ1. Therefore, the position of the image placement area GH is shifted parallel to the center of the printable area GA1 from the edge, according to the enlargement ratio. The amount of shift may be proportional to the enlargement ratio, or it may be a predetermined amount according to the enlargement ratio. Of course, the user may be able to freely adjust it using a pointing device or the like in the change specification unit 60.

[0074] The amount of movement in this case may be changed according to the width of the exclusion region Gin. The figure shows the case where the width of the exclusion region Gin is set to width W2, which is larger than the initial width W1, and the image placement region GH is moved further towards the center by width W3. This is an example where, if the exclusion region Gin is enlarged due to a change in the shape of the mug OJ1, it is determined that the deformation of the connection part with the handle hn is large, and the image placement region GH, where the image IM is placed, is moved further away from the deformed part. Of course, the opposite may also be true, such as moving towards the ends.

[0075] In this embodiment, the print medium MT is provided with an exclusion area Gin where no image is placed, and the printable area GA is modified taking the exclusion area Gin into consideration. Furthermore, when the modification specification unit 60 specifies a deformation of at least a part of the print medium MT, which is a mug OJ1, the setting of the exclusion area Gin is changed according to the deformation. This allows the printable area GA to be set appropriately.

[0076] C. Third Embodiment: Next, the image processing apparatus 100 of the third embodiment will be described. The image processing apparatus 100 of the third embodiment also has the same hardware configuration as the first and second embodiments. In the third embodiment, the handling of the image placement area GH is different. The handling of the image placement area GH in response to changes in the shape of the printing medium MT will be explained using Figure 13, with a mug OJ1 as an example. Figure 13 shows how the printable area GA and the image placement area GH change when the printing medium is enlarged or reduced. The top of the figure shows the printable area GA0 before the shape change. An image placement area GH is set at a predetermined position (initial position) inside this printable area GA0, and a pre-prepared image IM is placed there. Image IM is placed at the center of the image placement area GH.

[0077] The middle section of the figure shows the state after the size of the mug OJ1 has been enlarged by approximately 1.3 times in the XYZ three-axis directions. In this example, the shape of the mug OJ1 has been changed by 1.3 times in all XYZ directions, and the exclusion area Gin and the image placement area GH have also been enlarged in conjunction with this. In addition, the image IM has been enlarged in conjunction with the enlargement of the image placement area GH.

[0078] In this example, the image processing device 100 changes the size of the image placement area GH in accordance with the change in the shape of the print medium MT, which is the mug OJ1, and changes the size of the image IM itself accordingly. Alternatively, when the size of the image placement area GH is changed by changing the shape of the mug OJ1, the size of the image IM may not be changed. For example, the bottom panel shows the case where the size in the X and Z axes remains unchanged from the middle panel, but the magnification in the Y axis direction is set to about 0.7. Here, as the dimension of the mug OJ1 in the Y axis direction is shortened, the vertical and horizontal dimensions of the image placement area GH are also changed to about 0.7 times, but the size of the image IM has not been changed. As a result, the image IM does not fit entirely within the image placement area GH, and the right side of the image IM is partially cut off. Rendering may be performed as is, and the mug OJ1 with the image IM printed on the outer surface of the cup body cp may be displayed on the image display unit 151, or the image size may be reduced by the editing operation unit 70 to correct the image so that the image IM fits inside the image placement area GH. Alternatively, the width of the image placement area GH can be widened in the X and Z directions, allowing the entire image IM to be displayed while keeping its size the same. This would allow for a degree of automation in placing the image IM to be printed on the print medium MT, and with minimal editing, the image IM can be placed at the desired size and position, and the printed result can be previewed.

[0079] In the example shown in Figure 13, when the shape in the Y-axis direction was reduced, the vertical and horizontal dimensions of the image placement area GH were also reduced according to the reduction ratio. In contrast, as shown in Figure 14, when reduced in the Y-axis direction, the horizontal width of the image placement area GH is not changed, and the portion of the image that extends beyond the printable area GA1 in the Y-axis direction is excluded from the image placement area GH. In this example, the reference position of the image placement area GH is the bottom edge, and the position of the bottom edge of the image placement area GH, i.e., the position of the image IM, is also determined based on the bottom edge of the image placement area GH, and consequently, the separation distance H1 from the bottom edge of the printable area GA1 to the bottom edge of the image IM does not change before and after reduction. Note that the reference position is not limited to the bottom edge; the top edge or the center of the image placement area GH may also be used as the reference. Furthermore, as shown in the lower part of the figure, if the image IM extends beyond the image placement area GH, the size of the image placement area GH may be changed, and then the placement of the image IM or its size may be adjusted so that the image IM fits within the image placement area GH. These changes in arrangement and size adjustments may be performed automatically or manually using the editing operation unit 70. As with the other embodiments described above, the rendered preview image displayed on the image display unit 151 will be changed as a result of these modifications.

[0080] D. Fourth Embodiment: Next, the fourth embodiment will be described. The image processing apparatus 100 of the fourth embodiment has the same hardware configuration as the first to third embodiments. In the fourth embodiment, a tote bag Bg is selected from the print media database 50 as the print medium MT to be printed. In the mug OJ1 described in the first to third embodiments, its three-dimensional shape was changed by directly specifying the magnification in each axis direction (X, Y, Z) and dimensions in predetermined units. In contrast, in the fourth embodiment, the size of the print medium MT is specified using an index indicating size, such as large, medium, and small. Specifically, as shown in Figure 15, five types are available, in order from smallest to largest: "Mini" bag represented by the code BgM, "Small" bag represented by the code BgS, "Half" bag represented by the code BgH, "Large" bag represented by the code BgL, and "Extra" bag represented by the code BgE.

[0081] As illustrated, the size of these printable media MTs cannot be continuously changed, and several sizes are provided in advance. Moreover, the dimensions of these bags, such as length, width, and handle length, are not proportional. For example, the size of the handle hs differs greatly between a bag held in the hand and a bag worn over the shoulder. For a bag worn over the shoulder, even if the bag body bg is large, the length of the handle hs does not change significantly. Also, in this example, the printable area is limited to the area enclosed by the strap of the handle hs. As shown in the table in the figure, the printable area ma for the mini bag BgM is 80 x 130 mm, the printable area sa for the small bag BgS is 80 x 234 mm, the printable area ha for the half bag BgH is 80 x 260 mm, the printable area la for the large bag BgL is 80 x 312 mm, and the printable area ea for the extra bag BgE is 80 x 325 mm.

[0082] In this embodiment, the printable area is defined by exclusion areas provided on both the left and right (X direction) sides of each bag. The printable area is the total dimensions minus the left and right exclusion areas. In this example, the left and right exclusion areas are of equal width, so the table shows the size of one exclusion area; however, if they differ, they should be specified separately. For illustrative purposes, this exclusion area pia is shown only for the extra bag BgE, but as shown in the table, it is set for each bag. Similarly, for illustrative purposes, the handle code hs and the bag body code bg are also shown only for the extra bag BgE.

[0083] If the shape of the printing medium MT is limited to a pre-defined set of types, the shape of the printing medium MT is specified using indicators such as "mini," "small," and "half," instead of specifying by scaling ratio or dimensions in each axis direction, as in the first to third embodiments. Alternatively, it can be specified by selecting from thumbnail images as shown in the figure. When any of the bags, for example "half," is selected as the printing medium MT, the rendered half bag BgH is displayed on the image display unit 151. During rendering, various texture parameters TXT and information on the light source LG are output to the rendering execution unit 121, as in the embodiments described above, and a preview image with the surface texture of the half bag BgH is displayed.

[0084] Furthermore, similar to the first to third embodiments, when the image data ORG is acquired, the image IM corresponding to the image data ORG is placed in the printable area set on the bag, for example, in the printable area ah if there is a half bag BgH. At this time, the image data ORG is converted to image data MGP2 by the CMS20 and output to the rendering execution unit 121 for rendering, as in the embodiments described above.

[0085] In this embodiment, the shape of the printing medium MT is changed by selecting from a number of pre-prepared shapes. A printable area is set for each selected bag, and the image is placed in this area. As in other embodiments, an image placement area may be provided within the printable area, and editing of the image placement area, as well as scaling and rotation of the image within the image placement area, may be performed via the editing operation unit 70. In this example, regardless of the size of the bag shape, the width of the printable area is the same at 80 mm, so the range of image placement may be limited to specifying only the position in the Y direction, without considering scaling the image.

[0086] According to the image processing apparatus 100 of the fourth embodiment described above, even when the shape is changed by selecting one of several types of forms, an image can be placed by setting a printable area for each form, and the state can be rendered and displayed as a preview image on the image display unit 151.

[0087] E. Fifth Embodiment: In the embodiments described above, the print medium MT and the image IM printed thereon are virtual entities handled within the image processing device 100. Rendering is performed using various data, and the image, including its texture, is displayed as a preview image on the image display unit 151. In contrast, the fifth embodiment describes an image printing system 300 that actually performs printing using the image processing device 100. Figure 16 is a schematic diagram showing the configuration of this image printing system 300. The image printing system 300 comprises the image processing device 100 described in the first to fourth embodiments, an image display unit 151 that displays the preview image, and a printing device 200 that performs printing. In the figure, the image processing device 100 is shown to receive image data ORG, print medium data MTD of the print medium MT, and texture / environment parameters TXD. The print medium data MTD includes data such as the specifications of the print medium MT stored in the print medium database 50, and the scaling ratio of its shape. The environment parameters TXD include various information necessary for previewing, such as the texture parameters TXT and lighting information LGT. The contents of this data have already been explained in the configuration and operation of the rendering execution unit 121.

[0088] The printing device 200 prints an image onto a three-dimensional printing medium MT. While ceramics such as mugs are used as an example here, various other printing mediums MT can be used, including plastic bottles, plastic smartphone cases, fabric pen cases, and tote bags. Direct printing is possible on the three-dimensional printing medium MT, but the printing device 200 uses transfer printing.

[0089] Transfer printing is performed using the following procedure. First, ink is applied to the recording surface of the transfer paper PRS by printing with the printing device 200. Then, with the recording surface of the transfer paper PRS pressed against the printing medium MT, both the transfer paper PRS and the printing medium MT are heated. As a result, an image is printed onto the object by thermal transfer. Transfer printing methods include sublimation transfer and DTF (Direct to Film) transfer.

[0090] This embodiment describes an example of sublimation transfer. The printing device 200 prints a specified image onto the transfer paper PRS. The printing device 200 inverts the image data received from the image processing device 100, flipping the image horizontally. The horizontally flipped image is printed on the recording surface of the transfer paper PRS. The surface of the transfer paper PRS is the recording surface. With the recording surface of the printed transfer paper PRS pressed against the surface of the printing medium MT, the printed transfer paper PRS and the printing medium MT are heated. For heat transfer, a press machine appropriate to the shape of the printing medium MT is used. If the object to be printed is fabric, a flat press machine is used. If the object to be printed is a cylindrical object such as a mug or glass, a mug press machine is used. If the object to be printed is a three-dimensional object such as a smartphone case, a vacuum press machine is used. As a result of the transfer process, a product with the image printed on its surface is obtained.

[0091] In this example, a printable medium MT of a predetermined shape is used. However, if a 3D printer 400 is prepared and the scaling ratio is input to the image processing device 100, and the size of the printable medium MT is changed to an arbitrary size, the 3D printer 400 may be used on the spot to mold the printable medium MT of the changed size using the data from the image processing device 100. Such molding of arbitrary shapes is not limited to 3D printers; for example, if the printable medium MT is a garment, it can also be achieved by a manufacturing system that cuts the fabric with a cutting machine and sews it with an automatic sewing machine.

[0092] In the above embodiment, a bag was used as an example. While it was explained that multiple printing media with different shapes can be selected and specified, it is not limited to bags; similar methods can be applied to clothing, shoes, and other items. When the printing medium has a complex shape and is composed of multiple parts sewn together or joined, the printable area and image placement area can be set for each part that makes up the printing medium, and data for each part can be prepared in accordance with changes in the shape of the printing medium.

[0093] F. Other embodiments: (1) The disclosure can also be implemented in the following forms. One other embodiment is configured as an image processing apparatus. This image processing apparatus includes an image acquisition unit that acquires image data of an image to be printed on a printing medium having a three-dimensional shape; a change designation unit that designates a change to the shape of at least a part of the printing medium; a change implementation unit that changes the shape of the printing medium and the printable area on the printing medium in accordance with the designated change; an image placement unit that places the image in the printable area after the change; and a rendering unit that renders the printing medium after the shape has been changed and the image data placed in the printable area.

[0094] This allows users to modify at least a portion of the print medium's shape, place images within the printable area that changes accordingly, and immediately view a rendering of how it will look. Furthermore, data such as shape and surface texture only needs to be prepared for one print medium, eliminating the need to prepare data for a wide variety of three-dimensional shapes. Additionally, users can easily obtain a preview using a rendered image to adjust changes to the print medium's shape, image placement, and size.

[0095] (2) In the above configuration, the change specification unit accepts a specification of a shape change using parameters for each axis of the three-dimensional shape, and the parameters are [1] A numerical value that specifies the rate, ratio, or amount of enlargement or reduction. [2] A numerical value indicating the dimensions after enlargement or reduction in a predetermined unit. [3] Multiple selectable indicators showing the degree of change, It can be one of the following. This allows for specifying shape changes using various methods. The scaling ratio can be specified as a percentage or a ratio where 1.0 is equal to the original size, and the ratio can be specified as "1:A" with the original value set to 1. Of course, a specification such as "2:3" is also acceptable. The amount of scaling is a specification such as "make it 2cm larger".

[0096] Numerical values ​​indicating the dimensions of enlargement or reduction in predetermined units refer to the size after shape modification, for example, specified as "XX centimeters in length and width," or dimensions along each axis using Cartesian coordinates. Multiple selectable indicators indicating the degree of change are indicators such as "large," "medium," and "small," or indicators such as "standard," "enlarged," and "reduced." The indicators may also be numerical values ​​such as "1" and "2," and the sizes indicated by the indicators do not need to be in a linear proportional relationship, nor do they need to be enlarged or reduced in the same way in all directions. Indicators such as "for adults" and "for children," or subjective indicators such as "cute" and "magnificent," are also acceptable. It is sufficient that the size of each part or direction of the printed material is linked to these indicators.

[0097] (3) In the configuration of (1) and (2) above, when the parameter is the index, each of the index may be set to at least one of the amount of change in size for each axis of the three-dimensional shape and the amount of translation for each axis. This makes it easy to define the change. Each axis of the three-dimensional shape may be each axis when the three-dimensional shape is represented in a Cartesian coordinate system, or, when representing a printing medium mainly consisting of a cylindrical shape such as a mug, the diameter and height may be used, and the axis in the diameter direction and the axis in the height direction may be considered as the axes.

[0098] (4) In the configurations of (1) to (3) above, the print medium is composed of multiple objects, and the change designation unit, when the parameter is the index, stores in advance the degree of change for each of the multiple objects associated with each of the index, and when one of the indexes is selected, the degree of change associated with the selected index is designated as the change designation for each of the objects. This allows for flexible changes to the shape of the print medium.

[0099] (5) In the configurations of (1) to (4) above, the change specification section may be configured to allow the modification of a part of the shape of the printed medium separately from the other parts of the shape. In this way, taking a mug as an example, the cup body that holds the liquid and the handle can be treated as separate objects, and when the cup body is enlarged by a predetermined ratio, the handle can be kept at its original size or enlarged by a smaller ratio than the enlargement of the cup body. When a printed medium serves as an article, changing the shape of each part by simple proportion may result in an imbalance as a three-dimensional shape. By composing the printed medium from multiple objects and defining the degree of change for each object, it is possible to maintain an appropriate overall balance even when the shape is changed, such as by enlarging or reducing it.

[0100] (6) In the configurations of (1) to (5) above, the modification implementation unit may change the shape of the image placement area, which is an area within the printable area where the image is preferentially placed, in accordance with the change in the printable area. This allows the image to be placed in the image placement area prepared in advance within the printable area, enabling appropriate placement of the image. Of course, the image placement position may be changed by the user, such as by moving or resizing it.

[0101] The following editing operations may be made possible for the images placed in the image placement section. In this case, if the edited image display section 80 of the image processing device 100 in the first to fourth embodiments is a touch panel device, the operation methods in parentheses can also be used. • Moving images (tap to select, drag to move) • Adding images (drag and drop) • Delete images (Tap to select image, double-tap to confirm deletion, delete upon agreement) • Resizing images (tap to select image, pinch to zoom in / out) • Image cropping (Tap to select image, then drag one of the four sides to crop) • Image rotation (tap to select image, pinch to rotate) Other operations, such as changing the image color or adding text, may also be enabled. Furthermore, it would be beneficial to provide features such as changing the layering order of multiple images, adjusting transparency, and providing filter functions for image editing. Implementing some or all of the above features is optional.

[0102] (7) In the above configuration, the modification implementation unit shall, (1) The shape of the image placement area is within the printable area. (2) A shape that is linked to a change in the shape of the printing medium, (3) The shape obtained by modifying the shape of the image placement area so that it exceeds the printable area. It may be changed to one of the following. This way, for example, if enlarging the image placement area causes it to extend beyond the printable area, the image can be changed to stay within the printable area, thus prioritizing the placement of the image within the printable area. Alternatively, the shape of the image placement area can be linked to changes in the shape of the printing medium, creating a relationship between the shapes of the printing medium and the image. For example, if the printing medium is made elongated in a particular direction, the image placement area can also be made to a corresponding shape. Furthermore, if the image placement area exceeds the printable area, the shape of the image placement area can be modified to prevent this, thereby maintaining the image placement area appropriately.

[0103] (8) In the configurations of (1) to (7) above, the printing medium is provided with exclusion areas where the image is not to be placed, and the modification implementation unit may modify the printable area taking the exclusion areas into consideration. In this way, the printable area can be modified by defining exclusion areas where the image is not to be placed. Printing mediums with three-dimensional shapes may have areas where printing is difficult or impossible, such as the connection point between the cup body and the handle of a mug. In such cases, the printable area can be modified by defining exclusion areas for the entire printing medium, making it easy to modify the printable area. Note that such exclusion areas can also be set by factors other than the shape of the printing medium. For example, there may be areas that cannot be printed because the printable area of ​​the printing device that prints on the printing medium is smaller than the printing medium. If the occurrence of such a situation is known in advance, areas that cannot be printed due to factors on the printing device side may be set as exclusion areas.

[0104] (9) In the configurations of (1) to (8) above, when a deformation of at least a part of the printing medium is specified by the change specification unit, the excluded area may be changed in accordance with the deformation. This makes it easy to expand the excluded area, for example, when the size of the handle of a mug is changed, because it can be assumed that the excluded area will be created by the shape of at least a part of the printing medium.

[0105] (10) In the configurations of (1) to (9) above, the image placement unit may place the image in the modified printable area when the printable area is modified by the modification unit. In this way, when the printable area is modified, the position of the image is also modified, making it easier to specify the position of the image.

[0106] (11) In the configurations of (1) to (9) above, the image placement unit may, when the printable area is changed by the modification unit, modify the image so that it fits within the modified printable area without changing the image placement. This avoids changing the position of the image.

[0107] (12) In the configurations of (1) to (9) above, the image placement unit may include an image editing unit that performs at least one of the following actions on the image in the modified printable area according to external instructions: moving, resizing, rotating, cropping, splitting, and additional placement. This makes it easy to place an image within the modified printable area or to modify it so that it fits within the printable area. If the shape of the print medium is changed, the configuration may be such that the placement in the image placement area is cleared and the user is prompted to perform the image placement process again.

[0108] (13) In the configurations of (1) to (12) above, a display unit for displaying the rendering results may be provided. This makes it easy to check the rendering results. The display unit is not limited to a flat display, but may also be a 3D display or a hologram or other device that displays in stereoscopic vision. Instead of displaying on the display unit, the results may be printed and output.

[0109] (14) The present disclosure can also be implemented as an image printing system. This image printing system includes an image acquisition unit that acquires image data of an image to be printed on a printing medium having a three-dimensional shape; a change designation unit that specifies a change to the shape of at least a part of the printing medium; a change implementation unit that changes the shape of the printing medium and the printable area on the printing medium in accordance with the specified change; an image placement unit that places the image in the printable area after the change; a rendering unit that renders the printing medium after the shape has been changed and the image data placed in the printable area; a display unit that displays the rendering result; and a printing unit that prints the image on the printing medium using the image data. In this way, a preview obtained by rendering can be confirmed and the image can be actually printed on the printing medium. Accordingly, the printed medium can be acquired after confirming the result of printing the image on a printing medium having a three-dimensional shape by previewing.

[0110] For printing such images, a 3D printing device capable of directly printing on a three-dimensional medium can be used, or a transfer printing device that forms an image on the surface of the medium via transfer paper or a transfer sheet can be used. In the former case, UV ink is used to prevent the ink from peeling off the medium, and the ink is cured by irradiation with ultraviolet light, which has been put into practical use. Such printing can be achieved on a three-dimensional medium using a 5-axis ink ejection device that can control three-dimensional movement and the angle of the head. Furthermore, when forming the medium with a 3D printer, a configuration in which colored filament is melted and printed on the surface of the medium can also be adopted. Note that three-dimensional shapes include flat mediums such as thick acrylic plates. If the medium is made of a material that does not have high heat resistance, such as synthetic resins such as acrylic, transfer printing that involves heat transfer should be avoided, and printing using UV ink should be adopted instead.

[0111] (15) The Disclosure can also be implemented as an image processing program that generates a rendered image of a print medium on which an image is printed. This image processing program generates a rendered image of a print medium on which an image is printed, and includes a function to acquire image data of an image to be printed on a print medium having a three-dimensional shape, a function to accept a specification for changing the shape of at least a part of the print medium, a function to change the shape of the print medium and the printable area on the print medium in accordance with the specified change, a function to place the image in the changed printable area, and a function to render the print medium and the image data placed in the printable area after the shape has been changed, all implemented by a computer. In this way, the user can change at least a part of the shape of the print medium through computer control, place an image in the printable area that is changed in accordance with the shape change, and immediately check how it will look in that case by rendering it.

[0112] (16) In each of the above embodiments, some of the configurations implemented by hardware may be replaced with software. At least some of the configurations implemented by software can also be implemented by discrete circuit configurations. Furthermore, if some or all of the functions of this disclosure are implemented by software, the software (computer program) may be provided in the form of being stored on a computer-readable recording medium. "Computer-readable recording medium" is not limited to portable recording media such as flexible disks and CD-ROMs, but also includes various internal storage devices in a computer such as RAM and ROM, and external storage devices fixed to a computer such as hard disks. In other words, "computer-readable recording medium" has a broad meaning that includes any recording medium on which data packets can be fixed rather than temporary.

[0113] This disclosure is not limited to the embodiments described above, and can be implemented in various configurations without departing from its spirit. For example, the technical features in the embodiments corresponding to the technical features in each form described in the summary of the invention can be replaced or combined as appropriate in order to solve some or all of the above-described problems, or to achieve some or all of the above-described effects. Furthermore, if a technical feature is not described as essential in this specification, it can be deleted as appropriate. [Explanation of symbols]

[0114] 20...Color management system, 30...Image placement unit, 40...Change execution unit, 50...Print media database, 60...Change specification unit, 70...Editing operation unit, 80...Edited image display unit, 100...Image processing unit, 111...Image editing unit, 121...Rendering execution unit, 131...First storage unit, 132...Second storage unit, 135...Memory, 141...Communication unit, 151...Rendered image display unit, 190...Site, 200...Printing device, 300...Image printing system, 400...Printer

Claims

1. An image acquisition unit that acquires image data of an image to be printed on a printing medium having a three-dimensional shape, A change designation unit that specifies a change in the shape of at least a part of the printed medium, A modification implementation unit that modifies the shape of the printing medium and the printable area on the printing medium in accordance with the specified modification, An image placement unit for placing the image in the modified printable area, A rendering unit that renders the print medium after its shape has been changed and the image data placed in the printable area, Equipped with an image processing device.

2. An image processing apparatus according to claim 1, The change specification unit accepts a specification for a shape change using parameters for each axis of the three-dimensional shape, and the parameters are: [1] A numerical value that specifies the rate, ratio, or amount of enlargement or reduction. [2] A numerical value indicating the dimensions after enlargement or reduction in a predetermined unit. [3] Multiple selectable indicators to show the degree of change, An image processing device that is one of the following.

3. The image processing apparatus according to claim 2, wherein, when the parameter is the index, each of the index sets at least one of the amount of change in size for each axis of the three-dimensional shape and the amount of translation for each axis.

4. An image processing apparatus according to claim 2, The aforementioned print medium is composed of multiple objects, The aforementioned change designation section is: When the parameter is the index, the degree of change of each of the multiple objects is pre-associated and stored with each of the index. When one of the aforementioned indicators is selected, the degree of change associated with the selected indicator shall be the designation of the change for each of the aforementioned objects. Image processing device.

5. The image processing apparatus according to any one of claims 1 to 4, wherein the change specification unit can specify a change in the shape of a part of the printing medium separately from the shape of other parts of the printing medium.

6. The image processing apparatus according to any one of claims 1 to 4, wherein the modification implementation unit changes the shape of the image placement area, which is an area within the printable area where the image is preferentially placed, in accordance with the change in the printable area.

7. An image processing apparatus according to claim 6, The aforementioned modification implementation unit shall, (4) The shape of the image placement area is within the printable area. (5) A shape that is linked to a change in the shape of the printing medium, (6) A modified shape in which the image placement area exceeds the printable area. An image processing device that changes to one of the following.

8. An image processing apparatus according to any one of claims 1 to 4, The aforementioned printing medium includes an exclusion area where the image is not to be placed. The modification implementation unit modifies the printable area, taking into consideration the excluded area. Image processing device.

9. The image processing apparatus according to claim 8, wherein when a deformation of at least a part of the printing medium is specified by the deformation specification unit, the setting of the exclusion area is changed in accordance with the deformation.

10. The image processing apparatus according to any one of claims 1 to 4, wherein the image placement unit places the image in the modified printable area when the printable area is modified by the modification unit.

11. The image processing apparatus according to any one of claims 1 to 4, wherein when the printable area is changed by the modification implementation unit, the image arrangement unit does not change the arrangement of the image, but modifies the image so that it fits within the modified printable area.

12. The image processing apparatus according to any one of claims 1 to 4, wherein the image placement unit comprises an image editing unit that performs at least one of the following actions on the modified printable area: moving, resizing, rotating, trimming, dividing, and additional placement of the image in accordance with an external instruction.

13. The image processing apparatus according to any one of claims 1 to 4, further comprising a display unit for displaying the rendering results.

14. An image acquisition unit that acquires image data of an image to be printed on a printing medium having a three-dimensional shape, A change designation unit that specifies a change in the shape of at least a part of the printed medium, A modification implementation unit that modifies the shape of the printing medium and the printable area on the printing medium in accordance with the specified modification, An image placement unit for placing the image in the modified printable area, A rendering unit that renders the print medium after its shape has been changed and the image data placed in the printable area, A display unit that displays the rendering results, A printing unit that prints the image onto the printing medium using the image data, An image printing system equipped with this system.

15. An image processing program that generates a rendered image of a printed medium on which an image has been printed, A function to acquire image data of an image to be printed on a printing medium having a three-dimensional shape, A function that accepts the specification of a change in the shape of at least a part of the aforementioned printing medium, A function to change the shape of the printing medium and the printable area on the printing medium in accordance with the specified changes, A function to place the image in the modified printable area, A function for rendering the print medium after its shape has been modified and the image data placed in the printable area, An image processing program that implements this using a computer.

Images