IMAGE PROCESSING METHOD, IMAGE PROCESSING DEVICE, GAME DEVICE, AND PROGRAM

Abstract

To convert an image into a picturesque image by achieving both picturesque appearance and clearness of an edge of a predetermined drawn object.SOLUTION: In an image processing method according to the preset invention, a computer performs the following: an acquisition step S10 of acquiring distance information indicating a distance from a camera and image information indicating color information of each pixel; a determination step S20 of determining settings of a modulation filter of each pixel for modulating an image into a picturesque image by converting the color information of each pixel based on the distance information of each pixel; and a conversion step S30 of converting the color information of each pixel based on the determined modulation filter setting of each pixel.SELECTED DRAWING: Figure 6

Description

[Technical field]

[0001] The present invention relates to an image processing method, an image processing device, a game device, and a program. [Background technology]

[0002] Well-known modulation filters that modulate an image to a painterly style include the Symmetric Nearest Neighbor (SNN) filter, the median filter, the Kuwahara filter, etc. These modulation filters convert the color information of each pixel based on the color information of other pixels (e.g., other pixels surrounding each pixel), thereby modulating the entire image to a painterly style.

[0003] Patent documents 1 and 2 disclose a technology for generating an image with fog (fog, clouds, mist, mist, dust, smoke, tornadoes, dew, etc.) on the original image, in which the alpha value, a parameter that controls the fog density, is changed according to the distance from the viewpoint (virtual camera), and the fog density becomes denser the farther the distance from the viewpoint becomes. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2006-318386 A [Patent Document 2] JP 2006-318389 A Summary of the Invention [Problem to be solved by the invention]

[0005] In general processing using the modulation filter that modulates the image to a painterly style, the color information of all pixels is converted uniformly in the same manner (e.g., degree of blurring, degree of smearing). The degree of conversion is realized, for example, by adjusting the number of other pixels referenced when converting the color information, the weight of the color information of other pixels, etc.

[0006] In the case of the general processing described above, although it is possible to generate an image that has a painterly overall appearance, it is not possible to generate an image that has both clear edges of a given drawing subject and a painterly appearance. Patent documents 1 and 2 do not describe or suggest the problem and a means for solving the problem regarding a modulation filter that modulates an image to have a painterly appearance.

[0007] An object of the present invention is to render an image while achieving both clarity of edges of a rendering subject and a painterly appearance. [Means for solving the problem]

[0008] According to the present invention, The computer an acquisition step of acquiring image information indicating distance information and color information for each pixel from the camera; determining, for each pixel, a setting of a modulation filter that converts the color information of each pixel to modulate the image into a painterly style based on the distance information of each pixel; a conversion step of converting the color information for each pixel based on the determined settings of the modulation filter for each pixel; An image processing method is provided that performs the following:

[0009] Further, according to the present invention, an acquisition unit that acquires image information indicating distance information indicating a distance from the camera and color information for each pixel; a determination unit that determines, for each pixel, a setting of a modulation filter that converts the color information of each pixel and modulates the image into a painterly style based on the distance information of each pixel; a conversion unit that converts the color information for each pixel based on the determined setting of the modulation filter for each pixel; An image processing apparatus is provided having the following:

[0010] Further, according to the present invention, Computer, An acquisition means for acquiring image information indicating distance information indicating a distance from the camera and color information for each pixel; a determining means for determining, for each pixel, a setting of a modulation filter for converting the color information of each pixel and modulating the image into a painterly style based on the distance information of each pixel; a conversion means for converting the color information for each pixel based on the determined setting of the modulation filter for each pixel; A program is provided to function as a

[0011] Further, according to the present invention, an input receiving unit that receives an operation input from a player; a player character control unit that manages the position and orientation of the player character in the virtual three-dimensional space; a non-player character control unit that manages positions and orientations of non-player characters within the virtual three-dimensional space; a camera control unit that manages a position and an orientation of a virtual camera in the virtual three-dimensional space; a drawing unit that draws the virtual three-dimensional space captured by the virtual camera as an image on a two-dimensional plane based on positions and orientations of the player character, the non-player character, and the virtual camera in the virtual three-dimensional space; a display control unit that causes an output device to display the two-dimensional image; having The drawing unit is an acquisition unit that acquires image information indicating distance information and color information for each pixel from the virtual camera; a determination unit that determines, for each pixel, a setting of a modulation filter that converts the color information of each pixel and modulates the image into a painterly style based on the distance information of each pixel; a conversion unit that converts the color information for each pixel based on the determined setting of the modulation filter for each pixel; A gaming device having the following is provided. Effect of the Invention

[0012] According to the present invention, it is possible to render an image while achieving both clarity of edges of a given rendering target and a painterly appearance. [Brief description of the drawings]

[0013] [Figure 1] FIG. 1 is a diagram illustrating an example of a hardware configuration of an image processing apparatus according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a functional block diagram of an image processing apparatus according to an embodiment of the present invention; [Diagram 3] FIG. 4 is a diagram for explaining a conversion process according to the present embodiment. [Figure 4] FIG. 4 is a diagram illustrating an example of reference area information according to the present embodiment. [Diagram 5] FIG. 11 is a diagram showing the contents defined in an example of reference area information according to the present embodiment. [Figure 6] 5 is a flowchart showing an example of a processing flow of the image processing device of the present embodiment. [Figure 7] 5 is a flowchart showing an example of a processing flow of the image processing device of the present embodiment. [Figure 8] 5 is a flowchart showing an example of a processing flow of the image processing device of the present embodiment. [Figure 9] FIG. 2 is an example of a functional block diagram of the game device according to the present embodiment. [Figure 10] FIG. 2 is a functional block diagram of a drawing unit according to an embodiment of the present invention; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] <First embodiment> First, an overview of the processing of the image processing device of this embodiment will be described. When the image processing device of this embodiment acquires image information indicating "distance information indicating the distance from the camera" and "color information" for each pixel, it determines the setting of a "modulation filter that converts the color information of each pixel to modulate the image into a painterly style" for each pixel based on the distance information of each pixel. The setting determined here is the degree of conversion of the color information of each pixel (e.g., degree of blurring, degree of bleeding). By adjusting this setting, the degree of modulation of the image is adjusted. Then, the image processing device converts the color information for each pixel based on the determined setting of the modulation filter for each pixel.

[0015] In this way, according to the image processing device of this embodiment, the degree of conversion of the color information of each pixel (e.g., degree of blurring, degree of bleeding) can be set for each pixel, and the color information of each pixel can be converted according to the contents set for each pixel. Therefore, according to the image processing device of this embodiment, the degree of conversion can be partially changed within one image. In other words, according to the image processing device of this embodiment, it is possible to generate one image in which there are a mixture of parts with a strong degree of conversion, parts with a weak degree of conversion, and parts that have not been converted. As a result, according to the image processing device of this embodiment, it is possible to modulate an image into a painterly style by achieving both clarity of edges of a specified drawing object and a painterly feel.

[0016] Next, the configuration of the image processing device of this embodiment will be described. First, an example of the hardware configuration of the image processing device will be described. Each functional unit of the image processing device of this embodiment is realized by any combination of hardware and software, centering on the CPU (Central Processing Unit) of any computer, memory, programs loaded into the memory, a storage unit such as a hard disk that stores the programs (programs that are stored before the device is shipped, as well as programs downloaded from storage media such as CDs (Compact Discs) or servers on the Internet, can be stored), and a network connection interface. Those skilled in the art will understand that there are various variations in the realization method and device.

[0017] Fig. 1 is a block diagram illustrating a hardware configuration of an image processing device according to this embodiment. As shown in Fig. 1, the image processing device has a processor 1A, a memory 2A, an input / output interface 3A, a peripheral circuit 4A, and a bus 5A. The peripheral circuit 4A includes various modules. The image processing device does not have to have the peripheral circuit 4A. The image processing device may be composed of a plurality of devices that are physically and / or logically separated. In this case, each of the plurality of devices can have the above hardware configuration.

[0018] The bus 5A is a data transmission path for the processor 1A, the memory 2A, the peripheral circuit 4A, and the input / output interface 3A to transmit and receive data to each other. The processor 1A is an arithmetic processing device such as a CPU or a GPU (Graphics Processing Unit). The memory 2A is a memory such as a RAM (Random Access Memory) or a ROM (Read Only Memory). The input / output interface 3A includes an interface for acquiring information from an input device, an external device, an external server, an external sensor, etc., and an interface for outputting information to an output device, an external device, an external server, etc. Examples of the input device include a controller, a keyboard, a mouse, a microphone, a pointing device, a touch panel, a physical button, a camera, etc. Examples of the output device include a display, a speaker, a printer, a mailer, etc. The processor 1A can issue commands to each module and perform calculations based on the results of those calculations.

[0019] Next, a functional configuration of the image processing device of this embodiment will be described. An example of a functional block diagram of the image processing device 10 is shown in Fig. 2. As shown in the figure, the image processing device 10 has an acquisition unit 11, a determination unit 12, and a conversion unit 13.

[0020] The acquisition unit 11 acquires image information, which is information about an image and indicates color information and distance information for each pixel.

[0021] The image is, for example, an image showing one scene in a game. More specifically, the image is an image obtained by capturing (taking a picture) of a plurality of depiction objects, the position information of which is managed in a three-dimensional space, from a virtual camera installed at a predetermined viewpoint, and projecting the captured picture into a two-dimensional perspective. Examples of depiction objects include, but are not limited to, characters appearing in a game, and objects that form part of the background (plants such as trees, structures such as houses, etc.). For example, a depth buffer method (Z-buffer method) is used as a method for removing hidden surfaces when depicting a two-dimensional image.

[0022] The color information indicates the color of the object represented by each pixel, and the distance information indicates the distance from the camera (viewpoint) to the object represented by each pixel.

[0023] The determination unit 12 determines the settings of a modulation filter (hereinafter, sometimes simply referred to as a "modulation filter") that converts color information of each pixel to modulate the image into a painterly style, for each pixel, based on the distance information of each pixel.

[0024] The modulation filter is a filter that converts the color information of each pixel based on the color information of other pixels to convert the image into a painterly style. Examples of the modulation filter include, but are not limited to, a Symmetric Nearest Neighbor (SNN) filter, a median filter, and a Kuwahara filter.

[0025] An overview of the process using the modulation filter will be explained with reference to Fig. 3. In Fig. 3, a number is shown. Each pixel is identified by a number assigned to each row and an alphabet assigned to each column. For example, a pixel located in a row assigned with 1 and in a column assigned with B is called pixel 1B.

[0026] In the process using the modulation filter, a reference area is set so as to include the pixel to be processed. The reference area may include only the pixel to be processed, or may include the pixel to be processed and other pixels (e.g., pixels located around the pixel to be processed). In the example of FIG. 3, pixel 3C is the pixel to be processed, and the shaded area is the reference area. The reference area shown in the figure is a square of a predetermined size with the pixel to be processed at the center. The reference area includes pixels 2B to 2D, pixels 3B to 3D, and pixels 4B to 4D. In the process using the modulation filter, color information of the pixel to be processed is converted based on color information of pixels in the reference area. There are various ways of conversion, and there is no particular restriction in this embodiment. Note that the reference area shown in the figure is a square, but the shape of the reference area is not limited to this.

[0027] Next, the setting determined for each pixel by the determination unit 12 will be described. The setting determined by the determination unit 12 is the degree of conversion of color information of each pixel (e.g., degree of blurring, degree of bleeding). By adjusting this setting, the degree of modulation of the image is adjusted.

[0028] The parameter for adjusting the degree of conversion of the color information of each pixel is, for example, the size of the reference area. The larger the reference area, the greater the degree of blurring and bleeding, and the greater the degree of painterly texture. On the other hand, the smaller the reference area, the less the degree of blurring and bleeding, and the less the degree of painterly texture.

[0029] The determination unit 12 determines the size of the reference area to be set for each pixel based on the distance information of each pixel and a predetermined rule. The predetermined rule may be a calculation formula for calculating the size of the reference area from the distance indicated by the distance information, a table defining the relationship between the distance indicated by the distance information and the size of the reference area, or other rules.

[0030] The size of the reference area can be expressed, for example, by the number of pixels. In this case, the size of the reference area determined by the determination unit 12 is 1 pixel or more and M pixels or less. The value of M is an arbitrary value greater than 1.

[0031] Here, as an example of processing by the determination unit 12, we will explain a process in which the determination unit 12 determines the size of the reference area for each pixel based on reference area information that divides the distance from the camera into N stages (N is an integer greater than or equal to 2) and defines the size of the reference area for each stage, and distance information.

[0032] 4 is an example of reference area information that defines the size of the reference area for each stage of distance from the camera. The reference area information shown in the figure has a stage number, a distance range, a threshold, and a calculation formula.

[0033] The stage number is a number given to each of the multiple stages, and is information for identifying the multiple stages. N=5, and the distance from the camera is divided into five stages. The distance range indicates the distance range of each of the multiple stages, for example, with an upper limit value and a lower limit value. The threshold value indicates the size of the reference area when the distance is maximum in each stage. The size of the reference area is indicated by the number of pixels. The calculation formula is a formula for calculating the size L of the reference area. The operator determines the value of N, the distance range of each stage, the threshold value, and the calculation formula, and registers the reference area information including the determined contents in the image processing device 10. Note that the specific examples of the value of N, the distance range of each stage, the threshold value, and the calculation formula shown in the figure are merely examples, and are not limited to these.

[0034] Figure 5 shows the relationship between the distance from the camera and the size of the reference area defined in the reference area information in Figure 4. In the figure, the vertical axis indicates the size of the reference area, and the horizontal axis indicates the distance from the camera.

[0035] As can be seen from Fig. 4, the threshold value for stage 1 is 9 pixels, so the size of the reference area when the distance is maximum in stage 1 (the boundary between stages 1 and 2) is 9 pixels, as shown in Fig. 5. And as can be seen from Fig. 4, the calculation formula for the size L of the reference area in stage 1 is L = (threshold), so the size L of the reference area in stage 1 is constant at 9 pixels, as shown in Fig. 5.

[0036] Similarly, as shown in FIG. 4, the threshold value in stage 2 is 1 pixel, so that the size of the reference area when the distance in stage 2 is maximum (the boundary between stage 2 and stage 3) is 1 pixel, as shown in FIG. 5. As shown in FIG. 4, the calculation formula for the size L of the reference area in stage 2 is L=f1(d). d is the distance from the camera indicated by the distance information. The function f1(d) is a function generated using any of the interpolation methods, such as Elminate interpolation, linear interpolation, Lagrange interpolation, and spline interpolation. The multiple values ​​(multiple pairs of d and L) given to calculate the function using these interpolation methods are determined by the operator. FIG. 5 shows a curve of a function in which the size of the reference area becomes smaller as the distance from the camera increases.

[0037] Similarly, as shown in Fig. 4, the threshold value for stage 3 is 1 pixel, so the size of the reference area when the distance is maximum in stage 3 (the boundary between stages 3 and 4) is 1 pixel, as shown in Fig. 5. And as shown in Fig. 4, the calculation formula for the size L of the reference area in stage 3 is L = (threshold value), so the size L of the reference area in stage 3 is constant at 1 pixel, as shown in Fig. 5.

[0038] Similarly, from FIG. 4, the threshold value for stage 4 is 25 pixels, so that the size of the reference area when the distance is maximum in stage 4 (the boundary between stage 4 and stage 5) is 25 pixels, as shown in FIG. 5. From FIG. 4, the calculation formula for the size L of the reference area in stage 4 is L=f2(d). The function f2(d) is generated using any of the interpolation methods, such as Elminate interpolation, linear interpolation, Lagrange interpolation, and spline interpolation. The multiple values ​​(multiple pairs of d and L) given to calculate the function using these interpolation methods are determined by the operator. FIG. 5 shows the curve of a function in which the size of the reference area increases as the distance from the camera increases.

[0039] Similarly, from Fig. 4, the threshold value for stage 5 is 25 pixels, so the size of the reference area when the distance is maximum in stage 5 is 25 pixels, as shown in Fig. 5. And from Fig. 4, the calculation formula for the size L of the reference area in stage 5 is L = (threshold), so the size L of the reference area in stage 5 is constant at 25 pixels, as shown in Fig. 5.

[0040] Returning to FIG. 2, the conversion unit 13 converts color information for each pixel based on the setting of the modulation filter for each pixel (size of the reference area) determined by the determination unit 12. Specifically, the conversion unit 13 sets a reference area of ​​a size determined by the determination unit 12 for each pixel, and converts color information of each pixel based on color information of other pixels in the reference area. For example, the conversion unit 13 sets the reference area of ​​each pixel so that the relative positional relationship with each pixel is a predetermined positional relationship. The relative positional relationship is, for example, "each pixel is the center of the reference area", but is not limited to this. The specific processing for converting color information is not particularly limited, and processing such as a Symmetric Nearest Neighbor (SNN) filter, a median filter, and a Kuwahara filter can be used.

[0041] When the size of the reference area is set to "1 pixel", the conversion unit 13 does not convert the color information of the pixel. In other words, the color information corresponding to the pixel acquired by the acquisition unit 11 remains the color information of the pixel even after the image modulation process.

[0042] Next, an example of the flow of processing by the image processing device 10 of this embodiment will be described with reference to the flowchart of FIG.

[0043] First, the image processing device 10 acquires image information indicating distance information indicating the distance from the camera and color information for each pixel (acquisition step S10). The image information is information about an image showing, for example, a scene in a game. The details of the distance information, color information, and image are as described above. For example, the image processing device 10 can acquire image information about an image showing a scene in the game during the game before the image is output to a display or the like.

[0044] Next, the image processing device 10 determines, for each pixel, the setting of a modulation filter that converts the color information of each pixel to modulate the image into a painterly style, based on the distance information of each pixel (determination step S20). Specifically, as shown in the flowchart of FIG. 7, when the image processing device 10 acquires the distance information of each pixel (S21), it determines the size of a reference area for each pixel based on the distance information (S22). For example, the image processing device 10 refers to the reference area information as shown in FIG. 4, and specifies which stage of distance range the distance indicated by the distance information falls within. Then, the image processing device 10 determines the size L of the reference area based on the calculation formula for the specified stage. The image processing device 10 can perform this processing for each pixel.

[0045] Returning to FIG. 6, after the determination step S20, the image processing device 10 converts color information for each pixel based on the setting of the modulation filter for each pixel (the size of the reference area) determined in the determination step S20 (conversion step S30). Specifically, as shown in the flowchart of FIG. 8, the image processing device 10 sets a reference area for each pixel based on the setting of the modulation filter for each pixel (the size of the reference area) determined in the determination step S20 (S31). For example, the image processing device 10 sets the reference area for each pixel so that the relative positional relationship with each pixel is a predetermined positional relationship. The relative positional relationship is, for example, "each pixel is the center of the reference area", but is not limited thereto. Next, the image processing device 10 converts color information for each pixel based on color information of other images within the set reference area. The specific processing for converting color information is not particularly limited, and processing such as a Symmetric Nearest Neighbor (SNN) filter, a median filter, and a Kuwahara filter can be used.

[0046] According to the image processing device 10 described above, the degree of conversion of color information of each pixel (e.g., degree of blurring, degree of bleeding) can be set for each pixel, and the color information of each pixel can be converted according to the contents set for each pixel. Therefore, according to the image processing device 10, the degree of conversion can be partially changed within one image. In other words, according to the image processing device 10, it is possible to generate one image in which there are a mixture of parts with a strong degree of conversion, parts with a weak degree of conversion, and parts that have not been converted. As a result, according to the image processing device of this embodiment, it is possible to convert an image into a painting-like style by achieving both clarity of edges of a specified drawing object and a painting-like feel.

[0047] Furthermore, according to the image processing device 10 that performs modulation processing based on reference area information as shown in FIG. 4 and FIG. 5, it is possible to determine the size of the reference area so as to have the following characteristics, and perform modulation processing of the image.

[0048] (Feature 1) When the distance indicated by the distance information is within the distance range of stage 3 (D1 to D2), the size of each pixel (1 pixel) is determined as the size of the reference area. In other words, for drawing targets whose distance from the camera is within this distance range, no painterly modulation is performed.

[0049] (Feature 2) When the distance indicated by the distance information is within the distance range of stage 4 (larger than D2), the larger the distance indicated by the distance information, the larger the reference area is determined. In other words, for a drawing target whose distance from the camera is within the distance range, the larger the distance from the camera, the more the modulation is made to have a more painterly feel.

[0050] (Feature 3) When the distance indicated by the distance information is within the distance range of stage 2 (smaller than D1), the smaller the distance indicated by the distance information, the larger the reference area is determined. In other words, for a drawing target whose distance from the camera is within the distance range, the smaller the distance from the camera, the more painterly the modulation is performed.

[0051] For example, the distance range of stage 3 is a distance range in which the image is in focus. The distance range of stage 4 is a distance range in which the image is out of focus because the distance from the viewpoint is large, and the image becomes blurred as the distance increases. The distance range of stage 2 is a distance range in which the image is out of focus because the distance from the viewpoint is small, and the image becomes blurred as the distance decreases. Based on the reference area information as shown in Figs. 4 and 5, the image processing device 10 can modulate the image in a manner similar to how such actual objects are seen. As a result, a more realistic image that matches the actual situation can be generated.

[0052] Here, a modified example of this embodiment will be described. In the above-described embodiment, the size of the reference area is used as a parameter for adjusting the degree of conversion of the color information of each pixel. Instead of the size of the reference area, the color difference between each pixel may be used as the parameter. The larger the color difference, the stronger the degree of blurring and bleeding, and the greater the degree of painting. On the other hand, the smaller the color difference, the weaker the degree of blurring and bleeding, and the less the degree of painting. In this modified example, the same action and effect as this embodiment is realized.

[0053] <Second embodiment> The game device of this embodiment includes the functional units of the image processing device 10 described in the first embodiment. Fig. 9 shows an example of a functional block diagram of the game device 100. As shown in the figure, the game device 100 has an input receiving unit 1, a player character control unit 2, a non-player character control unit 3, a camera control unit 4, a drawing unit 5, and a display control unit 6. The drawing unit 5 has the functional units of the image processing device 10, which will be described in detail later.

[0054] In the game provided by the game device 100 of this embodiment, player characters and non-player characters exist in a virtual three-dimensional space. The player characters and non-player characters move within the virtual three-dimensional space and fight or converse with other characters. The virtual three-dimensional space is represented by a Cartesian coordinate system defined by mutually orthogonal X-axis, Y-axis, and Z-axis. Hereinafter, the coordinates of the Cartesian coordinate system may be referred to as "coordinates of the virtual three-dimensional space."

[0055] The input reception unit 1 receives operation input from a player. The input reception unit 1 can receive operation input from a player via any input device such as a game controller, a pointing device, a touch panel, a physical button, a mouse, a keyboard, a microphone, a camera, etc. The specific content of the operation input is a design item determined for each game. The player performs an operation input to make the player character perform a predetermined action (e.g., movement), for example.

[0056] The player character control unit 2 controls the player character based on an operational input from the player. For example, the player character control unit 2 controls the current position of the player character in the virtual three-dimensional space by expressing the coordinates (x p , y p , z p The player character control unit 2 also manages the orientation of the player character (e.g., the direction the face is facing) based on, for example, the rotation angles (θ xp , θ yp , θ xp The player character control unit 2 then determines a new position and orientation of the player character in the virtual three-dimensional space based on the operation input from the player, and manages the position and orientation of the player character using the coordinates (x p , y p , z p ) and rotation angle (θ xp , θ yp , θ xp ) to update the

[0057] The non-player character control unit 3 controls the non-player characters. For example, the non-player character control unit 3 controls the current position of the non-player character in the virtual three-dimensional space by the coordinates (x np , y np , z np The non-player character control unit 3 also manages the orientation of the non-player character (e.g., the direction in which the face is facing) by, for example, the rotation angles (θ xnp , θynp , θ xnp The non-player character control unit 3 then determines the new position and orientation of the non-player character in the virtual three-dimensional space by any means (based on a predetermined algorithm) and manages the position and orientation of the non-player character by the above coordinates (x np , y np , z np ) and rotation angle (θ xnp , θ ynp , θ xnp ) to update the

[0058] The camera control unit 4 controls a virtual camera located in the virtual three-dimensional space based on an operational input from the player or based on a movement of a player character in the virtual three-dimensional space caused by the operational input from the player. For example, the camera control unit 4 controls the current position of the virtual camera in the virtual three-dimensional space based on the coordinates (x c , y c , z c The camera control unit 4 also manages the orientation of the virtual camera (e.g., the direction in which the optical axis faces) based on, for example, the rotation angles (θ xc , θ yc , θ xc The camera control unit 4 determines a new position and direction of the virtual camera in the virtual three-dimensional space based on the operation input from the player or based on the movement of the player character in the virtual three-dimensional space caused by the operation input from the player, and manages the position and direction of the virtual camera in the virtual three-dimensional space based on the above coordinates (x c , y c , z c ) and rotation angle (θ xc , θ yc , θ xc ) to update the

[0059] There is no particular limitation on the method for determining the position and orientation of the virtual camera, and any technique can be adopted. For example, the virtual camera may be set to capture the image of the player character from a position behind the player character and at a predetermined distance away. In this case, the camera control unit 4 sets the coordinates (x p , y p , z p) and the rotation angle (θ xp , θ yp , θ xp ) and changes the coordinates (x c , y c , z c ) and the rotation angle (θ xc , θ yc , θ xc ) can be changed. In the above example, the shooting direction of the virtual camera is defined based on the player character, but it may be defined based on an object different from the player character (e.g., a non-player character, an arbitrary object, etc.). Also, the moving path and rotation direction of the virtual camera may be determined in advance. Then, the camera control unit 4 may change the current position and orientation of the virtual camera according to the latitude path and rotation direction.

[0060] The drawing unit 5 draws the three-dimensional space captured by the virtual camera as a two-dimensional image based on the current positions and orientations of the player characters managed by the player character control unit 2, the current positions and orientations of the non-player characters managed by the non-player character control unit 3, and the current position and orientation of the virtual camera managed by the camera control unit 4. Note that in the virtual three-dimensional space, there may be objects (e.g., buildings, plants, etc.) whose positions do not change, and the coordinates indicating the positions of each of the objects in the virtual three-dimensional space may be registered in advance. The drawing unit 5 may further use information about the objects to draw the two-dimensional image.

[0061] As shown in FIG. 10, the drawing unit 5 has an acquisition unit 11, a determination unit 12, a conversion unit 13, an information acquisition unit 14, a two-dimensional image drawing unit 15, and an output unit 16.

[0062] The information acquisition unit 14 acquires information (e.g., coordinates (x p , y p , z p ) and rotation angle (θ xp , θ yp , θ xp)), information indicating the current position and direction of the non-player character managed by the non-player character control unit 3 (e.g., coordinates (x np , y np , z np ) and rotation angle (θ xnp , θ ynp , θ xnp )), and information indicating the current position and orientation of the virtual camera managed by the camera control unit 4 (e.g., target (x c , y c , z c ) and rotation angle (θ xc , θ yc , θ xc The information acquisition unit 14 may further acquire information indicating the position and orientation of an object whose position does not change. The information acquisition unit 14 may also acquire information indicating the results of various game processes.

[0063] The two-dimensional image drawing unit 15 draws the three-dimensional space captured by the virtual camera as a two-dimensional plane image based on the information acquired by the information acquisition unit 14. There is no particular restriction on the method of realizing the drawing, and any technology can be adopted. For example, the two-dimensional image drawing unit 15 may execute pre-processing such as coordinate transformation (world coordinate transformation, camera coordinate transformation), clipping processing, perspective transformation, and other geometry processing. Next, the two-dimensional image drawing unit 15 may generate drawing data based on the result of the pre-processing. The drawing data includes color data, texture coordinates, coordinates of vertices of primitive surfaces, normal vectors, α values, and the like. Then, the two-dimensional image drawing unit 15 generates image information related to an image in which a player character or the like after perspective transformation is drawn based on the drawing data. The image information indicates distance information indicating the distance from the virtual camera and color information for each pixel. The two-dimensional image drawing unit 15 may use hidden surface removal processing using a depth buffer method (Z-buffer method) or the like.

[0064] The acquisition unit 11, the determination unit 12, and the conversion unit 13 execute the processes described in the first embodiment based on the image information generated by the two-dimensional image rendering unit 15.

[0065] The output unit 16 outputs image information indicating the image that has been generated by the two-dimensional image drawing unit 15 and has been subjected to modulation processing by the image processing device 10 (hereinafter, referred to as "modulated image information").

[0066] Returning to FIG. 9, the display control unit 6 causes the output device to display an image indicated by the image information after the modulation process output by the output unit 16 of the drawing unit 5. Examples of the output device include a display and a projection device, but are not limited to these. The output device may be an external device that is physically and / or logically separate from the game device 100 and connected to the game device 100. Alternatively, the output device may be physically and logically integrated with the game device 100.

[0067] An example of the hardware configuration of the game device 100 is similar to the example of the hardware configuration of the image processing device 10 described in the first embodiment.

[0068] According to the game device 100 described above, the same operational effects as those of the image processing device 10 of the first embodiment are achieved.

[0069] As is clear from the above description, the game device 100 does not always render a static space that does not change, but renders scenes in a three-dimensional space where the situation changes from moment to moment due to operational input from the player and control of the CPU. Since the situation changes from moment to moment, the relative positional relationship (distance and direction) between the virtual camera and the rendering target (player character, etc.) also changes dynamically. In such an environment where the situation changes from moment to moment, the game device 100 can render the "same rendering target" in a manner appropriate to the situation, such as sometimes rendering it with edges and sometimes rendering it in a painterly manner.

[0070] Below, examples of reference forms are given. 1. The computer: an acquisition step of acquiring image information indicating distance information and color information for each pixel from the camera; determining, for each pixel, a setting of a modulation filter that converts the color information of each pixel to modulate the image into a painterly style based on the distance information of each pixel; a conversion step of converting the color information for each pixel based on the determined settings of the modulation filter for each pixel; An image processing method that performs 2. In the image processing method according to claim 1, The computer includes: In the converting step, a reference area is set for each pixel, and the color information of each pixel is converted based on the color information of other pixels in the reference area; In the determining step, a size of the reference area is determined for each pixel based on the distance information. 3. In the image processing method according to 2, The computer includes: In the determination process, the image processing method determines the size of the reference area for each pixel based on reference area information that divides the distance from the camera into N stages (N is an integer greater than or equal to 2) and defines the size of the reference area for each stage, and on the distance information. 4. In the image processing method according to 3, The computer includes: The image processing method includes determining, in the determining step, the size of the reference area for each pixel based on the reference area information, which is a formula for calculating the size of the reference area. 5. In the image processing method according to any one of 2 to 4, The computer includes: In the determining step, When the distance indicated by the distance information is included in the range from D1 to D2, the size of each pixel is determined as the size of the reference area; When the distance indicated by the distance information is greater than D2, determining a larger reference area as the distance indicated by the distance information increases; When the distance information is smaller than D1, the smaller the distance indicated by the distance information, the larger the reference area is determined. 6. An acquisition unit that acquires image information indicating distance information and color information for each pixel from the camera; a determination unit that determines, for each pixel, a setting of a modulation filter that converts the color information of each pixel and modulates the image into a painterly style based on the distance information of each pixel; a conversion unit that converts the color information for each pixel based on the determined setting of the modulation filter for each pixel; An image processing device comprising: 7. Computer An acquisition means for acquiring image information indicating distance information indicating a distance from the camera and color information for each pixel; a determining means for determining, for each pixel, a setting of a modulation filter for converting the color information of each pixel and modulating the image into a painterly style based on the distance information of each pixel; a conversion means for converting the color information for each pixel based on the determined setting of the modulation filter for each pixel; A program that functions as a 8. An input receiving unit that receives operation input from a player; a player character control unit that manages the position and orientation of the player character in the virtual three-dimensional space; a non-player character control unit that manages positions and orientations of non-player characters within the virtual three-dimensional space; a camera control unit that manages a position and an orientation of a virtual camera in the virtual three-dimensional space; a rendering unit that renders the virtual three-dimensional space captured by the virtual camera as an image on a two-dimensional plane based on positions and orientations of the player character, the non-player character, and the virtual camera in the virtual three-dimensional space; a display control unit that causes an output device to display the two-dimensional image; having The drawing unit is an acquisition unit that acquires image information indicating distance information and color information for each pixel from the virtual camera; a determination unit that determines, for each pixel, a setting of a modulation filter that converts the color information of each pixel and modulates the image into a painterly style based on the distance information of each pixel; a conversion unit that converts the color information for each pixel based on the determined setting of the modulation filter for each pixel; A game device having the above configuration. [Explanation of symbols]

[0071] 1A Processor 2A Memory 3A input / output I / F 4A Peripheral circuit 5A Bus 1 Input reception section 2 Player character control unit 3 Non-player character control unit 4 Camera control section 5. Drawing section 6 Display control section 10 Image processing device 11 Acquisition Department 12 Decision Section 13 Conversion section 14 Information acquisition department 15 2D image drawing section 16 Output section 100 Gaming Devices

Claims

1. The computer an acquisition step of acquiring image information indicating, for each pixel, distance information indicating a distance from a camera that photographs the player character from a position behind and at a predetermined distance away, and color information; determining, for each pixel, a setting of a modulation filter that converts the color information of each pixel to modulate an image based on the distance information of each pixel; a conversion step of converting the color information for each pixel based on the determined settings of the modulation filter for each pixel; Run In the converting step, setting a reference area for each pixel, and converting the color information of each pixel based on the color information of other pixels within the reference area; In the determining step, determining, for each pixel, a size of the reference area based on the distance information; The distance is d ０ more than d １ for pixels less than 1, determining a first fixed value as the size of the reference area; The distance is d １ more than d ２ For pixels less than the distance, a smaller reference area is determined as the distance increases based on a first function that calculates the reference area from the distance; The distance is d ２ more than d ３ for pixels less than 1, determining a second fixed value as the size of the reference area; The distance is d ３ more than d ４ For pixels less than the distance, a larger reference area is determined as the distance increases based on a second function that calculates the reference area from the distance; The distance is d ４ For the above pixels, a third fixed value is determined as the size of the reference area; The second fixed value < the first fixed value < the third fixed value is satisfied, the second fixed value is a size of one pixel; The first function is such that the distance d １ The first fixed value is calculated when the distance is d ２ Calculating the second fixed value at The second function is such that the distance d ３ The second fixed value is calculated when the distance is d ４ The image processing method further comprises calculating the third fixed value when

2. an acquisition unit that acquires image information that indicates, for each pixel, distance information that indicates a distance from a camera that photographs the player character from a position behind and at a predetermined distance away, and color information; a determination unit that determines, for each pixel, a setting of a modulation filter that converts the color information of each pixel to modulate an image based on the distance information of each pixel; a conversion unit that converts the color information for each pixel based on the determined setting of the modulation filter for each pixel; having The conversion unit is setting a reference area for each pixel, and converting the color information of each pixel based on the color information of other pixels within the reference area; The determination unit is determining, for each pixel, a size of the reference area based on the distance information; The distance is d ０ more than d １ for pixels less than 1, determining a first fixed value as the size of the reference area; The distance is d １ more than d ２ For pixels less than the distance, a smaller reference area is determined as the distance increases based on a first function that calculates the reference area from the distance; The distance is d ２ more than d ３ for pixels less than 1, determining a second fixed value as the size of the reference area; The distance is d ３ more than d ４ For pixels less than the distance, a larger reference area is determined as the distance increases based on a second function that calculates the reference area from the distance; The distance is d ４ For the above pixels, a third fixed value is determined as the size of the reference area; The second fixed value < the first fixed value < the third fixed value is satisfied, the second fixed value is a size of one pixel; The first function is such that the distance d １ The first fixed value is calculated when the distance is d ２ Calculating the second fixed value at The second function is such that the distance d ３ The second fixed value is calculated when the distance is d ４ The image processing device calculates the third fixed value at the time.

3. Computer, an acquisition means for acquiring image information indicating, for each pixel, distance information indicating a distance from a camera that photographs the player character from a position behind and at a predetermined distance away, and color information; a determining means for determining, for each pixel, a setting of a modulation filter that converts the color information of each pixel to modulate an image based on the distance information of each pixel; a conversion means for converting the color information for each pixel based on the determined setting of the modulation filter for each pixel; Function as a The conversion means is setting a reference area for each pixel, and converting the color information of each pixel based on the color information of other pixels within the reference area; The determining means is determining, for each pixel, a size of the reference area based on the distance information; The distance is d ０ more than d １ for pixels less than 1, determining a first fixed value as the size of the reference area; The distance is d １ more than d ２ For pixels less than the distance, a smaller reference area is determined as the distance increases based on a first function that calculates the reference area from the distance; The distance is d ２ more than d ３ for pixels less than 1, determining a second fixed value as the size of the reference area; The distance is d ３ more than d ４ For pixels less than the distance, a larger reference area is determined as the distance increases based on a second function that calculates the reference area from the distance; The distance is d ４ For the above pixels, a third fixed value is determined as the size of the reference area; The second fixed value < the first fixed value < the third fixed value is satisfied, the second fixed value is a size of one pixel; The first function is such that the distance d １ The first fixed value is calculated when the distance is d ２ Calculating the second fixed value at The second function is such that the distance d ３ The second fixed value is calculated when the distance is d ４ A program for calculating the third fixed value at the time.

4. an input receiving unit that receives an operation input from a player; a player character control unit that manages the position and orientation of the player character in the virtual three-dimensional space; a non-player character control unit that manages positions and orientations of non-player characters in the virtual three-dimensional space; a camera control unit that manages a position and a direction, within the virtual three-dimensional space, of a virtual camera that photographs the player character from a position behind the player character and at a predetermined distance away; a drawing unit that draws the virtual three-dimensional space captured by the virtual camera as an image on a two-dimensional plane based on positions and orientations of the player character, the non-player character, and the virtual camera in the virtual three-dimensional space; a display control unit that causes an output device to display the two-dimensional image; having The drawing unit is an acquisition unit that acquires image information indicating distance information and color information for each pixel from the virtual camera; a determination unit that determines, for each pixel, a setting of a modulation filter that converts the color information of each pixel to modulate an image based on the distance information of each pixel; a conversion unit that converts the color information for each pixel based on the determined setting of the modulation filter for each pixel; having The conversion unit is setting a reference area for each pixel, and converting the color information of each pixel based on the color information of other pixels within the reference area; The determination unit is determining, for each pixel, a size of the reference area based on the distance information; The distance is d ０ more than d １ for pixels less than 1, determining a first fixed value as the size of the reference area; The distance is d １ more than d ２ For pixels less than the distance, a smaller reference area is determined as the distance increases based on a first function that calculates the reference area from the distance; The distance is d ２ more than d ３ for pixels less than 1, determining a second fixed value as the size of the reference area; The distance is d ３ more than d ４ For pixels less than the distance, a larger reference area is determined as the distance increases based on a second function that calculates the reference area from the distance; The distance is d ４ For the above pixels, a third fixed value is determined as the size of the reference area; The second fixed value < the first fixed value < the third fixed value is satisfied, the second fixed value is a size of one pixel; The first function is such that the distance d １ The first fixed value is calculated when the distance is d ２ Calculating the second fixed value at The second function is such that the distance d ３ The second fixed value is calculated when the distance is d ４ The game device calculates the third fixed value at the time.

Images