Information processing device, information processing method, and program

Abstract

To provide processing for high-definition surface representation and reduction of image generation time in displaying an image for confirming a surface of an object.SOLUTION: An information processing apparatus for generating an image to be displayed on the basis of shape data which represents a shape of an object and surface characteristic data which represents surface characteristics of the object includes: acquisition means which acquires two pieces of shape data different in resolution and surface characteristic data; setting means which sets a display area in the object; and generation means which generates an image to be displayed on the basis of one of the shape data corresponding to the display area and the surface characteristic data.SELECTED DRAWING: Figure 5

Description

[Technical field]

[0001] The present invention relates to a technique for displaying an image for confirming the surface of an object. [Background technology]

[0002] In recent years, with the spread of online shopping, users are increasingly checking product information on information devices such as personal computers and smartphones to decide whether or not to purchase a product. When checking product information, an image obtained by rendering the product's shape data is displayed on a display. If the processing load of this rendering is large, image generation may be delayed in response to a user's instruction. Patent Document 1 discloses a technology for high-speed image generation by reducing the amount of information in image data of non-focused areas compared to the amount of information in image data of focused areas. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 7-65194 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in Patent Document 1, depending on the display range specified by the user, the entire displayed area may become the gaze area, and there is a problem that it is not possible to reduce the time required to generate an image.

[0005] SUMMARY OF THE PRESENT DISCLOSURE An object of the present invention is to provide a process for achieving both high-definition representation of the surface and a reduction in the time required to generate the image when displaying an image for confirming the surface of an object. [Means for solving the problem]

[0006] In order to solve the above problem, an information processing device according to the present invention includes a receiving means for receiving an instruction from a user to change a display magnification of an image including an object, and a threshold value for the display magnification, which is set to a first threshold value when the instruction from the user is an instruction to change from a first display magnification to a second display magnification that is greater than the first display magnification, and a threshold value for the display magnification is set to a second display magnification that is greater than the first threshold value when the instruction from the user is an instruction to change from the second display magnification to the first display magnification. big The display device is characterized in having a setting means for setting a second threshold value, a determination means for determining whether a display magnification specified by a user is equal to or greater than the set threshold value, and a generation means for generating an image corresponding to a display area of ​​the object based on first shape data representing a shape of the object and first surface characteristic data representing surface characteristics of the object if the display magnification specified by the user is equal to or greater than the set threshold value, and for generating an image corresponding to a display area of ​​the object based on second shape data having a higher resolution than the first shape data and second surface characteristic data having a lower resolution than the first surface characteristic data if the display magnification specified by the user is less than the set threshold value. Effect of the Invention

[0007] According to the present invention, when an image for confirming the surface of an object is displayed, it is possible to achieve both a high-definition representation of the surface and a reduction in the time required to generate the image. [Brief description of the drawings]

[0008] [Figure 1] A block diagram showing a hardware configuration of an information processing device. [Diagram 2] FIG. 1 is a block diagram showing a functional configuration of an information processing device. [Diagram 3] A diagram showing an example of a user interface. [Figure 4] Diagram showing the structure of object data [Diagram 5] 1 is a flowchart showing a process executed by an information processing device; [Figure 6] 1 is a flowchart showing a process executed by an information processing device; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Each embodiment will be described below with reference to the drawings. Note that the following embodiments do not necessarily limit the present invention. Furthermore, not all of the combinations of features described in each embodiment are necessarily essential to the solution of the present invention.

[0010] [First embodiment] When a user wants to check the shape of an object or the texture of a material, the user uses an input device to enlarge, reduce, rotate, and translate the object so that the area of ​​interest is displayed. When surface characteristic data is added to shape data to express the texture of the material, the processing load when generating an image is large, and the generation of the image may be delayed in response to the user's instruction. Therefore, in this embodiment, based on information representing the display range of the object, it is determined whether to use high-resolution shape data or low-resolution shape data for image generation. When a user wants to focus on the texture of a material, low-resolution shape data is used to generate the image, thereby shortening the image generation time while expressing the texture of the material in high definition.

[0011] <Hardware configuration of information processing device> The hardware configuration of an information processing device 1 in this embodiment will be described with reference to Fig. 1. In Fig. 1, the information processing device 1 includes a CPU 101, a ROM 102, and a RAM 103. The information processing device 1 also includes a VC (video card) 104, a general-purpose I / F (interface) 105, a SATA (serial ATA) I / F 106, and a NIC (network interface card) 107.

[0012] The CPU 101 executes an OS (operating system) and various programs stored in the ROM 102, HDD (hard disk drive) 112, etc., using the RAM 103 as a work memory. The CPU 101 also controls each component via a system bus 108. Note that, in the process according to the flowchart described below, program codes stored in the ROM 102, HDD 112, etc. are expanded in the RAM 103 and executed by the CPU 101. The display device 2 is connected to the VC 104. An input device 110 such as a mouse or keyboard and an imaging device 3 are connected to the general-purpose I / F 105 via a serial bus 109. The HDD 112 and a general-purpose drive 113 that reads and writes data from and to various recording media are connected to the SATA I / F 106 via a serial bus 111. The NIC 107 inputs and outputs information to and from external devices. The CPU 101 uses the HDD 112 and various recording media mounted on the general-purpose drive 113 as storage locations for various data. The CPU 101 displays a GUI (Graphical User Interface) provided by a program on the display device 2, and receives inputs such as user instructions accepted via the input device 110.

[0013] <Functional configuration of information processing device> Fig. 2 is a block diagram showing the functional configuration of the information processing device 1. The CPU 101 functions as the functional configuration shown in Fig. 2 by using the RAM 103 as a work memory and reading and executing a program stored in the ROM 102 or the HDD 112. Note that it is not necessary for all of the processes shown below to be executed by the CPU 101, and the information processing device 1 may be configured so that part or all of the processes are executed by one or more processing circuits other than the CPU 101.

[0014] The information processing device 1 includes an acquisition unit 201, a setting unit 202, a generation unit 203, and a display control unit 204. The acquisition unit 201 acquires object data from a storage device such as the HDD 112 based on a user's instruction. FIG. 4 shows the structure of the object data. The object data includes two types of shape data, surface property data for each material, and text data. The two types of shape data are high-resolution shape data and low-resolution shape data. The high-resolution shape data is data that represents a shape that is approximated to the shape of an object using a large number of polygons to express a detailed shape. The low-resolution shape data is shape data obtained by lowering the resolution of the high-resolution shape data. Each shape data is polygon data that includes a vertex list, a face list, a UV coordinate vertex list, a shape characteristic upper limit value, and a shape characteristic lower limit value. The vertex list is a list whose elements are a set of coordinates of each vertex of a polygon, and the face list is a list whose elements are a set of three vertices that constitute a face, written counterclockwise with respect to the normal of the face. The UV coordinate vertex list is a list whose elements are a set of coordinates of each vertex of a polygon in the UV coordinate system. Here, the UV coordinate system is a two-dimensional coordinate system in which polygons are expanded onto a plane in order to map surface characteristic data onto polygon faces. The upper and lower shape characteristic limits are values ​​that define the range in which shape data can be used. Specifically, the shape characteristic value is the sum of squares of the residuals when a plane is fitted to the polygon vertices contained in the shape of the rendering range. If the shape characteristic value is large, the shape of the rendering range will be complex and uneven. If the shape characteristic value is small, the shape of the rendering range will be flat and featureless.

[0015] The surface characteristic data is data representing the surface characteristics of an object, and includes an ambient light map, a diffuse light map, a specular light map, and a normal vector map. The ambient light map, the diffuse light map, the specular light map, and the normal vector map are each described in a UV coordinate system. The surface characteristic data in this embodiment represents surface characteristics expressed based on the Phong reflection model. The text data is data that holds the name (product name), size, color, and material as detailed information about the object. The setting unit 202 sets the display conditions of the image. The generation unit 203 performs rendering based on the object data to generate an image.

[0016] The display control unit 204 displays a UI for receiving user instructions and the generated image on the display device 2. Fig. 3 is a diagram showing an example of the UI. The UI 301 includes an image display area 302, a text display area 303, an instruction area 304, and an end button 305. The generated image is displayed in the image display area 302, and the size and material of the object are displayed in the text display area 303. When the enlargement / reduction button, the rotation button, or the translation button is pressed in the instruction area 304, the object is enlarged / reduced, rotated, or translated in the image displayed in the image display area 302.

[0017] <Processing performed by the information processing device> 5 is a flowchart showing the process executed by the information processing device 1. Hereinafter, each step (process) is represented by adding an S before the reference symbol. In S501, the acquisition unit 201 acquires display device information related to the display device 2 from a storage device such as the HDD 112, and stores it in the RAM 103. The display device information is information indicating the resolution and the number of vertical and horizontal pixels of the display device 2. In S502, the acquisition unit 201 acquires object data from a storage device such as the HDD 112 based on a user instruction.

[0018] In S503, the setting unit 202 initializes the display conditions of the image. Specifically, the setting unit 202 sets the display area so that the entire object is displayed in the image display area 302. Since the entire object is displayed, high-resolution shape data is set as shape data used to generate the image. The setting unit 202 sets a global orthogonal coordinate system, and places the center of the object at the origin of the global orthogonal coordinate system. The setting unit 202 also places a virtual camera on the X axis of the global orthogonal coordinate system as a viewpoint. A predetermined value is set for the angle of view of the virtual camera, and the virtual camera is placed at a position on the X coordinate so that the entire object is included in the angle of view. The angle of view of the virtual camera in this embodiment is 40 degrees. Note that the degree of enlargement or reduction changes depending on the X coordinate of the virtual camera. The setting unit 202 initializes the rotation angles of the X axis, Y axis, and Z axis of the object to 0 degrees. The number of pixels of the virtual camera is set based on the number of pixels assigned to the image display area 302 among the number of pixels of the display device 2 represented by the display device information. The virtual light source is placed at a predetermined position so as not to overlap with the object.

[0019] In S504, the display control unit 204 displays the UI 301 on the display device 2. The size of the UI 301 is determined based on the display device information. At this time, the generation unit 203 performs rendering based on the text data, and the text resulting from the rendering is displayed in the text display area 303. In S505, the generation unit 203 performs rendering using one piece of shape data and surface characteristic data based on the display conditions of the set image to generate an image. Specifically, the generation unit 203 determines the coordinate values ​​of the vertices in the global orthogonal coordinate system from the coordinate values ​​of the vertex list of the shape data. The generation unit 203 generates a transformation matrix for perspective transformation based on the positions of the object and virtual camera and the pixel data of the virtual camera, and converts the shape data in the global orthogonal coordinate system into data in the visual field coordinate system. The generation unit 203 calculates the distance from the virtual camera to each polygon, and performs hidden surface processing to leave only those in the foreground. The generation unit 203 maps the surface characteristic data to the UV coordinates of the shape data after the hidden surface processing. The generation unit 203 generates an image by performing rendering using a Phong reflection model based on the normal vector of the polygon calculated based on the vertex coordinate list and the face list, the line-of-sight vector from the virtual camera to the polygon, and each map included in the surface characteristic data. The generated image is displayed in the image display area 302 by the display control unit 204.

[0020] In S506, the acquisition unit 201 reads the user's instruction input. In S507, the acquisition unit 201 determines whether the user's instruction input is an end instruction. If the user's instruction input is an end instruction by pressing the end button 305, the acquisition unit 201 deletes the UI 301 and ends the process.

[0021] In S508, the acquisition unit 201 determines whether or not the instruction is an instruction to change the settings of the display conditions. If the instruction input by the user is an instruction to change by pressing a button in the instruction area 304, in S509, the setting unit 202 updates the settings of the display conditions. The setting unit 202 moves the X coordinate of the virtual camera forward or backward and rotates it in the X-axis, Y-axis, and Z-axis directions according to each instruction for enlargement / reduction, rotation, and translation. This update of the settings changes the display area of ​​the object. If the instruction input by the user is not an instruction to change the settings, the process returns to reading the instruction input in S506.

[0022] In S510, the acquisition unit 201 determines whether the user's instruction input is an instruction to enlarge or reduce (change the display range). If the user's instruction input is an instruction to enlarge or reduce, in S511, the setting unit 202 calculates a shape characteristic value of an object in the changed display area. Specifically, the setting unit 202 calculates, as the shape characteristic value, the sum of squares of the residual when plane fitting is performed on polygon vertices included in the object in the display area. The setting unit 202 performs the perspective transformation and hidden surface removal used in rendering, and extracts the polygons to be generated. Plane fitting is performed by the least squares method on the extracted polygon vertices, and the least square error is taken as the shape characteristic value.

[0023] In S512, the setting unit 202 determines whether the calculated shape characteristic value is included in the range determined by the shape characteristic upper limit value and the shape characteristic lower limit value corresponding to the shape data used to generate the currently displayed image. If the calculated shape characteristic value is out of the range, in S513, the setting unit 202 updates the shape data. Specifically, the setting unit 202 changes the set shape data to shape data in which the calculated shape characteristic value is included in the range. If the user's instruction input is not an instruction to enlarge or reduce, if the calculated shape characteristic value is within the range, or if the shape data has been updated, the process returns to the rendering process in S505. The newly generated image is displayed in the image display area 302 by the display control unit 204.

[0024] <Advantages of the First Embodiment> As described above, the information processing device in this embodiment acquires two pieces of shape data and surface characteristic data with different resolutions, and sets a display area for an object. An image of a display object is generated based on one piece of shape data corresponding to the display area and the surface characteristic data. This makes it possible to achieve both high-definition representation of the surface and a reduction in image generation time when displaying an image for checking the surface of an object.

[0025] Consider a case where the object is a product. When the user focuses on the overall shape or decorative parts of the product, the product displayed in the image display area 302 has a complex shape with unevenness, and the shape characteristic value is a large value. In this case, high-resolution shape data that expresses the characteristics of the shape in detail is used for rendering. Even if the resolution of the surface characteristic data is lowered to reduce the rendering time, the quality of the surface characteristics has little effect on the user's recognition because the user focuses on the shape of the product. When the user focuses on the texture of the material that constitutes the product (enlargement is instructed), the product displayed in the image display area 302 has a simple shape with few unevenness, and the shape characteristic value is a small value. In this case, low-resolution shape data is used for rendering, and the time required for rendering is reduced. Also, the resolution of the object's shape has little effect on the user's recognition because the user focuses on the texture of the material. In this way, by achieving both high-definition expression of the surface and reduced image generation time, it is possible to quickly respond to when the user wants to change the display content and display the area to be focused on in high definition.

[0026] [Second embodiment] In the first embodiment, when an object is enlarged and displayed, low-resolution shape data is used for rendering. In this embodiment, when an object is enlarged and displayed, low-resolution shape data and high-resolution surface characteristic data are used for rendering, and when an object is reduced and displayed, high-resolution shape data and low-resolution surface characteristic data are used for rendering. Note that the hardware configuration and functional configuration of the information processing device in this embodiment are the same as those in the first embodiment, so a description thereof will be omitted. Below, differences between this embodiment and the first embodiment will be mainly described. Note that the same configurations as those in the first embodiment will be described with the same reference numerals.

[0027] <Processing performed by the information processing device> FIG. 6 is a flowchart showing the process executed by the information processing device 1. Note that the process of S601, and S604 to S609 is the same as S501, and S504 to S509 in the first embodiment, and therefore the description will be omitted. In S602, the acquisition unit 201 acquires two types of object data based on a user's instruction. The two types of object data are object data used at a low display magnification and object data used at a high display magnification. In S603, the setting unit 202 initializes the display conditions of the image. Specifically, the setting unit 202 initializes the display magnification in addition to the setting process in the first embodiment. In S610, the acquisition unit 201 determines whether the user's instruction input is an instruction to change the display magnification.

[0028] When the user's instruction input is an instruction to change the display magnification, in S611, the setting unit 202 determines whether the changed display magnification is equal to or greater than a threshold. When the changed display magnification is equal to or greater than the threshold and object data corresponding to a high display magnification is set, the setting unit 202 determines that object data suitable for the changed display magnification is set. Also, when the changed display magnification is less than the threshold and object data corresponding to a low display magnification is set, the setting unit 202 determines that object data suitable for the changed display magnification is set. On the other hand, when the changed display magnification is equal to or greater than the threshold and object data corresponding to a low display magnification is set, the setting unit 202 determines that object data not suitable for the changed display magnification is set. Also, when the changed display magnification is less than the threshold and object data corresponding to a high display magnification is set, the setting unit 202 determines that object data not suitable for the changed display magnification is set.

[0029] In S612, the setting unit 202 updates the object data. Specifically, when the changed display magnification is equal to or greater than a threshold, the setting unit 202 sets object data corresponding to a high display magnification, and when the changed display magnification is less than the threshold, the setting unit 202 sets object data corresponding to a low display magnification.

[0030] <Effects of the second embodiment> As described above, the information processing device in this embodiment switches the object data used for rendering depending on the display magnification. This makes it possible to achieve both high-definition representation of the surface and a reduction in the time required to generate the image when displaying an image for checking the surface of the object.

[0031] [Variations] In the above-described embodiment, the low-resolution shape data is generated by performing a process to reduce the resolution of the high-resolution shape data, but data representing a spherical shape may be used as the low-resolution shape data regardless of the shape of the object. If the shape represented by the low-resolution shape data is a spherical shape with a large curvature, it has multiple surface directions with respect to the incident light, making it easier to recognize the surface characteristics.

[0032] In the above embodiment, the shape data used for rendering is selected from two types of shape data, high resolution shape data and low resolution shape data, but it may be selected from three or more types of shape data. For example, the shape data used for rendering may be selected from three types of shape data with different resolutions, such as high resolution, medium resolution, and low resolution.

[0033] Furthermore, the surface characteristic data in the above-described embodiment is data representing surface characteristics expressed based on the Phong reflectance model, but the data may be data representing surface characteristics based on other reflectance models, such as surface characteristics based on a bidirectional reflectance distribution function (BRDF).

[0034] Furthermore, the acquisition unit 201 in the above-described embodiment acquires object data based on a user's instruction, but may acquire predetermined object data.

[0035] In addition, in the above-described embodiment, the shape data to be used for rendering is selected from pre-generated shape data of different resolutions, but it is also possible to generate multiple types of shape data by lowering the resolution by performing downsampling processing on one piece of shape data.

[0036] In the above embodiment, rendering is performed using surface property data for each material, but even if there are multiple materials in the display area, only the surface property data of the material at the center of the display area may be mapped to the shape data. Also, only the surface property data of the material with the highest occupancy rate in the display area may be used.

[0037] Furthermore, in the first embodiment described above, only the shape data is updated, but the resolution of the surface characteristic data may also be reduced depending on the time required for rendering processing.

[0038] In the first embodiment described above, the sum of squares of the residuals when plane fitting is performed on the polygon vertices included in the object in the display area is calculated as the shape characteristic value, but the shape characteristic value may be calculated by other methods. For example, the variance of the dot product of the normal vector of the polygon surface included in the object in the display area and the line of sight vector of the virtual camera may be calculated as the shape characteristic value. If the variance of the vector dot product is large, it can be determined that the polygon surface faces in various directions and that the shape of the object in the display area is complex.

[0039] In the second embodiment described above, one display magnification threshold is used, but multiple thresholds may be used to switch between different display magnifications. For example, a threshold for switching from a low display magnification to a high display magnification may be smaller than a threshold for switching from a high display magnification to a low display magnification.

[0040] [Other embodiments] The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions. [Explanation of symbols]

[0041] 1. Information processing device 201 Acquisition Department 202 Setting section 203 Generation part

Claims

1. A receiving means for receiving an instruction from a user to change the display magnification of an image including an object; a setting means for setting a first threshold value as a threshold value for the display magnification when the instruction from the user is an instruction to change from a first display magnification to a second display magnification larger than the first display magnification, and for setting a second threshold value larger than the first threshold value when the instruction from the user is an instruction to change from the second display magnification to the first display magnification; a determination means for determining whether or not a display magnification designated by a user is equal to or greater than the set threshold value; a generating means for generating an image corresponding to a display area of ​​the object based on first shape data representing a shape of the object and first surface characteristic data representing surface characteristics of the object when a display magnification designated by a user is equal to or greater than the set threshold value, and for generating an image corresponding to a display area of ​​the object based on second shape data having a higher resolution than the first shape data and second surface characteristic data having a lower resolution than the first surface characteristic data when a display magnification designated by a user is less than the set threshold value; 13. An information processing device comprising:

2. 2. The information processing apparatus according to claim 1, wherein the accepting means accepts an instruction from a user to enlarge or reduce the display.

3. The display magnification change control further includes a second determination unit that determines whether a value representing a shape characteristic of the object after the display magnification change is within a range corresponding to the shape data before the display magnification change, 3. The information processing apparatus according to claim 1, wherein the generating means determines shape data used to generate an image corresponding to a display area of ​​the object based on a result of the determination by the second determining means.

4. 4. The information processing apparatus according to claim 3, wherein the second determination means calculates a least square error obtained by performing plane fitting using a least squares method on polygon vertices contained in the object in the display area as a value representing the shape characteristic of the object.

5. 4. The information processing device according to claim 3, wherein the second determination means calculates the variance of an inner product of a normal vector of a polygon face included in the object in the display area and a line of sight vector of a virtual camera as a value representing a shape characteristic of the object.

6. 6. The information processing apparatus according to claim 1, wherein the generating means acquires surface characteristic data for each material contained in the object, and generates an image corresponding to the display area.

7. The information processing device according to any one of claims 1 to 5, characterized in that, when the display area includes a plurality of materials, the generation means generates an image corresponding to the display area by rendering using surface characteristic data of the material located at the center of the display area.

8. The information processing device according to any one of claims 1 to 5, characterized in that, when the display area contains a plurality of materials, the generation means generates an image corresponding to the display area by rendering using surface characteristic data of the material that occupies the largest proportion of the display area.

9. 9. The information processing apparatus according to claim 1, further comprising a display control unit that displays an image corresponding to the display area on a display unit.

10. A program for causing a computer to function as each of the means of the information processing device according to any one of claims 1 to 9.

11. a receiving step of receiving an instruction from a user to change a display magnification of an image including an object; a setting step of setting a first threshold value as a threshold value for the display magnification when the instruction from the user is an instruction to change from a first display magnification to a second display magnification larger than the first display magnification, and setting a second threshold value larger than the first threshold value when the instruction from the user is an instruction to change from the second display magnification to the first display magnification; a determination step of determining whether or not the display magnification designated by the user is equal to or greater than the set threshold value; a generating step of generating an image corresponding to a display area of ​​the object based on first shape data representing a shape of the object and first surface characteristic data representing surface characteristics of the object when the display magnification designated by the user is equal to or greater than the set threshold value, and generating an image corresponding to the display area of ​​the object based on second shape data having a higher resolution than the first shape data and second surface characteristic data having a lower resolution than the first surface characteristic data when the display magnification designated by the user is less than the set threshold value; 13. An information processing method comprising:

Images