Image processing method, image processing device, and program

Abstract

The present invention provides an image processing device that efficiently generates multiple parallel cross-sectional images of a three-dimensional shape model. [Solution] The image processing method of the present disclosure is an image processing method for generating a plurality of parallel cross-sectional images from a three-dimensional shape model, characterized by comprising: an acquisition step of acquiring a plurality of values ​​indicating the distance from a predetermined plane to each part of the three-dimensional shape model; a determination step of determining the distance from the predetermined plane based on the plurality of values; and an image generation step of generating the cross-sectional image at the determined distance.

Description

Technical Field

[0001] The present disclosure relates to an image processing method, an image processing apparatus, and a program.

Background Art

[0002] A plurality of cross-sectional images parallel to each other of a three-dimensional shape model are used in various fields and applications regardless of the object such as a real object or a virtual object.

[0003] Patent Document 1 proposes a method of generating a plurality of cross-sectional images parallel to each other of a part of an object when measuring the shape of the object, and performing measurement of the part based on the plurality of cross-sectional images.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] In the method of Patent Document 1, when generating a plurality of cross-sectional images parallel to each other of a part of an object, the cross-sectional images are captured at equal intervals. For this reason, a plurality of similar cross-sectional images are generated at positions where the change in the cross-sectional shape is small, so there is a problem in terms of efficiency from the viewpoints of calculation speed and memory efficiency of the computer.

Means for Solving the Problems

[0006] The image processing method according to this disclosure is an image processing method for generating a plurality of parallel cross-sectional images from a three-dimensional shape model, comprising: an acquisition step of acquiring a plurality of values ​​indicating the distance from a predetermined plane to each part of the three-dimensional shape model; a determination step of determining the distance from the predetermined plane based on the plurality of values; and an image generation step of generating the cross-sectional image at the determined distance, wherein in the acquisition step, the distance is detected as the distance at which the minimum value among the plurality of values ​​changes to a threshold or greater. [Effects of the Invention]

[0007] According to the image processing method disclosed herein, multiple parallel cross-sectional images of a three-dimensional shape model can be generated more efficiently. [Brief explanation of the drawing]

[0008] [Figure 1] This is a flowchart of the basic processing according to the first embodiment. [Figure 2] This is a flowchart showing the details of the process at S5 in Figure 1 according to the first embodiment. [Figure 3] This figure shows an example configuration of an information processing device according to the first embodiment. [Figure 4] This figure shows an example of a three-dimensional shape model according to the first embodiment. [Figure 5] This figure shows an example of a slice image generated from the three-dimensional shape model of Figure 4 according to the first embodiment. [Figure 6] This figure shows an example of the depth value distribution in Figure 5 according to the first embodiment. [Figure 7] This figure shows an example of a slice image generated from a three-dimensional shape model according to the first embodiment. [Figure 8] This is a flowchart showing the details of the process at S5 in Figure 1 according to the second embodiment. [Figure 9] This figure shows an example of the depth value distribution obtained by searching for the cross-sectional change position according to the second embodiment. [Figure 10]This figure shows an example of a detected cross-sectional change position according to the second embodiment. [Figure 11] This figure shows an example of a slice image generated from a three-dimensional shape model according to the second embodiment. [Modes for carrying out the invention]

[0009] A first embodiment of this disclosure will be described with reference to Figure 1. Figure 1 shows a flowchart of the process according to the first embodiment.

[0010] S1 is the process of loading the three-dimensional shape model from which slice images will be generated. The file format of the three-dimensional shape model is not limited; for example, STL, STEP, and OBJ are used.

[0011] S2 is the process of selecting the viewpoint direction when generating slice images of the loaded three-dimensional shape model. In order to accurately capture the characteristics of cross-sectional changes in the three-dimensional shape, it is preferable to select a viewpoint direction in which the irregularities of the shape can be observed when the depth value is obtained.

[0012] S3 is a process that extracts the locations where slice images are generated in the loaded 3D shape model and adds a reference position with a depth value of 0 as the first element to the cross-sectional change position list. The reference position with a depth value of 0 can be selected by the operator, or it can be automatically obtained and selected by S3 from the position of the face closest to the viewer when viewed from the viewpoint direction selected in S2 at the slice image generation locations extracted in S3. Furthermore, the slice image generation locations to be extracted can be the entire 3D shape model or a part of the 3D shape model.

[0013] S4 is a process that obtains the depth value distribution from the viewpoint direction selected in S2 for the generated slice image extracted in S3.

[0014] S5 is a process that detects the position of cross-sectional change based on the depth value distribution obtained in S4. Details of the S5 process will be described later in the explanation of Figure 2.

[0015] S6 is a process of determining whether the cross-section change position newly added to the cross-section change position list in the process of S5 has reached the deepest position when viewed from the viewpoint direction selected in S2. If it is determined that the deepest position has not been reached, return to S4, set the cross-section change position added to the list as a new reference position with a depth value of 0, obtain the depth value distribution, and proceed to the process of S5. If it is determined that the deepest position has been reached, proceed to the process of step S7.

[0016] S7 is a process of generating cross-section images at all positions in the cross-section change position list detected by the processes from S1 to S6.

[0017] That is, in S1 to S7, the following processes are performed. First, for the loaded three-dimensional shape model, a plurality of values indicating the distances from a predetermined plane (the reference position with a depth value of 0 selected in S3) to each part of the three-dimensional shape model are obtained (S4). Subsequently, based on the plurality of values obtained in S4, the distance from the predetermined plane is determined (S5). And cross-section images at the determined distances are generated (S7).

[0018] FIG. 2 is a flowchart showing the details of the process in S5 of FIG. 1.

[0019] S21 is a process of calculating the standard deviation in the acquired depth value distribution and setting it as a threshold value used for detecting the cross-section change position.

[0020] S22 represents a process of searching for the minimum depth value in the depth value distribution from among the unexplored cells. At this time, all cells having the same depth value as the minimum depth value are set as explored.

[0021] S23 is a process of determining whether the minimum depth value is greater than or equal to the threshold value set in S21. If the minimum depth value is less than the threshold value, return to S22 to search for the minimum depth value in the depth value distribution again. If the minimum depth value is greater than or equal to the threshold value, proceed to step S24.

[0022] S24 is a process in which, if it is determined in S23 that the minimum depth value is greater than or equal to a threshold, that depth value is detected as a cross-sectional change position, and the depth position corresponding to that depth value is added to the cross-sectional change position list.

[0023] The above processing flow enables efficient slice image generation that minimizes the number of images required, in accordance with the cross-sectional changes of the three-dimensional shape model.

[0024] Figure 3 shows an example of an image processing apparatus according to the present disclosure. An external storage unit 32, an internal storage unit 33, a display unit 34, and an input unit 35 are connected to a CPU (Central Processing Unit) 31, which is the processing unit.

[0025] The CPU 31 executes processing according to the instructions of the program. The external storage unit 32 is a device that stores the program (hereinafter referred to as "the program") which describes the processing to be executed by the CPU 31 in this disclosure, the processing results of the CPU 31, etc., and examples include a hard disk.

[0026] The internal memory unit 33 is a device that temporarily stores information used during processing by the CPU 31, and examples include RAM (Random Access Memory). The display unit 34 is a device that displays slice images and the like generated by processing by the CPU 31, and examples include a display. The input unit 35 is a device used by the operator to input information necessary for processing by the CPU 31, and examples include a keyboard and a mouse.

[0027] A specific example of the slice image generation procedure in this embodiment will be explained with reference to the flowchart shown in Figure 1.

[0028] In S1, the three-dimensional shape model to be used for slice image generation, stored in the external storage unit 32, is read. In this embodiment, as an example, the three-dimensional shape model 40 shown in Figure 4 is used.

[0029] The three-dimensional shape model 40 shown in Figure 4 is a three-dimensional shape model in which a shape consisting of a rectangular parallelepiped and a pyramidal pyramid is arranged on a flat plate. Figure 4 shows the viewpoint direction V, which is perpendicular to the cross-section from which the slice image of the three-dimensional shape model 40 is generated, and the face F that is closest to the viewer when the three-dimensional shape model 40 is viewed from the viewpoint direction V. Also in Figure 4, two points P1 and P2 are the endpoints of the bounding box representing the slice image generation location 41 in the three-dimensional shape model 40. Furthermore, faces (a), (b), (c), (d), and (e) in Figure 4 indicate the cross-sectional position of the slice image generated relative to the slice image generation location 41.

[0030] The three-dimensional shape model data acquired in S1 is stored in the internal memory unit 33 by instruction from the CPU 31. The three-dimensional shape model data is also displayed on the display unit 34 by instruction from the CPU 31.

[0031] In S2, the viewpoint direction V is selected when generating slice images of the three-dimensional shape model 40 loaded in S1. This can be done by the operator using the input unit 35, or by automatically selecting the viewpoint direction according to predetermined rules. Alternatively, the viewpoint direction may be estimated and selected using a machine learning model.

[0032] The viewpoint direction V selected in S2 is stored in the internal memory unit 33 by the CPU 31. Alternatively, the viewpoint direction V may be displayed on the display unit 34, or the viewpoint of the three-dimensional shape model 40 displayed on the display unit 34 may be changed so that it is viewed from viewpoint direction V. Figure 5 shows the three-dimensional shape model 40 as viewed from viewpoint direction V.

[0033] In S3, the slice image generation locations 41 in the three-dimensional shape model 40 read in S1 are extracted. The slice image generation locations 41 can be extracted by specifying them using the input unit 35, automatically according to predetermined rules, or automatically using a machine learning model. At this point, a reference location with a depth value of 0 is added as the first element to the cross-sectional change location list.

[0034] In S4, the depth value distribution of the three-dimensional shape model as viewed from the viewpoint direction V selected in S2 is obtained for the slice image generation area 41 extracted in S3. The first time S4 ​​is executed, the face closest to the viewer from the viewpoint direction V is selected as the reference position for depth value 0, and the depth value distribution from the reference position is obtained. For subsequent executions of S4, the next deepest position in the cross-section change position list is selected as the reference position for depth value 0, and the depth value distribution from the reference position is obtained again, repeating the process from S4 to S6 until the deepest part is reached. The obtained depth value distribution is stored in the internal storage unit 33. At this point, all cells with a depth value of 0 are set to have been explored.

[0035] Figure 6 shows the depth value distribution when the slice image generation area 41 of the three-dimensional shape model 40 in Figure 4 is obtained with the nearest face F as the reference position for depth value 0, viewed from the viewpoint direction V. In Figure 6, explored cells are filled in gray.

[0036] In S5, the cross-sectional change position is detected based on the depth value distribution obtained in S4. The process in S5 is explained below in accordance with the flowchart in Figure 2.

[0037] In S21, the standard deviation of the acquired depth value distribution is calculated and set as the threshold used for detecting the cross-sectional change position. In this example, the standard deviation from the depth value distribution shown in Figure 6, obtained in processing S4, is calculated to be 3.7. Therefore, the threshold used for detecting the cross-sectional change position is set to 3.7.

[0038] In S22, the minimum depth value in the depth value distribution is searched for among the unexplored cells. At this time, all cells with the same depth value as the minimum depth value are set to explored. When the depth value distribution in Figure 6 is searched, the minimum value of 5 is found among the unexplored cells. All cells with the same depth value as the minimum value of 5 are set to explored.

[0039] In S23, it is determined whether the minimum value found in S22 is greater than or equal to the threshold set in S21, and a conditional branch is made. If the minimum value is less than the threshold, the process returns to S22 to search for the minimum value in the depth value distribution. If it is greater than or equal to the threshold, the process proceeds to step S24. In this example, since the minimum value of 5 is greater than or equal to the threshold of 3.7, the process proceeds to step S24.

[0040] In S24, the depth value at which the minimum value was determined to be greater than or equal to the threshold in S23 is detected as the cross-sectional change position, and the depth position corresponding to that depth value is added to the cross-sectional change position list. In this example, when a depth value of 5 is found, it is determined to be greater than or equal to the threshold, so the depth position corresponding to depth value 5 is added to the cross-sectional change position list, and the process moves on to step S6. This completes the processing of the flow in Figure 2, and the process moves on to S6 in Figure 1.

[0041] In S6, it is determined whether the cross-sectional change position newly added to the cross-sectional change position list in S5 has reached the deepest position when viewed from the viewpoint direction selected in S2. If it is determined that it has not reached the deepest position, the process returns to S4, obtains the depth value distribution with the newly added cross-sectional change position as the reference position for depth value 0, and proceeds to S5 again. If it is determined that it has reached the deepest position, the process proceeds to S7.

[0042] In S7, cross-sectional images are generated at all locations in the list of cross-sectional change locations detected by processing S1 to S6. The planes (a), (b), (c), (d), and (e) in Figure 4 are the cross-sectional locations of the slice images generated for the slice image generation location 41.

[0043] Figure 7 shows the slice images of faces (a), (b), (c), (d), and (e) generated by performing the S7 process on the slice image generation area 41. In this example, the depth positions of faces (a), (b), (c), (d), and (e) are 0, 5, 8, 9, and 10, respectively.

[0044] The series of slice images obtained in this manner, as shown in Figure 7, and their respective depth positions, are stored in the external storage unit 32.

[0045] As a result, multiple parallel cross-sectional images of a three-dimensional shape model can be efficiently generated.

[0046] (Second embodiment) In this second embodiment, we will describe a case in which the detection of the cross-sectional change position at S5 in Figure 1 is performed using a different procedure than in the first embodiment. Note that parts similar to those in the first embodiment will be omitted from the explanation as appropriate. Figure 8 shows a flowchart of the method for detecting the cross-sectional change position at S5 in Figure 1 according to this embodiment.

[0047] First, steps S1-S4 in Figure 5 perform the same processing as in the first embodiment. Similar to the first embodiment, a depth value distribution as shown in Figure 6 is obtained, and 0 is added to the first element of the cross-sectional change position list. Next, the process proceeds to S5. In this embodiment, the following processing S81-S85 is performed.

[0048] S81 is a process that sets a threshold value of 20% of the difference between the maximum and minimum values ​​in the depth value distribution as a threshold for detecting the position of cross-sectional change. In this embodiment, as an example, the value is set to 20% of the difference, but it may be appropriately changed to an appropriate coefficient that can more accurately capture the change in depth value according to the depth value distribution. In Figure 6, the value of 20% of the difference between the maximum and minimum values ​​in the depth value distribution is 2, so the threshold value for detecting the position of cross-sectional change is set to 2.

[0049] S82 is a process that selects the minimum depth value from the unexplored cells in the depth value distribution and sets all cells with that depth value to explored. In the case of the depth value distribution shown in Figure 6, the minimum depth value among the unexplored cells is 5, so all cells with a depth value of 5 are set to explored. Figure 9 shows the depth value distribution after the S82 process has been executed on the depth value distribution in Figure 6.

[0050] S83 represents the process of calculating the difference between the minimum depth value selected in S82 and the maximum value in the cross-section change position list. If the depth value distribution is as shown in Figure 6, then 5 is calculated as the difference between the minimum depth value of 5 and the maximum value of 10 in the cross-section change position list.

[0051] S84 represents a process that determines whether the difference calculated in S83 is greater than or equal to a threshold for detecting the cross-sectional change position and branches accordingly. If it is greater than or equal to the threshold, the process proceeds to S85; otherwise, it returns to the process in S82. Even if it is less than the threshold, if the process reaches the deepest part of the slice image generation area 41, the depth value of the deepest part is detected as the cross-sectional change position, and the process proceeds to S85. In the case of the depth value distribution shown in Figure 6, the difference 5 obtained in S83 is greater than or equal to the threshold 2 for detecting the cross-sectional change position, so the process proceeds to S85.

[0052] S85 represents the process of detecting the minimum depth value selected in S82 from the depth value distribution as the cross-sectional change position, and adding the depth position corresponding to the depth value of the cell in question to the cross-sectional change position list.

[0053] In S6 of Figure 5, similar to the first embodiment, it is determined whether the new cross-sectional change position detected in S81-85 is the deepest part of the three-dimensional shape model. If the deepest part has been reached, the process proceeds to S7. If the deepest part has not yet been reached, the process returns to S4.

[0054] In S7 of Figure 5, similar to the first embodiment, slice images are generated at all cross-sectional change locations in the cross-sectional change location list to which elements were added by the processing up to S6. For example, the cross-sectional change locations detected as slice image generation locations 41 of the three-dimensional shape model 40 shown in Figure 10 are faces (a), (b), (f), (d), and (e), with depth positions of 0, 5, 7, 9, and 10, respectively.

[0055] Figure 11 shows the slice images generated for these cross-sectional change positions. The series of slice images generated in this manner are stored in the external storage unit 32 or the like.

[0056] Multiple parallel cross-sectional images of a three-dimensional shape model can be efficiently generated by methods different from those of the first embodiment.

[0057] (Other embodiments) Embodiments of this disclosure also include a control program capable of executing the method described above, and a computer-readable recording medium storing the control program. As a recording medium for supplying the control program, for example, ROM, disks, external storage devices, etc., may be used. To give specific examples, as a computer-readable non-temporary recording medium, flexible disks, optical disks, magneto-optical disks, magnetic tapes, non-volatile memory such as USB memory, SSDs, etc., can be used.

[0058] Furthermore, this disclosure can also be implemented by supplying a program that implements one or more functions of the embodiments to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program. It can also be implemented by a circuit (e.g., an ASIC) that implements one or more functions.

[0059] Furthermore, this disclosure is not limited to the embodiments described above, and the components can be modified and implemented in practice without departing from the gist of the invention. Multiple components from each of the embodiments described above may also be combined as appropriate. Additionally, some components may be removed from the multiple components shown in each embodiment. Furthermore, components from different embodiments may also be combined as appropriate.

[0060] This embodiment includes the following configuration.

[0061] (Item 1) An image processing method for generating multiple parallel cross-sectional images from a three-dimensional shape model, An acquisition step of obtaining multiple values ​​indicating the distance from a predetermined surface to each part of the three-dimensional shape model, A determination step of determining the distance from the predetermined surface based on the plurality of values, The process involves generating an image of the cross-sectional image at the determined distance, In the acquisition step, the distance is detected as the distance at which the minimum value among the plurality of values ​​changes to a threshold or greater. An image processing method characterized by the following:

[0062] (Item 2) The image processing method according to item 1, characterized in that, in the acquisition step, the threshold is the standard deviation of the plurality of values.

[0063] (Item 3) The image processing method according to item 2, characterized in that, in the acquisition step, the threshold is a value based on the difference between the maximum and minimum values ​​of the plurality of values.

[0064] (Item 4) The image processing method according to item 2, characterized in that, in the acquisition step, the threshold is a value obtained by multiplying the difference between the maximum and minimum values ​​of the plurality of values ​​by a coefficient.

[0065] (Item 5) The image processing method according to any one of items 1 to 4, characterized in that, after the image generation step, a display step is performed to display the cross-sectional image generated in the image generation step.

[0066] (Item 6) The image processing method according to any one of items 1 to 4, characterized in that, after the image generation step, a storage step is performed to store the cross-sectional image generated in the image generation step.

[0067] (Item 7) An image processing apparatus having a processing unit that generates multiple parallel cross-sectional images from a three-dimensional shape model, The processing unit determines the distance from the predetermined surface based on a plurality of values ​​indicating the distance from the predetermined surface to each part of the three-dimensional shape model, and generates the cross-sectional image at the determined distance. The processing unit detects the distance as the distance at which the minimum value among the plurality of values ​​changes to a threshold value or more. An image processing apparatus characterized by the following:

[0068] (Item 8) The image processing apparatus according to item 7, characterized in that the threshold is the standard deviation of the plurality of values.

[0069] (Item 9) The image processing apparatus according to item 7, characterized in that the threshold value is a value based on the difference between the maximum and minimum values ​​of the plurality of values.

[0070] (Item 10) The image processing apparatus according to item 9, characterized in that the threshold is a value obtained by multiplying the difference between the maximum and minimum values ​​of the plurality of values ​​by a coefficient.

[0071] (Item 11) The image processing apparatus according to any one of items 7 to 10, further comprising a display unit for displaying the cross-sectional image.

[0072] (Item 12) The image processing apparatus according to any one of items 7 to 11, further comprising a storage unit for storing the cross-sectional image.

[0073] (Item 13) A program for causing a computer to perform the image processing method described in any one of items 1 through 6.

[0074] (Item 14) A computer-readable recording medium containing the program described in item 13. [Explanation of Symbols]

[0075] 31 CPU 40 Three-Dimensional Shape Models 41 Slice image generation location

Claims

1. An image processing method for generating multiple parallel cross-sectional images from a three-dimensional shape model, An acquisition step of obtaining multiple values ​​indicating the distance from a predetermined surface to each part of the three-dimensional shape model, A determination step of determining the distance from the predetermined surface based on the plurality of values, The process involves generating an image of the cross-sectional image at the determined distance, In the acquisition step, the distance is detected as the distance at which the minimum value among the plurality of values ​​changes to a threshold or greater. An image processing method characterized by the following:

2. The image processing method according to claim 1, characterized in that, in the acquisition step, the threshold is the standard deviation of the plurality of values.

3. The image processing method according to claim 1, characterized in that, in the acquisition step, the threshold is a value based on the difference between the maximum and minimum values ​​of the plurality of values.

4. The image processing method according to claim 3, characterized in that, in the acquisition step, the threshold is a value obtained by multiplying the difference between the maximum and minimum values ​​of the plurality of values ​​by a coefficient.

5. The image processing method according to claim 1, characterized in that, after the image generation step, a display step is performed to display the cross-sectional image generated in the image generation step.

6. The image processing method according to claim 1, characterized in that, after the image generation step, a storage step is performed to store the cross-sectional image generated in the image generation step.

7. An image processing apparatus having a processing unit that generates multiple parallel cross-sectional images from a three-dimensional shape model, The processing unit determines the distance from the predetermined surface based on a plurality of values ​​indicating the distance from the predetermined surface to each part of the three-dimensional shape model, and generates the cross-sectional image at the determined distance. The processing unit detects the distance as the distance at which the minimum value among the plurality of values ​​changes to a threshold value or more. An image processing apparatus characterized by the following:

8. The image processing apparatus according to claim 7, characterized in that the threshold is the standard deviation of the plurality of values.

9. The image processing apparatus according to claim 7, characterized in that the threshold value is a value based on the difference between the maximum and minimum values ​​of the plurality of values.

10. The image processing apparatus according to claim 9, characterized in that the threshold value is the value obtained by multiplying the difference between the maximum and minimum values ​​of the plurality of values ​​by a coefficient.

11. The image processing apparatus according to claim 7, further comprising a display unit for displaying the cross-sectional image.

12. The image processing apparatus according to claim 7, further comprising a storage unit for storing the cross-sectional image.

13. A program for causing a computer to execute the image processing method described in any one of claims 1 to 6.

14. A computer-readable recording medium storing the program described in claim 13.

Images