Information processing method, information processing device, and information processing program

Abstract

An information processing device acquires, in a two-dimensional plane on which a three-dimensional model of an object is projected, an input instruction for specifying a boundary plane component, which is a plane component for specifying a boundary of a three-dimensional cut-out region cut out from the three-dimensional model, and generates a three-dimensional provisional model demarcated by the boundary plane component in the three-dimensional model. A boundary normal component of the three-dimensional cut-out region is estimated by correcting the shape of the three-dimensional provisional model, the boundary normal component being a component in a normal direction orthogonal to the two-dimensional plane, and a region demarcated by the boundary normal component is cut out from the three-dimensional provisional model as the three-dimensional cut-out region.

Description

Technical Field

[0001] The present disclosure relates to techniques for manipulating 3-dimensional models of objects. Background Art

[0002] Patent document 1 discloses that in a 3D model generation device that generates a 3D model of a subject based on camera images of a close camera and a distant camera by using a background contour three-dimensional measurement method, the 3D model is generated by assuming that the subject exists outside the viewing angle range of the close camera.

[0003] After the 3D model is generated, a part of the 3D region desired by the user may be cut out from the 3D model according to the user's operation. In this case, if the user is required to specify the region from multiple directions, in addition to the plane component of the region boundary, the normal component of the boundary orthogonal to the plane is specified, the burden of the operation increases.

[0004] Prior Art Literature

[0005] Patent Literature

[0006] Patent Document 1: Japanese Patent Application Publication No. 2022-29730 Summary of the invention

[0007] The present disclosure has been made to solve such a problem, and an object of the present disclosure is to provide a technology capable of accurately cutting out a target three-dimensional cutout region from a three-dimensional model of an object without inputting an operation of designating a region from multiple directions.

[0008] An information processing method in one embodiment of the present invention is an information processing method in a computer, which obtains an input indication of a boundary plane component specified in a 2-dimensional plane onto which a 3-dimensional model of an object is projected, wherein the boundary plane component is a plane component of a boundary of a 3-dimensional cut-out area cut out from the 3-dimensional model; generates a 3-dimensional provisional model defined by the boundary plane component in the 3-dimensional model; estimates a boundary normal component of the 3-dimensional cut-out area by correcting the shape of the 3-dimensional provisional model, wherein the boundary normal component is a component of a normal direction orthogonal to the 2-dimensional plane; cuts out an area defined by the boundary normal component from the 3-dimensional provisional model as the 3-dimensional cut-out area; and outputs the 3-dimensional cut-out area.

[0009] According to the present disclosure, even without inputting an operation of designating a region from multiple directions, a target 3D cutout region can be accurately cut out from a 3D model of an object. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 This is a block diagram showing an example of the configuration of the information processing system in Embodiment 1 of the present disclosure.

[0011] Figure 2 This is a flowchart showing an example of processing by the information processing device in Embodiment 1.

[0012] Figure 3 This is a diagram showing an example of a display screen displayed on a display when a boundary plane component is designated.

[0013] Figure 4 It is shown Figure 2 This is a flowchart showing details of the processing for generating a three-dimensional provisional model shown in step S2.

[0014] Figure 5 This is a diagram for explaining the process of extracting points surrounded by vertices of a boundary plane component.

[0015] Figure 6 It is shown Figure 2 This is a flowchart showing details of the cut-out process shown in step S3.

[0016] Figure 7 FIG. 1 is a diagram showing an example of a display screen of a three-dimensional cutout region projected onto a two-dimensional plane.

[0017] Figure 8 This is a flowchart showing the details of the cutout process in the second embodiment.

[0018] Fig. 9 It is a diagram showing a plurality of object elements segmented from a three-dimensional provisional model. DETAILED DESCRIPTION

[0019] (Knowledge that forms the basis of this disclosure)

[0020] When an operator is working on an object at a work site, the operator may sometimes proceed with the work while confirming instructions from a remote person outside the work site. In this case, if the remote person can confirm which part of the object the operator is observing, the remote person can smoothly instruct the operator. To achieve this, for example, if a camera such as an action camera or smart glasses is mounted on the operator's head, and the image of the work site captured by the camera is sent in real time to a remote terminal owned by the remote person, the remote person can confirm which part of the object the operator is observing.

[0021] However, there are work sites where the output of images to the outside is prohibited from the viewpoint of safety. In this case, there is a problem that a remote person cannot confirm the part that the worker is observing.

[0022] Therefore, the inventors of the present invention have studied the following technology: reproducing a 3D model of an object in a virtual space, photographing the 3D model using a virtual camera synchronized with the position and posture of an operator on site, and displaying the obtained virtual camera image on a remote terminal of the remote person.

[0023] Furthermore, there is a need to cut out a part of the area from the 3D model reproduced in this way and observe it in detail. For example, a situation where a part of the components constituting the object is observed in detail. In this case, a method is considered in which the user inputs an operation to designate the area to be cut out from the image of the 3D model displayed on the display, and the designated area is cut out from the 3D model.

[0024] In the case of adopting this method, if the boundary is only specified from the image of the 3D model observed from a specific direction (for example, the front direction), the boundary in the normal direction orthogonal to the projection surface of the 3D model cannot be demarcated, and the area cannot be cut out in three dimensions. Therefore, a method is considered in which the 3D model observed from a direction different from the specific direction is displayed on a display, and the user is prompted to input an operation to specify the boundary in the normal direction line by line.

[0025] However, this method requires operations to specify boundaries from multiple directions, which increases the burden on the user. Furthermore, this method further increases the burden on the user if the shape of the normal direction of the cut-out region is complex.

[0026] Therefore, the inventor of the present invention obtained the following knowledge and finally came up with the present disclosure: if the boundary specified by the user is obtained as a boundary plane component based on an image of the 3D model observed from a specific direction, a 3D provisional model that tentatively represents the area to be cut out is generated based on the acquired boundary plane component, and the component of the normal direction of the boundary is estimated based on the 3D provisional model, the above-mentioned problem can be solved.

[0027] (1) An information processing method in one aspect of the present disclosure is an information processing method in a computer, the method comprising obtaining an input instruction for specifying a boundary plane component in a 2-dimensional plane onto which a 3-dimensional model of an object is projected, the boundary plane component being a plane component of a boundary of a 3-dimensional cut-out region cut out from the 3-dimensional model; generating a 3-dimensional provisional model defined by the boundary plane component in the 3-dimensional model; estimating a boundary normal component of the 3-dimensional cut-out region by correcting the shape of the 3-dimensional provisional model, the boundary normal component being a component of a normal direction orthogonal to the 2-dimensional plane; cutting out a region defined by the boundary normal component from the 3-dimensional provisional model as the 3-dimensional cut-out region; and outputting the 3-dimensional cut-out region.

[0028] According to this structure, if the user specifies the boundary plane component of the area to be cut out in the 2D plane onto which the 3D model of the object is projected, a 3D provisional model is generated based on the 3D model defined by the boundary plane component. Then, by correcting the shape of the 3D provisional model, the boundary normal component is estimated, and the area defined by the boundary normal component is cut out as the 3D cut-out area. Therefore, the user can cut out the 3D cut-out area by only inputting an input indication of the boundary of the 3D cut-out area when observing the 3D model from one direction. As a result, even without inputting an operation to specify the boundary from multiple directions, the 3D cut-out area as the target can be accurately cut out from the 3D model of the object.

[0029] (2) In the information processing method described in (1) above, the estimation of the boundary normal component may also include: acquiring 3D main data of the object; detecting points suitable for the 3D main data from the 3D provisional model by matching the 3D main data with the 3D provisional model; and estimating the normal component of the suitable point as the boundary normal component.

[0030] According to this configuration, by matching the 3D provisional model with the 3D master data, points suitable for the 3D master data are extracted from the 3D provisional model, and the suitable points are estimated as boundary normal components, so that the boundary normal components can be accurately estimated.

[0031] (3) In the information processing method described in (2) above, the matching may also include: detecting feature points of the 3D main data and feature points of the 3D provisional model; comparing feature quantities of the feature points of the 3D main data and feature quantities of the feature points of the 3D provisional model, detecting feature points of the 3D main data that match the feature points of the 3D provisional model; and detecting the feature points of the 3D provisional model in which the matching feature points can be detected as suitable points.

[0032] According to this configuration, since the boundary normal component is estimated by matching the feature amount of the three-dimensional master data with the feature amount of the three-dimensional provisional model, the boundary normal component can be estimated more accurately.

[0033] (4) In the information processing method described in (2) above, the three-dimensional master data may be three-dimensional CAD data of the object.

[0034] According to this configuration, since three-dimensional CAD data is used as three-dimensional master data, the boundary normal component can be estimated more accurately.

[0035] (5) In the information processing method described in (1) above, the estimation of the boundary normal component may also include: dividing the three-dimensional provisional model into a plurality of object elements constituting the object; determining one object element among the plurality of object elements that is located closest to the front side relative to the two-dimensional plane; and determining the boundary normal component based on a component of a normal direction of a boundary of the one object element.

[0036] In the case where the 3D model is composed of a plurality of object elements, the user inputs an input instruction for the boundary plane component while displaying the 3D model so that the object element to be cut out is displayed at the position closest to the front. According to this structure, the 3D provisional model is divided into a plurality of object elements, and the boundary normal component is determined based on the component of the normal direction of the boundary of the object element located at the position closest to the front among the divided plurality of object elements, so that the boundary normal component can be estimated with high accuracy.

[0037] (6) In the information processing method described in (5) above, the plurality of object elements may be divided by inputting the three-dimensional provisional model into an object identifier.

[0038] According to this configuration, the three-dimensional provisional model is divided into a plurality of object elements using the object identifier, so that the plurality of object elements can be divided accurately.

[0039] (7) In the information processing method described in (5) above, the plurality of object elements may be divided by clustering the three-dimensional provisional models.

[0040] According to this configuration, since the three-dimensional tentative model is divided into a plurality of object elements by clustering the three-dimensional tentative model, the division can be easily achieved.

[0041] (8) In the information processing method described in any one of (1) to (7) above, the boundary plane component may include a plurality of edges divided by a plurality of vertices, and the generation of the 3D provisional model may include: for each of a plurality of edge vectors corresponding to the plurality of edges, calculating the outer product of a focus point vector connecting a portion from a starting point of the edge vector to a focus point and the edge vector, wherein the focus point represents each point among all the points constituting the 3D model; extracting, from among all the points, focus points for which all the outer products calculated for each of the plurality of edge vectors are positive; and determining the extracted focus points as points of the 3D provisional model.

[0042] According to this configuration, it is possible to accurately identify points inside the boundary plane component and easily generate a three-dimensional provisional model.

[0043] (9) In another embodiment of the present invention, an information processing device includes a processor, which performs the following processing: obtaining an input indication of a boundary plane component specified in a 2-dimensional plane onto which a 3-dimensional model of an object is projected, wherein the boundary plane component is a plane component of a boundary of a 3-dimensional cut-out area cut out from the 3-dimensional model; generating a 3-dimensional provisional model defined by the boundary plane component in the 3-dimensional model; estimating a boundary normal component of the 3-dimensional cut-out area by correcting the shape of the 3-dimensional provisional model, wherein the boundary normal component is a component of a normal direction orthogonal to the 2-dimensional plane; cutting out an area defined by the boundary normal component from the 3-dimensional provisional model as the 3-dimensional cut-out area; and outputting the 3-dimensional cut-out area.

[0044] According to this configuration, an information processing device is provided that can accurately cut out a target three-dimensional cutout region from a three-dimensional model of an object without inputting an operation for designating a boundary from multiple directions.

[0045] (10) In another embodiment of the present invention, an information processing program causes a computer to perform the following processing: obtaining an input indication of a boundary plane component specified in a two-dimensional plane onto which a three-dimensional model of an object is projected, wherein the boundary plane component is a plane component of a boundary of a three-dimensional cut-out area cut out from the three-dimensional model; generating a three-dimensional provisional model defined by the boundary plane component in the three-dimensional model; estimating a boundary normal component of the three-dimensional cut-out area by correcting the shape of the three-dimensional provisional model, wherein the boundary normal component is a component of a normal direction orthogonal to the two-dimensional plane; cutting out an area defined by the boundary normal component from the three-dimensional provisional model as the three-dimensional cut-out area; and outputting the three-dimensional cut-out area.

[0046] According to this configuration, there is provided an information processing program capable of accurately cutting out a target three-dimensional cutout region from a three-dimensional model of an object without inputting an operation for designating a boundary from multiple directions.

[0047] The present disclosure can also be realized as an information processing system that operates according to such an information processing program. In addition, it is needless to say that such a computer program can be circulated via a computer-readable non-transitory recording medium such as a CD-ROM or a communication network such as the Internet.

[0048] In addition, the embodiments described below all illustrate a specific example of the present disclosure. The numerical values, shapes, constituent elements, steps, the order of steps, etc. shown in the following embodiments are examples, and the main purpose is not to limit the present disclosure. In addition, regarding the constituent elements in the following embodiments that are not recorded in the independent claims representing the highest concept, they are described as arbitrary constituent elements. In addition, the respective contents can also be combined in all embodiments.

[0049] (Implementation method 1)

[0050] Figure 1 This is a block diagram showing an example of the configuration of the information processing system in Embodiment 1 of the present disclosure.

[0051] The information processing device 1 includes a memory 11, a processor 12, a display 13, and an operation unit 14. The memory 11 stores a three-dimensional model and three-dimensional CAD data. The three-dimensional CAD data is an example of master data.

[0052] A 3D model is, for example, a model obtained by reproducing an object in a virtual space. A 3D model may also be composed of 3D point group data representing the shape of an object. A 3D model may also be composed of a 3D mesh model in which a surface is represented by multiple meshes by meshing the point group data. A 3D model may also be composed of 3D object data in which a texture image obtained by photographing the surface of the object is added to the 3D mesh data. The so-called virtual space refers to an imaginary 3D space constructed in a computer.

[0053] As the target object, for example, a manufactured object assembled by an operator in a factory can be used. An example of a manufactured object is an electronic product, a piece of iron, and a car. An example of an electronic product is a television, a refrigerator, a washing machine, etc. However, this is an example, and the target object may also be a device installed at a site where the operator performs the work. An example of a device is a manufacturing production line for manufacturing electronic products, cars, iron, etc.

[0054] The three-dimensional model is generated in advance by scanning the target object using a three-dimensional scanner or the like, for example, and is stored in the memory 11 .

[0055] The 3D CAD data is design data of a target object and is composed of data that three-dimensionally represents the shapes of a plurality of object elements constituting the target object and the arrangement relationship of each object element.

[0056] The processor 12 is constituted by, for example, a central processing unit (CPU) and includes an acquisition unit 121 , a provisional model generation unit 122 , an estimation unit 123 , a cutout unit 124 , and an output unit 125 .

[0057] The acquisition unit 121 acquires an input instruction for designating a boundary plane component in a 2D plane onto which the 3D model of the target object is projected, the boundary plane component being a plane component of a boundary of a 3D cut-out region cut out from the 3D model. The acquisition unit 121 generates a display screen of the 3D model by rendering the 3D model in the 2D plane, thereby projecting the 3D model onto the 2D plane, and displays the generated display screen on the display 13.

[0058] The user uses the operation unit 14 to input the boundary of the area that he wants to cut out from the projection image of the 3D model included in the display screen displayed on the display 13. The acquisition unit 121 acquires information representing the input boundary as an input instruction for specifying the boundary plane component. The information representing the boundary includes 2D coordinate data representing the position of the boundary in the 2D plane. Therefore, the input instruction includes 2D coordinate data representing the position of the boundary in the 2D plane. As the 2D coordinate data representing the position of the boundary, for example, the coordinate data of all points of the boundary can be used, or the coordinate data representing the position of the vertices of the boundary can be used. For example, if the boundary is a quadrilateral, the coordinate data of the four vertices of the quadrilateral are used as the coordinate data representing the position of the boundary.

[0059] The boundary plane component is composed of two-dimensional coordinate data indicating the position of the boundary. For example, when the input instruction is composed of coordinate data indicating the positions of four vertices of the boundary, the boundary plane component is composed of two-dimensional coordinate data indicating the sides of a quadrilateral surrounded by the four vertices.

[0060] The 2D plane is a surface set in the 3D virtual space where the 3D model is set. The position and posture of the 2D plane are configured to be changeable according to the position and posture of the virtual camera used when rendering the 3D model. The acquisition unit 121 acquires the user's instruction input to the operation unit 14 to change the position and posture of the virtual camera, and changes the position and posture of the 2D plane according to the acquired instruction. In this way, the user can display the projection image of the 3D model observed from any direction on the display 13.

[0061] The provisional model generation unit 122 generates a 3D provisional model defined by the boundary plane component in the 3D model. The 3D provisional model is composed of points whose positions on the 2D plane are located inside the boundary plane component among all the points constituting the 3D model. The 3D provisional model is a model in which the normal component orthogonal to the 2D plane is provisionally shown, and is a model in which the 3D cutout area is provisionally shown.

[0062] The estimating unit 123 estimates the boundary normal component of the three-dimensional cutout region by correcting the shape of the three-dimensional provisional model. The boundary normal component is a normal component orthogonal to the two-dimensional plane.

[0063] Here, the estimation unit 123 obtains the 3D CAD data of the target object from the memory 11, and detects points suitable for the 3D CAD data from the 3D provisional model by three-dimensionally matching the 3D CAD data and the 3D provisional model. Then, the estimation unit 123 estimates the normal components of the detected suitable points as the boundary normal components.

[0064] Matching includes the process of detecting the feature points of the 3D CAD data and the feature points of the 3D provisional model, and comparing the feature quantities of the feature points of the 3D CAD data and the feature quantities of the feature points of the 3D provisional model. Feature points are detected using algorithms such as SIFT (Scale-Invariant Feature Transform), SURF (SpeededUp Robust Features), and ORB (Oriented FAST and Rotated BRIEF). The feature quantities can be feature quantities corresponding to these algorithms, i.e., SIFT feature quantities, SURF feature quantities, or ORB feature quantities. In addition, feature points can also be feature points representing edges.

[0065] The matching process can be performed by, for example, the following method: comparing the feature quantities of the feature points of the 3D CAD data with the feature quantities of the feature points of the 3D provisional model, detecting the feature points that match the feature points of the 3D provisional model from the 3D CAD data, and detecting the feature points of the 3D provisional model from which the matching feature points can be detected as the suitable points. In the matching process, feature point matching can be performed by matching the feature points with the highest similarity using a nearest neighbor search or the like. The estimation unit 123 detects the normal components of the feature points of the 3D provisional model from which the corresponding points can be detected from the 3D CAD data as the boundary normal components of the 3D cut-out area.

[0066] The cutting unit 124 cuts out the area defined by the boundary normal component estimated by the estimating unit 123 from the 3D provisional model as a 3D cutting area. The 3D cutting area is a model cut out from the 3D model according to the input instruction of the user. The data of the 3D cutting area has the same structure as the data structure of the 3D model that is the cutting source.

[0067] The output unit 125 outputs the 3D cut-out region cut out by the cut-out unit 124. For example, the output unit 125 may store the 3D cut-out region in the memory 11. The output unit 125 may also output a display image obtained by projecting the 3D cut-out region onto a 2D plane to the display 13. Thus, the 3D cut-out region is displayed on the display 13.

[0068] Figure 2 1 is a flowchart showing an example of the processing of the information processing device 1 in Embodiment 1. In step S1 , the acquisition unit 121 acquires an input instruction for designating a boundary plane component from the operation unit 14 by the user. Figure 31 is a diagram showing an example of a display screen G1 displayed on the display 13 when specifying a boundary plane component. The display screen G1 displays a 3D model M1 projected onto a 2D plane. The 3D model M1 is a 3D model showing the internal structure of a TV set that is being assembled. The user operates the operation unit to input a boundary 300 on the display screen G1. Here, a quadrilateral frame enclosing an electronic component unit arranged on a substrate constituting the 3D model M1 is input as the boundary 300. If the user inputs an operation to change the line of sight in the display screen G1, the posture of the virtual camera is changed accordingly. In addition, if the user inputs an operation to change the magnification in the display screen G1, the position of the virtual camera is changed accordingly. Thus, the user can observe the 3D model M1 from any direction and position. In this example, the line of sight is set in the normal direction of the substrate of the 3D model M1, that is, in the front direction. In this case, the 2D plane is parallel to the substrate. Here, the boundary 300 is input so as to enclose the electronic component unit, but this is an example, and the user can specify any area within the 3D model M1. In addition, the boundary 300 can have any shape other than a quadrilateral, such as a triangle, a pentagon, a circle, an ellipse, a free closed curve, or the like.

[0069] Next, in step S2 , the provisional model generation unit 122 executes a three-dimensional provisional model generation process for generating a three-dimensional provisional model based on the boundary plane component indicated by the input instruction. The details of this process will be described later.

[0070] Next, in step S3, the estimation unit 123 and the cutting unit 124 execute a cutting process for cutting out a three-dimensional cutting region from the three-dimensional provisional model. The details of this process will be described later.

[0071] Next, in step S4 , the output unit 125 outputs the three-dimensional cutout region cut out by the cutout process. For example, the output unit 125 may display a display image obtained by projecting the three-dimensional cutout region onto a two-dimensional plane on the display 13 .

[0072] Figure 4 It is shown Figure 2 The flowchart of the details of the generation process of the three-dimensional provisional model shown in step S2 is shown in FIG. In step S11, the provisional model generation unit 122 determines a plurality of vertices of the boundary plane component. Figure 3 In the example of , the boundary 300 is a quadrilateral, so the four vertices constituting the quadrilateral are determined as the vertices of the boundary plane component. In addition, when the boundary 300 has a shape without vertices such as a free closed surface, the provisional model generation unit 122 can determine the key points on the boundary 300 as vertices. For example, the provisional model generation unit 122 can determine the points with large curvature changes as key points.

[0073] Next, in step S12 , points surrounded by the identified vertices are extracted from the three-dimensional model M1 . Figure 5 2 is a diagram for explaining the process of extracting points surrounded by vertices of boundary plane components. Figure 5 In , the X and Y axes are two-dimensional coordinate axes set in the two-dimensional plane on which the three-dimensional model M1 is projected. The Z axis is a coordinate axis in the normal direction orthogonal to the two-dimensional plane.

[0074] exist Figure 5 In , the boundary plane component is composed of a quadrilateral ABCD defined by vertices A, B, C, and D. The focus point pi represents each point among all the points constituting the three-dimensional model M1. In addition, when the three-dimensional model M1 is composed of a mesh model, the focus point pi is a vertex of the mesh.

[0075] The provisional model generation unit 122 calculates, for the plane component of the focus point pi, the outer product of the edge vectors corresponding to the respective sides of the quadrilateral ABCD and the focus point vectors connecting the portions from the starting points of the respective edge vectors to the focus point pi. In detail, the provisional model generation unit 122 calculates the outer product of the edge vector AB and the focus point vector Api, the outer product of the edge vector BC and the focus point vector Bpi, the outer product of the edge vector CD and the focus point vector Cpi, and the outer product of the edge vector DA and the focus point vector Dpi. The provisional model generation unit 122 extracts the focus point pi for which all the four obtained outer products are positive as points within the boundary plane component. On the other hand, the provisional model generation unit 122 deletes the focus point pi for which at least one of the four obtained outer products is equal to or less than 0 as a point outside the boundary plane component.

[0076] Next, in step S13, the provisional model generation unit 122 generates a 3D model composed of the remaining focus points pi as a 3D provisional model. Regarding the 3D provisional model, although the plane component is determined, the normal component orthogonal to the 2D plane is composed of tentatively determined points. The 3D provisional model is temporarily stored in the memory 11.

[0077] Figure 6 It is shown Figure 2 Flowchart of details of the cutting process shown in step S3 of FIG. 1 . In step S21 , the estimating unit 123 acquires the three-dimensional provisional model generated by the provisional model generating unit 122 from the memory 11 .

[0078] Next, in step S22 , the estimating unit 123 acquires the three-dimensional CAD data from the memory 11 .

[0079] Next, in step S23, the estimation unit 123 detects feature points from each of the 3D provisional model and the 3D CAD data. Feature points are detected using algorithms such as SIFT and ORB as described above. In addition, when the 3D model is composed of a mesh model, the 3D provisional model is also composed of a mesh model. Therefore, feature points are calculated for each vertex of the mesh that constitutes the 3D provisional model.

[0080] Next, in step S24 , the estimating unit 123 detects feature points of the 3D CAD data that match the feature points of the 3D provisional model by comparing feature amounts of the feature points of the 3D provisional model with feature amounts of the feature points of the 3D CAD data.

[0081] Next, in step S25 , the estimating unit 123 estimates normal components of feature points whose matching can be detected from the three-dimensional CAD data among the feature points of the three-dimensional provisional model as boundary normal components of the three-dimensional cutout region.

[0082] Next, in step S26, the cutting unit 124 extracts points surrounded by the boundary normal component from the 3D provisional model, and generates a 3D model composed of the extracted points as a 3D cutting area. Figure 2 Step S4.

[0083] Figure 7 is a diagram showing an example of a display screen G2 of a 3D cutout region M2 projected onto a 2D plane. The display screen G2 shows Figure 3 The 3D cut-out region M2 is generated by cutting out the region surrounded by the boundary 300 in the display screen G1 shown in FIG. In the display screen G2, the line of sight is set in the front direction of the 3D cut-out region M2, so the 3D cut-out region M2 observed from the front direction is displayed. The user can also set the line of sight in a direction intersecting the 3D cut-out region M2 and display the 3D cut-out region M2 on the display 13. Thus, the user can confirm the height shape of the 3D cut-out region M2.

[0084] As described above, according to the present embodiment, the user can cut out the 3D cut-out region M2 by simply inputting the boundary of the 3D cut-out region M2 when observing the 3D model M1 from one direction. As a result, the target 3D cut-out region M2 can be accurately cut out from the 3D model of the object without inputting the operation of specifying the boundary from multiple directions.

[0085] Furthermore, in this embodiment, by matching the 3D provisional model with the 3D CAD data, points suitable for the 3D main data are extracted from the 3D provisional model, and the set of suitable points is estimated as the boundary normal component, so that the boundary normal component can be accurately estimated.

[0086] (Implementation method 2)

[0087] Embodiment 2 cuts out a 3D cutout region M2 by dividing the 3D provisional model into a plurality of object elements. In addition, in this embodiment, the same reference numerals are given to the same components as those in Embodiment 1, and the description thereof is omitted. Figure 1 .

[0088] Reference Figure 1 The estimation unit 123 divides the 3D provisional model into a plurality of object elements. The plurality of components constituting the target object are equivalent to the object elements. For example, when the 3D provisional model is an electronic component unit, a circuit substrate, an integrated circuit arranged on the circuit substrate, a connector group arranged on the circuit substrate, a circuit element group such as a resistor arranged on the circuit substrate, etc. are equivalent to the object elements. However, this is an example, and the object elements may also be composed of any constituent elements of the target object.

[0089] The estimation unit 123 determines one object element that is arranged at the most front side relative to the two-dimensional plane among the plurality of object elements, and determines the boundary normal component based on the normal component of the boundary of the determined one object element. The so-called two-dimensional plane refers to the two-dimensional plane on which the three-dimensional model M1 is projected when the user specifies the boundary plane component. The so-called front side refers to the side where the virtual camera is arranged relative to the normal direction of the two-dimensional plane.

[0090] The estimation unit 123 divides the 3D provisional model into a plurality of object elements by inputting the 3D provisional model into an object identifier. The object identifier is an identifier generated by machine learning in order to identify predetermined object elements from the input 3D provisional model. For example, when the input 3D provisional model is an electronic component unit, the object identifier identifies object elements such as a circuit board, an integrated circuit, a connector group, and a circuit element group. The recognition result output by the object identifier includes position data indicating the 3D regions where the identified plurality of object elements are respectively arranged in the input 3D provisional model, and labels of the identified plurality of object elements.

[0091] Next, the processing of the information processing device 1 in the second embodiment is described. The main routine of the processing in the second embodiment is the same as that described in the first embodiment. Figure 2 The flowchart is the same as Figure 2 The details of the cutout processing in step S3 are different from those in the first embodiment, and therefore, the details of the cutout processing will be described below.

[0092] Figure 84 is a flowchart showing details of the cutout process in Embodiment 2. In step S41 , the estimating unit 123 acquires the three-dimensional provisional model generated by the provisional model generating unit 122 from the memory 11 .

[0093] Next, in step S42 , the estimation unit 123 inputs the three-dimensional tentative model into an object identifier to thereby divide the three-dimensional tentative model into a plurality of object elements. Fig. 9 is a diagram showing a plurality of object elements divided from a 3D provisional model. Fig. 9 As shown, the 3D tentative model is divided into a plurality of object elements B1 to B5. In this example, the object elements B1 to B5 are divided into five, but this is an example, and the 3D tentative model may be divided into 2 to 4 or more than 6 object elements. The number of divided object elements varies depending on the type of the input 3D tentative model and the number of object elements set as the recognition target by the object recognizer.

[0094] Next, in step S43, the estimation unit 123 specifies one object element that is located closest to the front side from among the divided object elements B1 to B5. Fig. 9 In the example, the direction in which the virtual camera 90 is arranged is set to the normal direction Z. The so-called closest to the front side refers to the side closest to the virtual camera 90 in the normal direction Z. Among the object elements B1 to B5, the object element B1 is arranged closest to the virtual camera 90 in the normal direction Z. Therefore, the object element B1 is determined as one object element.

[0095] Next, in step S44, the estimating unit 123 determines the normal component of one object element as the boundary normal component of the three-dimensional cutout region M2. Fig. 9 In the example of , the normal component of the object element B1 represented by the thick line E1 is determined as the boundary normal component.

[0096] Next, in step S45 , the cutting unit 124 extracts points surrounded by the boundary normal components from the three-dimensional provisional model, and generates a model composed of the extracted points as a three-dimensional cutting region.

[0097] As described above, according to the second embodiment, the 3D provisional model is divided into a plurality of object elements, and the boundary normal component is determined based on the normal component of the boundary of the object element located closest to the front among the divided plurality of object elements, so that the boundary normal component can be estimated with high accuracy.

[0098] The following modified examples of the present disclosure can be adopted.

[0099] (1) In Embodiment 2, the 3D tentative model is input to the object identifier to divide the object into multiple elements, but the present disclosure is not limited to this. The estimation unit 123 may divide the 3D tentative model into multiple object elements by applying clustering processing to the 3D tentative model. As the clustering processing, for example, hierarchical clustering may be used, or non-hierarchical clustering such as the k-means method may be used.

[0100] (2) In Figure 1 In the embodiment, the information processing device 1 is composed of an independently operated computer, but the present disclosure is not limited thereto, and the information processing device 1 may also be composed of a cloud server. In this case, the information processing device 1 is connected to the remote terminal via a network such as the Internet so as to be communicable. Further, in this case, the information processing device 1 only needs to obtain an input instruction for specifying a boundary plane component from the remote terminal. Further, in this case, the information processing device 1 only needs to send display data for displaying the display screens G1 and G2 on the remote terminal to the remote terminal.

[0101] (3) The 3D master data is composed of 3D CAD data, but the present disclosure is not limited thereto, and may be composed of any data as long as it is 3D data that serves as a reference for representing an object. For example, the 3D master data may be BIM (Building Information Modeling) data.

[0102] Industrial Applicability

[0103] The present disclosure is useful in the technical field of cutting out a region of interest from a 3D model.

Claims

1. An information processing method is an information processing method in a computer. obtaining an input indication specifying a boundary plane component in a 2D plane onto which a 3D model of the object is projected, the boundary plane component being a plane component of a boundary of a 3D cutout region cut out from the 3D model; generating a 3D provisional model defined by the boundary plane components in the 3D model; By correcting the shape of the three-dimensional provisional model, a boundary normal component of the three-dimensional cut-out region is estimated, wherein the boundary normal component is a component in a normal direction orthogonal to the two-dimensional plane; cutting out a region defined by the boundary normal component from the 3D provisional model as the 3D cut-out region; The 3-dimensional cut-out region is output.

2. The information processing method according to claim 1, wherein: The estimation of the boundary normal component includes: Acquiring 3D master data of the object; detecting points suitable for the 3D master data from the 3D provisional model by matching the 3D master data with the 3D provisional model; and The component of the normal direction of the appropriate point is estimated as the boundary normal component.

3. The information processing method according to claim 2, wherein: The matching includes: detecting feature points of the 3D master data and feature points of the 3D provisional model; comparing feature quantities of the feature points of the 3D master data with feature quantities of the feature points of the 3D provisional model, and detecting, from the 3D master data, feature points of the 3D master data that match the feature points of the 3D provisional model; and The feature points of the three-dimensional provisional model in which the matched feature points can be detected are detected as the suitable points.

4. The information processing method according to claim 2, wherein: The 3D master data is 3D CAD data of the object.

5. The information processing method according to claim 1, wherein: The estimation of the boundary normal component includes: dividing the three-dimensional tentative model into a plurality of object elements constituting the object; identifying one object element among the plurality of object elements that is disposed most forwardly with respect to the two-dimensional plane; and The boundary normal component is determined according to the component of the normal direction of the boundary of the one object element.

6. The information processing method according to claim 5, wherein: The plurality of object elements are divided by inputting the three-dimensional tentative model into an object identifier.

7. The information processing method according to claim 5, wherein: The plurality of object elements are divided by clustering the three-dimensional provisional model.

8. The information processing method according to claim 1, wherein: The boundary plane component includes a plurality of edges demarcated by a plurality of vertices, The generation of the 3D tentative model includes: For each of the plurality of edge vectors corresponding to the plurality of edges, calculating the outer product of a focus point vector connecting a portion from a starting point of the edge vector to a focus point and the edge vector, wherein the focus point represents each point among all the points constituting the three-dimensional model; Extracting, from all the points, the focus points for which all the outer products calculated by each of the edge vectors are positive; and, The extracted focus points are determined as points of the three-dimensional provisional model.

9. An information processing device comprising a processor, The processor performs the following processing: obtaining an input indication specifying a boundary plane component in a 2D plane onto which a 3D model of the object is projected, the boundary plane component being a plane component of a boundary of a 3D cutout region cut out from the 3D model; generating a 3D provisional model defined by the boundary plane components in the 3D model; By correcting the shape of the three-dimensional provisional model, a boundary normal component of the three-dimensional cut-out region is estimated, wherein the boundary normal component is a component in a normal direction orthogonal to the two-dimensional plane; cutting out a region defined by the boundary normal component from the 3D provisional model as the 3D cut-out region; The 3-dimensional cut-out region is output.

10. An information processing program that causes a computer to execute the following processing: obtaining an input indication specifying a boundary plane component in a 2D plane onto which a 3D model of the object is projected, the boundary plane component being a plane component of a boundary of a 3D cutout region cut out from the 3D model; generating a 3D provisional model defined by the boundary plane components in the 3D model; By correcting the shape of the three-dimensional provisional model, a boundary normal component of the three-dimensional cut-out region is estimated, wherein the boundary normal component is a component in a normal direction orthogonal to the two-dimensional plane; cutting out a region defined by the boundary normal component from the 3D provisional model as the 3D cut-out region; The 3-dimensional cut-out region is output.

Images