Feature point extraction device, feature point extraction method, and posture estimation device

Abstract

Provided is a technology capable of improving the accuracy of estimating the posture of a target object. An object state estimation unit 17 estimates an object state indicating a relative positional relationship of a target object with respect to another object or a person. An invisible area estimation unit 15 estimates invisible areas of the target object on the basis of design information of the target object and the estimated object state. On the basis of the estimated invisible areas, a visible area feature point extraction unit 16 extracts feature points from the areas of the target object other than the invisible areas, and stores the extracted feature points in a storage device 4.

Description

Feature point extraction device, feature point extraction method, and posture estimation device

[0001] The present invention relates to a feature point extraction device, a feature point extraction method, and a posture estimation device.

[0002] It is known that the posture of a subject can be estimated by selecting feature points in advance in a 3D CAD (3-dimensional computer-aided design) of the subject and then estimating the coordinates of the feature points in measurement data (RGBD) captured by a camera that correspond to the selected feature points.

[0003] Since the accuracy of posture estimation depends on the accuracy of estimating the coordinates of feature points in the measurement data, it is important to select 3D CAD feature points whose coordinates are easy to estimate. For example, if the selected 3D CAD feature points include feature points that are invisible when photographed with a camera, the accuracy of estimating the coordinates of the feature points in the measurement data will decrease.

[0004] As a system capable of efficiently identifying the orientation of a target object, Patent Document 1 discloses a system that generates virtual measurement data to correspond to actual measurement data from shape data defined for the target article, performs filter processing to pseudo-convert the virtual measurement data based on the measurement characteristics of a measuring instrument, extracts feature point data from the actual measurement data and the virtual measurement data, stores the feature point data extracted from the virtual measurement data as model data of the target article, and compares the feature point data of the actual measurement data with data obtained by coordinate-transforming the feature point data included in the model data, thereby calculating the position and orientation of the target article.

[0005] Japanese Patent Application Laid-Open No. 2021-18662

[0006] In the invention described in Patent Document 1, feature points are extracted from filtered virtual measurement data, thereby excluding feature points that are invisible due to the shadow of the target object itself.

[0007] However, the technology of Patent Document 1 does not exclude feature points in areas that are invisible to the surroundings due to the shadows of other objects or people. Therefore, for example, when the target object is attached to another device or when a person holds the target object, the accuracy of estimating the coordinates of the feature points of the measurement data decreases, and the accuracy of estimating the posture of the target object decreases.

[0008] Therefore, an object of the present invention is to provide a technique that can improve the accuracy of estimating the posture of a target object.

[0009] In order to solve the above-mentioned problems, one representative feature point extraction device of the present invention is a feature point extraction device that includes a control unit and a memory unit, in which the control unit estimates an object state that indicates the relative position of a target object with respect to other objects or people, estimates an invisible region of the target object based on design information of the target object and the estimated object state, extracts feature points of regions of the target object other than the invisible region based on the estimated invisible region, and stores the extracted feature points in the memory unit.

[0010] According to the present invention, a technique is provided that can improve the accuracy of estimating the posture of a target object.

[0011] Problems, configurations, and effects other than those described above will become apparent from the following description of the embodiments.

[0012] FIG. 1 is a block diagram showing an example of a schematic configuration of a feature point extraction device according to a first embodiment; FIG. 2 is a diagram showing an example of a state in which a target object is attached to another object such as a device; FIG. 3 is a diagram showing an example of a state in which a target object is held in a person's hand; FIG. 4 is a block diagram showing an example of a hardware configuration of a computing device; FIG. 5 is a diagram showing an example of object information stored in a storage device; FIG. 6 is a flowchart showing an example of feature point extraction processing according to the first embodiment; FIG. 7 is a flowchart showing an example of object state estimation processing according to the first embodiment; FIG. 8 is a block diagram showing an example of a schematic configuration of a feature point extraction device according to a second embodiment; FIG. 9 is a flowchart showing an example of object state estimation processing according to the second embodiment.

[0013] Hereinafter, an embodiment will be described with reference to the drawings.

[0014] Hereinafter, the feature point extraction device according to the first embodiment will be described with reference to FIGS.

[0015] FIG. 1 is a block diagram showing an example of a schematic configuration of a feature point extraction device according to a first embodiment.

[0016] The feature point extraction device of the first embodiment extracts feature points in a visible area from a 3D CAD of a target object, excluding areas that are invisible to the camera due to other objects or people.

[0017] The feature point extraction device includes a design information input unit 7, an image input unit 8, and a calculation unit 10.

[0018] The arithmetic device 10 includes an object state estimation unit 17 , an invisible region estimation unit 15 , a visible region feature point extraction unit 16 , a feature point output unit 18 , and an object posture estimation unit 19 .

[0019] The arithmetic device 10 also includes a storage device 4. The storage device 4 stores object information 20. The object information 20 stores feature points for each object state.

[0020] The object state estimation unit 17 includes a target object detection unit 11, a person / other object detection unit 12, a relative position calculation unit 13, and a relative position classification unit .

[0021] The design information input unit 7 reads data related to the design information of the target object. The data related to the design information includes design information of the target object alone and assembly information. For example, as shown in FIG. 2 , the assembly information is relative relationship information between the target object 30 and the device 40 when the target object 30 is attached to another object such as a device 40.

[0022] The design information input unit 7 inputs design information of a single object to the invisible area estimation unit 15 and the visible area feature point extraction unit 16 , and inputs assembly information to the target object detection unit 11 and the person / other object detection unit 12 .

[0023] The target object detection unit 11 acquires the orientation of the target object from the assembly information input from the design information input unit 7, and inputs the orientation information of the target object to the relative position calculation unit 13. The orientation here is rotation information and translation information in the coordinate system defined in the assembly file.

[0024] The person / other object detection unit 12 estimates the posture of a person gripping a target object from the design information of the target object input from the design information input unit 7, and inputs the posture information of the person to the relative position calculation unit 13. Here, the person / other object detection unit 12 estimates the posture of a person gripping a target object by estimating parts of the target object that are likely to be gripped by a person from a 3D CAD of the target object. The person / other object detection unit 12 can also estimate a three-dimensional model of a hand gripping a target object from the shape of the target object, as shown in FIG. 3 , and input hand skeleton information to the relative position calculation unit 13.

[0025] Furthermore, the person / other object detection unit 12 acquires a 3D model of the other object by excluding the target object from the assembly information input from the design information input unit 7, acquires the orientation of the other object from the 3D model of the other object, and inputs the orientation information of the other object to the relative position calculation unit 13. The orientation here is rotation information and translation information in the coordinate system defined in the assembly file, similar to the orientation calculated by the target object detection unit 11.

[0026] The relative position calculation unit 13 estimates the relative orientation of the target object with respect to the person or other object based on the orientation information of the target object and the orientation information of the person or other object, and inputs the relative orientation to the relative position classification unit 14 .

[0027] Specifically, the posture of the person or other object defined in the coordinate system of the assembly file is converted into a coordinate system that defines the target object alone, thereby estimating the relative posture of the target object with respect to the person or other object.

[0028] The relative position classification unit 14 classifies the relative posture from the relative position calculation unit 13 into one of the object states based on the object information 20 stored in the storage device 4, and inputs the classified object state to the invisible area estimation unit 15.

[0029] Specifically, a plurality of relative orientations classified by a certain method are stored in the storage device 4, and the relative position classification unit 14 classifies the input relative orientations using the same method. As a method for classifying relative orientations, for example, there is a method using a support vector machine (SVM), which is one of classification methods using machine learning, to classify an arbitrary number of rotational components represented by quaternions and translational components represented by vectors as inputs.

[0030] Furthermore, if the object information 20 of the target object is not stored in the storage device 4 , the relative position classification unit 14 stores the object state of the target object estimated from the relative orientation in the object information 20 .

[0031] The object state indicates the relative positional relationship between the target object and a person or other object. For example, the state in which the target object 30 shown in Fig. 2 is attached to another object such as a device 40 is called object state A, and the state in which the target object 30 is held in a person's hand 50 is called object state B, as shown in Fig. 3. The invisible area when the target object is photographed by a camera differs depending on the object state.

[0032] For example, in object state A in Fig. 2, region 31 that is the shadow of target object 30 itself and region 32 that is hidden when target object 30 is combined with device 40 become invisible regions when target object 30 is photographed with a camera. Also, in object state B in Fig. 3, region 31 that is the shadow of target object 30 itself and region 33 of target object 30 that is held by a person's hand 50 during work become invisible regions when target object 30 is photographed with a camera.

[0033] Based on the design information of the target object itself from the design information input unit 7 and the object state from the relative position classification unit 14, the invisible area estimation unit 15 estimates an area of ​​the target object that is invisible in that object state, and inputs the estimated area to the visible area feature point extraction unit 16. Specifically, the area where the target object is hidden by a shadow of another object when viewed from the surroundings is defined as the invisible area.

[0034] For example, in Figures 2 and 3, area 31, which is the shadow of the target object 30 itself, is estimated to be an invisible area based on the design information of the target object alone from the design information input unit 7, and areas 32 and 33, which are the shadow of a person or other object, are estimated to be invisible areas based on the object state from the relative position classification unit 14.

[0035] One method for estimating invisible areas is to place three-dimensional models of the target object and people / other objects on 3DCG software, render the object using cameras uniformly placed around it, and define the parts of the rendered image that appear less than a threshold number of times as invisible areas.

[0036] The visible area feature point extraction unit 16 extracts feature points from areas other than the invisible area of ​​the target object based on the design information of the object itself from the design information input unit 7 and the invisible area from the invisible area estimation unit 15, and stores the coordinates of the extracted feature points in the object information 20 of the storage device 4.

[0037] One example of a feature point extraction method is to use SIFT (Scale-Invariant Feature Transform) to extract feature points from the texture of the object surface in the visible region, excluding the point cloud in the invisible region. This method can improve processing speed by downsampling the feature points, but rather than random sampling, it is preferable to use the FPS (Farthest Point Sampling) algorithm to sample so that the sum of the distances between point clouds is maximized. This is expected to improve matching accuracy when the obtained feature points are used for pose estimation.

[0038] The image input unit 8 acquires an image of the target object captured by a camera, and inputs the image to the feature point output unit 18 and the object posture estimation unit 19 .

[0039] The feature point output unit 18 estimates the object state of the target object from the relative posture of the target object and other objects or people in the image input from the image input unit 8, acquires feature points corresponding to the object state of the target object in the image from the object information 20 in the storage device 4, and inputs the acquired feature points to the object posture estimation unit 19.

[0040] Specifically, the object state of the target object in the image is determined from the relative position relationship between the target object and people or other objects obtained from RGB images or distance images taken by a camera, and information on feature points corresponding to the determined object state is obtained from object information 20 in storage device 4.

[0041] The object posture estimation unit 19 estimates the posture of the target object in the image by estimating the positions of feature points of the target object in the image input from the image input unit 8 that correspond to the feature points input from the feature point output unit 18. The posture of the target object in the image can be estimated, for example, by using the least squares method to find the correspondence between the coordinates of the feature points input from the feature point output unit 18 and the coordinates of the feature points estimated in the image, thereby finding the most likely posture of the target object.

[0042] FIG. 4 is a block diagram showing an example of the hardware configuration of the arithmetic device 10.

[0043] 1 shows an example of the hardware configuration of a computing device 10 in which the functional units shown in Fig. 1 operate. The computing device 10 is a server or computer having a CPU (Central Processing Unit) 1, a ROM (Read-Only Memory) 2, a RAM (Random Access Memory) 3, a storage device 4, an input / output device 5, and a communication device 6.

[0044] The CPU 1 (control unit) is a central processing unit, and implements necessary functions by executing programs stored in the ROM 2 (or storage device 4).

[0045] The ROM 2 is an auxiliary storage device that stores programs, and is composed of nonvolatile storage elements.

[0046] The RAM 3 is a main storage device used when the CPU 1 executes processing, and is composed of volatile storage elements.

[0047] The memory device 4 (memory unit) is an auxiliary memory device for storing input data provided to the CPU 1 and output data output from the CPU 1, and is composed of non-volatile memory elements such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive).

[0048] The communication device 6 is an interface used by the computing device 10 to communicate with external devices, and is configured by a network interface card (NIC) etc. The communication device 6 is connected to a network (e.g., the Internet) and communicates with external devices via the network.

[0049] The input / output device 5 is an interface that receives input from a user (operator) and is composed of a keyboard, a touch panel, a card reader, a voice input device, etc. The input / output device 5 is also an interface that outputs data to the operator and is composed of a display, a speaker, a printer, etc.

[0050] In this embodiment, each functional unit can realize each process described below by executing a program on a computing device 10 having a hardware configuration such as that shown in FIG.

[0051] FIG. 5 is a diagram showing an example of the object information 20 stored in the storage device 4. As shown in FIG.

[0052] The object information 20 stores, for each object state, the coordinates of the feature points of the visible area excluding the area that is invisible to the camera for the target object.

[0053] The object information 20 is made up of records each including an object state ID 21 , a feature point ID 22 , an X coordinate 23 , a Y coordinate 24 , and a Z coordinate 25 .

[0054] The object state ID 21 is an ID for identifying an object state. For example, a state in which the target object 30 shown in Fig. 2 is attached to another object such as a device 40 is defined as object state A, and a state in which the target object 30 is held by a human hand 50 as shown in Fig. 3 is defined as object state B, and the object state ID for object state A is set to 0, and the object state ID for object state B is set to 1.

[0055] The feature point ID 22 is an ID for identifying a feature point of the target object. The X coordinate 23, the Y coordinate 24, and the Z coordinate 25 are the coordinates of the feature point.

[0056] 6 is a flowchart showing an example of feature point extraction processing in Example 1. This processing is realized by the CPU 1 executing programs stored in the ROM 2 that realize the design information input unit 7, the object state estimation unit 17, the invisible area estimation unit 15, and the visible area feature point extraction unit 16.

[0057] First, the design information input unit 7 reads data related to the design information of the target object, inputs the design information of the object itself to the invisible area estimation unit 15 and the visible area feature point extraction unit 16, and inputs assembly information to the target object detection unit 11 and the person / other object detection unit 12 (step S301).

[0058] The object state estimation unit 17 executes the object state estimation process shown in FIG. 7 , which will be described later, to acquire posture information of the target object and other objects or people based on the design information input from the design information input unit 7, calculate the relative position of the target object with respect to the other objects or people, estimate the object state of the target object, and input the object state to the invisible area estimation unit 15 (step S302).

[0059] The invisible area estimation unit 15 estimates the area of ​​the target object that is invisible in the object state based on the design information of the target object itself from the design information input unit 7 and the object state from the relative position classification unit 14, and inputs the area to the visible area feature point extraction unit 16 (step S303).

[0060] Next, the visible area feature point extraction unit 16 extracts feature points from areas other than the invisible area of ​​the target object based on the design information of the object itself from the design information input unit 7 and the invisible area from the invisible area estimation unit 15, and stores the coordinates of the extracted feature points in the object information 20 of the storage device 4 (step S304), thereby terminating this process.

[0061] According to the process of FIG. 6 , by obtaining feature points of an area excluding areas obscured by people or other objects from the design information of the target object, the accuracy of estimating the feature points of the target object in the image can be improved, and the accuracy of estimating the posture of the target object can be improved.

[0062] 7 is a flowchart showing an example of the object state estimation process in step S302. This process is realized by the CPU 1 executing a program that realizes the object state estimation unit 17 stored in the ROM 2.

[0063] First, the target object detection unit 11 acquires the orientation of the target object from the assembly information input from the design information input unit 7, and inputs the orientation information of the target object to the relative position calculation unit 13 (step S101).

[0064] The person / other object detection unit 12 estimates the posture of the person holding the target object from the design information of the target object input from the design information input unit 7, and inputs the posture information of the person to the relative position calculation unit 13 (step S102).

[0065] Next, the person / other object detection unit 12 acquires a 3D model of the other object by excluding the target object from the assembly information input from the design information input unit 7 (step S103).

[0066] Next, the person / other object detection unit 12 acquires the orientation of the other object from the 3D model of the other object, and inputs the orientation information of the other object to the relative position calculation unit 13 (step S104).

[0067] The relative position calculation unit 13 estimates the relative orientation of the target object with respect to the person or other object based on the orientation information of the target object and the orientation information of the person or other object, and inputs the relative orientation to the relative position classification unit 14 (step S105).

[0068] Based on the object information 20 stored in the storage device 4, the relative position classification unit 14 classifies the relative posture from the relative position calculation unit 13 into one of the object states, inputs the classified object state to the invisible area estimation unit 15 (step S106), and terminates this processing.

[0069] According to the process of FIG. 7, the relative orientation of the target object with respect to a person or other object is estimated from the design information, and the object state of the target object is estimated, thereby making it possible to estimate the area that will be invisible when the target object is photographed with a camera.

[0070] 8 is a flowchart showing an example of the posture estimation process according to the embodiment 1. This process is realized by the CPU 1 executing a program stored in the ROM 2 that realizes the image input unit 8, the feature point output unit 18, and the object posture estimation unit 19.

[0071] First, the image input unit 8 acquires an image of the target object, the posture of which is to be estimated, captured by a camera, and inputs the image to the feature point output unit 18 and the object posture estimation unit 19 (step S401).

[0072] The feature point output unit 18 estimates the object state of the target object from the relative posture of the target object and other objects or people in the image input from the image input unit 8 (step S402).

[0073] Next, the feature point output unit 18 acquires feature points corresponding to the object state of the target object in the image from the object information 20 in the storage device 4, and inputs the acquired feature points to the object posture estimation unit 19 (step S403).

[0074] The object posture estimation unit 19 estimates the position of the feature point of the target object in the image input from the image input unit 8 that corresponds to the feature point input from the feature point output unit 18, thereby estimating the posture of the target object in the image (step S404), and then ends this process.

[0075] According to the processing of FIG. 8, the posture of the target object in the image is estimated using feature points in an area excluding areas that are invisible in the object state of the target object in the image, so that the accuracy of estimating the posture of the target object can be improved even when the target object is attached to another object or is held by a human hand.

[0076] According to Example 1, by acquiring feature points of an area excluding areas that are obscured by people or other objects from the design information of the target object, the accuracy of estimating feature points of the target object in the image can be improved, and the accuracy of estimating the posture of the target object can be improved.

[0077] The feature point extraction device according to the second embodiment will be described below with reference to FIGS.

[0078] In the following description, the same components as those in the first embodiment are denoted by the same reference numerals, and differences will be mainly described. Points that are not specifically described are the same as those in the first embodiment.

[0079] The second embodiment differs from the first embodiment mainly in that feature points of an area, excluding areas that are obscured by people or other objects during work on the target object, are acquired from the work information of the target object.

[0080] For example, when using a wrench to remove a bolt fixed to a target object, there is an invisible area due to the shadow of the wrench. In Example 2, it is possible to extract feature points excluding such invisible areas during work that cannot be estimated from design information.

[0081] 9 is a block diagram showing an example of a schematic configuration of a feature point extraction device according to Example 2. The feature point extraction device shown in FIG. 9 includes a work information input unit 9 in addition to the configuration of the feature point extraction device according to Example 1 shown in FIG.

[0082] The work information input unit 9 reads data relating to work information of a target object, and inputs the read data to the target object detection unit 11 and the person / other object detection unit 12 .

[0083] The work information includes work manual information, work videos that record the work, etc. The work manual information is mainly photos that show the main points of the work, etc.

[0084] The target object detection unit 11 acquires the orientation of the target object from the image of the work information input from the work information input unit 9, and inputs the orientation information of the target object to the relative position calculation unit 13. The orientation here is rotation information and translation information in the coordinate system of the camera used to capture the image. If the work information is a work manual, the image of the work information is a photograph included in the work manual, and if the work information is a work video, the image is each frame of the video.

[0085] The person / other object detection unit 12 acquires the posture of the person working by human posture estimation from the image of the work information of the target object input from the work information input unit 9, and inputs the posture information of the person to the relative position calculation unit 13. The posture here is rotational information and translational information defined in the same coordinate system as the posture calculated by the target object detection unit 11.

[0086] In addition, the person / other object detection unit 12 obtains the posture of other objects excluding the target object by object posture estimation from the image of the work information input from the work information input unit 9, and inputs the posture information of the other objects to the relative position calculation unit 13.

[0087] The posture estimation by the target object detection unit 11 and the human / other object detection unit 12 may be posture estimation using feature points or markers, or posture information included in the work information may be acquired.

[0088] 10 is a flowchart illustrating an example of an object state estimation process according to Example 2. This process is realized by the CPU 1 executing a program that realizes the object state estimation unit 17 stored in the ROM 2.

[0089] In the process of FIG. 10, steps S201 to S203 are executed instead of steps S101 to S104 of FIG.

[0090] First, the target object detection unit 11 acquires the orientation of the target object from the image of the work information input from the work information input unit 9, and inputs the orientation information of the target object to the relative position calculation unit 13 (step S201).

[0091] The person / other object detection unit 12 obtains the posture of the person working by human posture estimation from each frame of the photo or video included in the work information of the target object input from the work information input unit 9, and inputs the person posture information to the relative position calculation unit 13 (step S202).

[0092] Next, the person / other object detection unit 12 obtains the orientations of other objects excluding the target object by object orientation estimation from each frame of the photo or video included in the work information input from the work information input unit 9, and inputs the orientation information of the other objects to the relative position calculation unit 13 (step S203), and proceeds to step S105. The processing from step S105 onwards is the same as in FIG. 7 .

[0093] According to Example 2, by obtaining feature points of an area from the work information of the target object, excluding areas that are obscured by people or other objects while working on the target object, the accuracy of estimating feature points of the target object in the image can be improved, and the accuracy of estimating the posture of the target object during work can be improved.

[0094] In each of the above-described embodiments, the functional block configurations are merely examples. Some functional configurations shown as separate functional blocks may be configured as an integrated unit, or a configuration shown in a single functional block diagram may be divided into two or more functions. Furthermore, some of the functions of each functional block may be provided by other functional blocks.

[0095] In the above-described embodiments and modifications, the program is stored in ROM 2, but the program may be stored in storage device 4. Furthermore, arithmetic device 10 may be provided with an input / output interface (not shown), and the program may be loaded from another device as needed via the input / output interface and a medium available to arithmetic device 10. Here, the medium refers to, for example, a storage medium detachable from the input / output interface, or a communication medium, i.e., a wired, wireless, optical, or other network, or a carrier wave or digital signal propagating through the network. Furthermore, some or all of the functions realized by the program may be realized by a hardware circuit or FPGA.

[0096] The above-described embodiments and modifications may be combined with each other. Although various embodiments and modifications have been described above, the present invention is not limited to these. Other embodiments conceivable within the scope of the technical concept of the present invention are also included within the scope of the present invention.

[0097] 1: CPU (Central Processing Unit) 2: ROM (Read Only Memory) 3: RAM (Random Access Memory) 4: Storage device 5: Input / output device 6: Communication device 7: Design information input section 8: Image input section 9: Work information input section 10: Arithmetic unit 11: Target object detection section 12: Person / other object detection section 13: Relative position calculation section 14: Relative position classification section 15: Invisible area estimation section 16: Visible area feature point estimation section 17: Object state estimation section 18: Feature point output section 19: Object posture estimation section 20: Object information 21: Object state ID 22: Feature point ID 23: X coordinate 24: Y coordinate 25: Z coordinate 30: Target object 31: Area cast in shadow by the target object 30 itself 32: Area cast in shadow by another object 33: Area of ​​human shadow 40: Device 50: Human hand

Claims

1. A feature point extraction device comprising a control unit and a memory unit, wherein the control unit estimates an object state indicating a relative positional relationship of a target object with respect to other objects or people, estimates an invisible region of the target object based on design information of the target object and the estimated object state, extracts feature points of regions of the target object other than the invisible region based on the estimated invisible region, and stores the extracted feature points in the memory unit.

2. A feature point extraction device according to claim 1, wherein the control unit estimates the posture of the target object and the posture of the other object or the posture of the person, estimates the relative posture of the target object with respect to the other object or the person based on the posture of the target object and the posture of the other object or the person, and estimates the object state based on the relative posture.

3. A feature point extraction device according to claim 2, wherein the control unit estimates the posture of the target object, and the posture of the other object or the posture of the person from the design information.

4. A feature point extraction device according to claim 3, wherein the control unit estimates the posture of the person by estimating the position at which the target object is grasped from the design information.

5. A feature point extraction device according to claim 3, wherein the design information includes assembly information between the target object and the other object.

6. A feature point extraction device according to claim 2, wherein the control unit estimates the posture of the target object, and the posture of the other object or the posture of the person from task information of the target object.

7. A feature point extraction device according to claim 6, wherein the work information includes at least one of a work video or a work manual for the target object.

8. A feature point extraction device according to claim 2, wherein the control unit stores the coordinates of the extracted feature points in the storage unit for each object state.

9. A feature point extraction method for a feature point extraction device having a control unit and a memory unit, wherein the control unit estimates an object state indicating the relative position of a target object with respect to other objects or people, estimates an invisible region of the target object based on design information of the target object and the estimated object state, extracts feature points of regions of the target object other than the invisible region based on the estimated invisible region, and stores the extracted feature points in the memory unit.

10. A posture estimation device comprising a control unit and a memory unit, wherein the control unit: estimates an object state indicating the relative positional relationship of a target object with respect to other objects or people; estimates an invisible area of ​​the target object based on design information of the target object and the estimated object state; extracts feature points of areas of the target object other than the invisible area based on the estimated invisible area; stores the extracted feature points in the memory unit; acquires an image of the target object; estimates the object state of the target object in the image; acquires feature points corresponding to the object state of the target object in the image from the memory unit; and estimates the positions of the feature points of the target object in the image that correspond to the feature points acquired from the memory unit, thereby estimating the posture of the target object in the image.

Images