Image processing apparatus and image processing method

Abstract

To provide a feature point detection technology that can suppress deterioration of accuracy of alignment between images due to change of appearance of a subject.SOLUTION: An image processing method comprises: detecting a feature point using density or a number depending on easiness of change of appearance of a subject from an area of the subject in an image; and outputting information associated with the detected feature point.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a technique for detecting feature points from an image of a subject. [Background technology]

[0002] When performing image processing using captured images captured by a camera, there is a process that is performed on the premise that images of the same subject match between captured images captured at different times. For example, in a process of extracting a foreground in a captured image by background subtraction, the shape of an object can be extracted with high accuracy by using images in which the same subject matches. In addition, when the relationship between multiple cameras is determined by camera calibration and the captured images of the multiple cameras are synthesized to generate a composite image, the camera parameters of each camera during the capture period must match the results of the calibration.

[0003] Patent Document 1 discloses a method for performing highly accurate registration of images captured at different times using feature points of subjects in the images. It is desirable for the feature points used for registration to be scattered within the image in order to perform well-balanced correction of the entire image. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent No. 4136044 Summary of the Invention [Problem to be solved by the invention]

[0005] When the appearance of a subject in an image to be aligned changes, or when the shape of the subject's shadow changes due to changes in sunlight or lighting, problems occur in which feature points used for alignment are lost or matched incorrectly. When a subject with a changing appearance occupies a wide area in an image, the conventional method of capturing subjects so that they are uniformly distributed has the problem that many feature points will not match, leading to a decrease in alignment accuracy. The present invention provides a feature point detection technology that makes it possible to suppress a decrease in alignment accuracy between images caused by changes in the subject's appearance. [Means for solving the problem]

[0006] One aspect of the present invention is characterized by comprising a detection means for detecting feature points from an area of ​​a subject in an image at a density or number corresponding to the susceptibility of the subject to changes in appearance, and an output means for outputting information related to the feature points detected by the detection means. Effect of the Invention

[0007] According to the present invention, it is possible to provide a feature point detection technique that can suppress a decrease in accuracy of alignment between images caused by a change in the appearance of a subject. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a block diagram showing an example of a system configuration. [Diagram 2] 3 is a flowchart of the operation of the image processing device 100. [Diagram 3] 3A is a diagram showing an example of a reference image, FIG. 3B is a diagram showing the result of dividing the reference image 300 of FIG. 3A into multiple tiles, and FIG. 3C is a diagram showing an example of the subject detection result for the reference image 300 of FIG. 3A. [Figure 4] 3 is a diagram showing an example of the configuration of a table generated by the area dividing unit 102. [Diagram 5] FIG. 1A is a diagram showing an example of a table in which a priority corresponding to each subject ID is registered, and FIG. 1B is a diagram showing an example of tiles to which priorities have been set. [Figure 6] FIG. 2 is a block diagram showing an example of the hardware configuration of a computer device applicable to the image processing device 100. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, the embodiments will be described in detail with reference to the attached drawings. Note that the following embodiments do not limit the invention according to the claims. Although the embodiments describe a number of features, not all of these features are essential to the invention, and the features may be combined in any manner. Furthermore, in the attached drawings, the same reference numbers are used for the same or similar configurations, and duplicated descriptions are omitted.

[0010] [First embodiment] First, a configuration example of a system according to this embodiment will be described with reference to the block diagram of Fig. 1. As shown in Fig. 1, the system according to this embodiment includes an imaging unit 107, an image input unit 106, a vibration correction unit 108, a feature point setting unit 109, and an image processing device 100.

[0011] First, the imaging unit 107 will be described. The imaging unit 107 captures a moving image and outputs an image of each frame in the moving image as a captured image, or captures a still image periodically or irregularly and outputs the still image as a captured image. Note that, although the number of imaging units 107 included in the system is one in FIG. 1, the system may include multiple imaging units 107.

[0012] Next, the image input unit 106 will be described. The image input unit 106 outputs the captured image output from the imaging unit 107 to the image processing device 100. Note that the image input unit 106 may output the captured image input from the imaging unit 107 to another device in addition to the image processing device 100. For example, the image input unit 106 may output the captured image input from the imaging unit 107 to a display device provided separately from the image processing device 100, and cause the captured image to be displayed on the display device.

[0013] Next, a description will be given of the feature point setting unit 109. The feature point setting unit 109 acquires information relating to feature points output from the image processing device 100, and outputs the acquired information to the vibration correction unit .

[0014] Next, the vibration correction unit 108 will be described. The vibration correction unit 108 performs a position adjustment process on the captured image output from the imaging unit 107 based on the information output from the feature point setting unit 109. For example, the vibration correction unit 108 detects corresponding feature points corresponding to the feature points indicated by the information output from the feature point setting unit 109 from the captured image (the subsequent image following the reference image) output from the imaging unit 107. Then, the vibration correction unit 108 performs position adjustment on the captured image using the correspondence between the feature points and the corresponding feature points.

[0015] Next, the image processing device 100 will be described. A computer device such as a PC (personal computer), a smartphone, or a tablet terminal device can be applied to the image processing device 100. The image processing device 100 uses a captured image used as a reference among the captured images input from the image input unit 106 as a reference image, and detects feature points from the area of ​​the subject in the reference image at a density according to the tendency of the subject to change in appearance. The "change in the subject's appearance" also includes changes in the position and posture of the subject. The image processing device 100 outputs information related to the detected feature points to a feature point setting unit 109. Each functional unit in the image processing device 100 will be described later.

[0016] It should be noted that various modifications of the configuration shown in Fig. 1 are possible. For example, the image processing device 100 may be implemented by a plurality of computer devices. Also, for example, the image processing device 100 may include an image input unit 106, a feature point setting unit 109, and a vibration correction unit 108. Also, for example, the image processing device 100 may be an imaging device including an imaging unit 107, an image input unit 106, a feature point setting unit 109, and a vibration correction unit 108.

[0017] Next, the operation of the image processing device 100 will be described with reference to the flowchart of Fig. 2. In step S201, the control unit 110 judges whether or not the input of the reference image has been completed. For example, the control unit 110 judges that "the input of the reference image has been completed" when a captured image to be used as a reference image is stored in the data storage unit 105. The captured image to be used as a reference image may be, for example, the captured image first stored in the data storage unit 105, or may be a captured image selected from a group of captured images captured by the imaging unit 107 in response to a user operation.

[0018] An example of a reference image is shown in Fig. 3(a). The reference image 300 shown in Fig. 3(a) includes, as subjects, a structure 305, a pillar 308, a partition 301, a person 306, lines 312-313, pillars 302-304, shadows 309-311, and a shadow 307. The partition 301 is a structure including a transparent acrylic plate and pillars 302-304 supporting the transparent acrylic plate. The shadows 309-311 are the shadows of the pillars 302-304, respectively, and the shadow 307 is the shadow of the structure 305.

[0019] In the reference image 300, the pillars 302-304, the structure 305, the pillar 308, and the lines 312-313 are fixed and are assumed to have almost no change in position or shape. On the other hand, the position or shape of the person 306, the shadows 309-311, and the shadow 307 may change. In addition, the partition 301 is determined to have a change in shape because the outline shape of the object reflected in the acrylic panel changes due to reflection or glare.

[0020] If the control unit 110 determines that the input of the reference image is complete, the process of step S202 is executed, and if the control unit 110 does not determine that the input of the reference image is complete, the process of step S201 is executed again.

[0021] In step S202, the control unit 110 outputs an instruction to execute a process of detecting a contour line (edge) of the subject from the reference image to the subject determination unit 101. Upon receiving the instruction, the subject determination unit 101 executes a process of detecting a contour line (edge) of the subject from the reference image stored in the data holding unit 105.

[0022] In step S203, the control unit 110 outputs an instruction to execute a process of detecting a subject from a reference image to the subject determination unit 101. Upon receiving the instruction, the subject determination unit 101 executes a process of detecting a subject from a reference image stored in the data holding unit 105. In the subject detection process, the type (attribute) of the subject is also detected.

[0023] The method of detecting a subject by the subject determination unit 101 is not limited to a specific detection method. For example, the subject determination unit 101 may detect a learned subject from a reference image using a trained model that has previously been machine-learned to learn subjects such as people, equipment, and structures. Also, for example, the subject determination unit 101 may detect a subject using template matching, or may detect a subject using information such as the color of the subject.

[0024] In this embodiment, the object determination unit 101 determines a rectangular (bounding box) region that contains the object in the reference image as the object region, and determines the object region as the detection result of the object. An example of the object detection result for the reference image 300 in FIG. 3(a) is shown in FIG. 3(c).

[0025] Fig. 3(c) shows that a bounding box 322 of the person 306 is obtained as a detection result of the person 306, and a bounding box 321 of the shadow 307 is obtained as a detection result of the shadow 307. Fig. 3(c) also shows that a bounding box 320 of the structure 305 is obtained as a detection result of the structure 305, and a bounding box 326 of the partition 301 is obtained as a detection result of the partition 301. Fig. 3(c) also shows that bounding boxes 323-325 of the shadows 309-311 are obtained as detection results of the shadows 309-311, respectively.

[0026] However, the object determination unit 101 may determine for each pixel in the reference image whether or not the pixel is an object pixel, and may determine a set of the object's pixels as an object region of the object, and may use the object region as a detection result of the object.

[0027] Furthermore, the subject determination unit 101 may use a plurality of computing units to execute the process of step S202 and the process of step S203 in parallel. Furthermore, the subject determination unit 101 may further perform image processing for detecting a subject from a reference image. For example, the subject determination unit 101 may perform processing to calculate a background difference or a hysteresis of the reference image in order to detect a region that changes in the reference image as a subject region.

[0028] In step S204, control unit 110 determines whether or not both the process of step S202 and the process of step S203 have been completed. If control unit 110 determines that both the process of step S202 and the process of step S203 have been completed, the process of step S205 is executed. On the other hand, if control unit 110 determines that at least one of the processes of step S202 and step S203 has not been completed, the process of step S202 or step S203 that has not been completed is executed.

[0029] In step S205, the control unit 110 outputs an instruction to the area division unit 102 to execute a process of dividing the reference image into a plurality of tiles. Upon receiving the instruction, the area division unit 102 divides the reference image stored in the data holding unit 105 into a plurality of tiles. The area division unit 102 then generates information for managing tiles including a part or the whole of a subject area and the type of subject corresponding to the subject area.

[0030] FIG. 3(b) shows the result of dividing the reference image 300 in FIG. 3(a) into a plurality of tiles. In FIG. 3(b), dotted lines indicate the boundaries of the tiles, and in FIG. 3(b), the reference image 300 is divided into 5 (Y direction)×8 (X direction)=40 tiles. In the following, the positions of the tiles in the bottommost tile column (the 0th tile column from the bottom) are defined as X0Y0, X1Y0, X2Y0, X3Y0, X4Y0, X5Y0, X6Y0, X7Y0, starting from the leftmost tile. Therefore, the positions of the tiles in the nth (n=0 to 4)th tile column from the bottom are X0Yn, X1Yn, X2Yn, X3Yn, X4Yn, X5Yn, X6Yn, X7Yn, starting from the leftmost tile.

[0031] In this case, the area division unit 102 generates a table in which, for each detected object, an ID (object ID) indicating the type of the object and information indicating the tile to which the bounding box of the object belongs are associated and registered. An example of the configuration of the table generated by the area division unit 102 is shown in FIG.

[0032] In row 401, the subject ID = subject ID "person" of person 306, tile position = the position of the tile in the lower left corner among the tiles belonging to bounding box 322 of person 306, and tile size = (the number of tiles in the X direction belonging to bounding box 322, the number of tiles in the Y direction belonging to the bounding box) are registered. Similarly, in row 402, the (subject ID, tile position, tile size) of partition 301 is registered, and in row 403, the (subject ID, tile position, tile size) of structure 305 is registered. In addition, in row 404, the (subject ID, tile position, tile size) of shadow 307 is registered, and in rows 405 to 407, the (subject ID, tile position, tile size) of shadows 309 to 311 are registered, respectively.

[0033] For tiles in which no subject is detected, i.e. tiles that do not belong to the bounding box of the subject, the region division unit 102 assigns a unique ID "background" to the subject ID=background. Note that the subject ID may be any information that identifies the type of subject, and may be a different value for each type of subject, for example.

[0034] Next, in step S206, control unit 110 determines whether or not the process in step S205 is complete. If control unit 110 determines that the process in step S205 is complete, the process in step S207 is executed. On the other hand, if control unit 110 determines that the process in step S205 is not yet complete, the process in step S205 is continued.

[0035] In step S207, control unit 110 outputs an instruction to execute a process of setting the priority of each tile divided in step S206 to priority setting unit 103. Upon receiving the instruction, priority setting unit 103 sets a priority for each tile divided in step S206 corresponding to the subject ID of the tile.

[0036] For example, a table in which a priority corresponding to each subject ID is registered is prestored in the data storage unit 105. An example of the table is shown in Fig. 5(a). In the table of Fig. 5(a), the priority is expressed in three stages from 0 to 2, with 0 being the lowest priority and 2 being the highest priority.

[0037] In this embodiment, a lower priority is set for subjects whose appearance is more likely to change, and a higher priority is set for subjects whose appearance is less likely to change. In the example of FIG. 5(a), as shown in rows 501 and 502, subjects corresponding to subject ID=person and subject ID=shadow are most likely to change in appearance, so the lowest priority=0 is set. Also, as shown in row 503, subjects corresponding to subject ID=partition are second most likely to change in appearance, so the next lowest priority=1 is set. Also, as shown in rows 504 and 505, subjects corresponding to subject ID=structure and subject ID=background are least likely to change in appearance, so the highest priority=2 is set. Note that in a use case where the sky or a crowd is captured in the background, it is better to lower the priority of the background.

[0038] For example, the control unit 110 sets the priority level of 0 registered in the table of FIG. 5(a) to each tile specified by the tile position and tile size registered in row 401 of FIG. 4 as the priority level corresponding to the subject ID=person registered in row 401. The control unit 110 performs similar processing for rows 402 to 407. Note that the control unit 110 sets the priority level of 2 registered in the table of FIG. 5(a) to the tile to which the subject ID=background is assigned as the priority level corresponding to the subject ID=background. As a result, as shown in FIG. 5(b), the corresponding priorities are set for each tile in FIG. 3(b).

[0039] In step S208, control unit 110 determines whether or not the process of step S207 has been completed. If control unit 110 determines that the process of step S207 has been completed, the process of step S209 is executed. On the other hand, if control unit 110 determines that the process of step S207 has not been completed, the process of step S207 continues to be executed.

[0040] In step S209, the control unit 110 outputs an instruction to the feature selection unit 104 to execute a selection process for selecting feature points from the reference image. Upon receiving the instruction, the feature selection unit 104 divides the reference image into a plurality of small tiles. The size of each small tile is equal to or smaller than the size of tiles divided by the area division unit 102. The feature selection unit 104 then selects, from among the small tiles, a small tile that has the properties of a feature point as the selected small tile. For example, the feature selection unit 104 selects, as the selected small tile, a small tile that includes the contour detected in step S202.

[0041] In step S210, the feature selection unit 104 selects selected small tiles belonging to tiles with a priority of 1 or more as target small tiles to be subjected to the following process. The feature selection unit 104 then acquires feature points having an intensity greater than a threshold from the target small tiles. The intensity is a value that is maximum in an area of ​​the subject that becomes a feature point, such as a line or a corner, and is obtained by performing a predetermined calculation using a feature detector used in the feature point selection algorithm. The method for acquiring feature points from the target small tiles is not limited to a specific method. The feature selection unit 104 then selects feature points from the feature points acquired from the target small tiles such that "the distance between the feature points is equal to or greater than a distance D corresponding to the priority P of the tile that includes the target small tile." The distance D is preset to be smaller as the priority P increases and larger as the priority P decreases.

[0042] In other words, the feature selection unit 104 does not detect feature points from small tiles in tiles with a priority of 0, but detects feature points from small tiles in tiles with a priority of 1 or higher at a density according to the priority.

[0043] The feature selection unit 104 may adopt a method of defining a predicted value of a shape change of a subject according to the type of the subject, and adjusting the upper limit number of feature points to be acquired from the region in which each subject exists, using the area of ​​the region in which the subject exists and the occurrence rate of a shape change. The adjustment of the upper limit number of feature points according to the occurrence rate of a shape change of the subject can be performed by selecting feature points having sufficient strength from a large number of feature point candidates, and deleting features in the vicinity of the selected feature points.

[0044] Furthermore, the priority defined by the priority setting unit 103 may be reflected in the defined value of the distance used when extracting neighboring feature points to be deleted after a feature point is selected in step S210.

[0045] In this way, the feature selection unit 104 detects feature points from the region of the subject in the captured image at a density according to the tendency of the subject's appearance to change. More specifically, the feature selection unit 104 detects feature points from the region of the subject in the captured image at a higher density the lower the tendency of the subject's appearance to change, and detects feature points at a lower density the higher the tendency of the subject's appearance to change. Therefore, the feature selection unit 104 may adopt any method as long as it can realize such processing.

[0046] In step S211, the feature selection unit 104 determines whether the process of step S210 is complete. If the feature selection unit 104 determines that the process of step S210 is complete, the process of step S212 is executed. On the other hand, if the feature selection unit 104 determines that the process of step S210 is not yet complete, the process of step S210 continues to be executed.

[0047] In step S212, the control unit 110 stores in the data storage unit 105 information on the feature points selected for each target small tile by the process in step S211 (for example, the positions of the feature points in the reference image).

[0048] Note that control unit 110 may use the result of detecting the contour of the subject to limit the tiles to be referenced in the processing from step S203 onward. For example, in the example of Fig. 3(b), the contour of the subject is not detected from the tile at position X0Y4 or the tile at position X1Y4, so control unit 110 may exclude such tiles from the subsequent processing (detection of the subject, acquisition of feature points).

[0049] By using the contour line of the subject, partition 301 may be divided into pillars 302-304 and other members, and the priority of tiles including pillars 302-304 (tiles at positions X2Y2, X2Y3, X3Y2, X3Y3, X4Y2, and X4Y3 in the example of FIG. 3(b)) may be set to a higher priority than the priority of partition 301. Furthermore, tiles of partition 301 that do not have a contour line may also be set to a higher priority than the priority of partition 301, as they are tiles of the subject that are less likely to cause changes in appearance.

[0050] In addition, although an example of dividing an image into tiles has been described in this embodiment, the division unit is not limited to tiles. For example, it is also possible to divide an image into regions according to the shape of the subject using semantic segmentation or the like, and set a similar priority for each divided region.

[0051] In addition, the feature selection unit 104 may be configured to detect a greater number of feature points from the area of ​​the subject in the captured image if the subject's appearance is less likely to change, and detect a fewer number of feature points if the subject's appearance is more likely to change.

[0052] In addition, the feature selection unit 104 may increase the upper limit of the number of feature points to be detected from the area of ​​the subject in the captured image, the lower the likelihood of the subject's appearance changing, and decrease it, the higher the likelihood of the subject's appearance changing.

[0053] In this manner, in this embodiment, feature points are detected from the region of a subject in a captured image at a higher density as the subject's appearance is less likely to change, and at a lower density as the subject's appearance is more likely to change. As a result, even if the appearance of a subject in an image to be aligned changes, the number of feature points detected from such a subject is small or zero, and feature points are detected at a higher density from a subject whose appearance is less likely to change, thereby preventing problems such as feature points used for alignment being lost or erroneously matched.

[0054] [Second embodiment] An example of the hardware configuration of a computer device applicable to the image processing device 100 will be described with reference to the block diagram of Fig. 6. Note that the example of the hardware configuration of a computer device applicable to the image processing device 100 is not limited to the configuration shown in Fig. 6, and can be modified / changed as appropriate.

[0055] The CPU 611 executes various processes using computer programs and data stored in the RAM 613. As a result, the CPU 611 controls the operation of the entire image processing device 100, and executes or controls various processes performed by the image processing device 100, which will be described below.

[0056] The ROM 612 stores setting data for the image processing device 100, computer programs and data related to the startup of the image processing device 100, computer programs and data related to the basic operations of the image processing device 100, and the like.

[0057] The RAM 613 has an area for storing computer programs and data loaded from the ROM 612 or the auxiliary storage device 614, and an area for storing computer programs and data received from the outside via the communication unit 616. The RAM 613 further has a work area used when the CPU 611 executes various processes. In this way, the RAM 613 can provide various areas as appropriate.

[0058] The auxiliary storage device 614 is a large-capacity information storage device such as a hard disk drive or a solid state drive. The auxiliary storage device 614 stores an OS (operating system), computer programs and data for causing the CPU 611 to execute or control various processes described as processes performed by the image processing device 100, and the like. The computer programs and data stored in the auxiliary storage device 614 are loaded into the RAM 613 under the control of the CPU 611. The data holding unit 105 in FIG. 1 can be implemented using the RAM 613 or the auxiliary storage device 614.

[0059] The display unit 615 has a liquid crystal screen or a touch panel screen, and can display the results of processing by the CPU 611 as images, characters, etc. The display unit 615 may be a projection device such as a projector that projects images and characters.

[0060] The communication unit 616 performs data communication with an external device via a network such as a LAN or the Internet. For example, the communication unit 616 can receive a captured image output from the image input unit 106 and output information related to feature points to the feature point setting unit 109.

[0061] The operation unit 617 is a user interface such as a keyboard, a mouse, and a touch panel screen, and the user can operate it to input various information and instructions to the CPU 611. For example, the user can operate the operation unit 617 to select a captured image to be used as a reference image from a group of captured images captured by the imaging unit 107.

[0062] The CPU 611, ROM 612, RAM 613, auxiliary storage device 614, display unit 615, communication unit 616, and operation unit 617 are all connected to a system bus 618. In Fig. 6, the display unit 615 and operation unit 617 are devices included in the image processing device 100. However, this is not limiting, and for example, the display unit 615 and / or operation unit 617 may be devices capable of communicating with the image processing device 100 via a wired or wireless network.

[0063] In the first embodiment, a case has been described in which each functional unit (excluding the data storage unit 105) of the image processing device 100 shown in Fig. 1 is implemented by software (computer program). In the first embodiment, each functional unit (excluding the data storage unit 105) of the image processing device 100 shown in Fig. 1 has been described as the subject of processing, but in reality, the function of the functional unit is realized by the CPU 611 executing a computer program corresponding to the functional unit. Note that one or more of the functional units of the image processing device 100 shown in Fig. 1 may be implemented by hardware.

[0064] The numerical values, processing timing, processing order, processing subject, data (information) structure / acquisition method / send destination / send source / storage location, etc. used in each of the above embodiments are given as examples to provide a concrete explanation, and are not intended to be limited to these examples.

[0065] In addition, a part or all of the embodiments described above may be used in appropriate combination. In addition, a part or all of the embodiments described above may be used selectively.

[0066] (Other embodiments) The present disclosure can also be realized by a process in which a program for implementing one or more functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that implements one or more functions.

[0067] The invention of this specification includes the following image processing device, image processing method, and computer program. (Item 1) a detection means for detecting feature points from an area of ​​a subject in an image at a density or number corresponding to the tendency of the subject to change in appearance; an output means for outputting information relating to the feature points detected by the detection means; An image processing device comprising: (Item 2) 2. The image processing device according to item 1, wherein the detection means detects feature points from the region at a higher density the lower the susceptibility of the subject to changes in appearance, and detects feature points at a lower density the higher the susceptibility of the subject to changes in appearance. (Item 3) A lower priority is assigned to objects that are more likely to change in appearance, and a higher priority is assigned to objects that are less likely to change in appearance; The detection means detects feature points at a higher density from an area of ​​a subject having a higher priority and detects feature points at a lower density from an area of ​​a subject having a lower priority. 3. The image processing device according to item 2, (Item 4) 2. The image processing device according to item 1, wherein the detection means detects a greater number of feature points from the region when the subject's appearance is less likely to change, and detects a smaller number of feature points from the region when the subject's appearance is more likely to change. (Item 5) A lower priority is assigned to objects that are more likely to change in appearance, and a higher priority is assigned to objects that are less likely to change in appearance; The detection means detects a greater number of feature points from areas of subjects with higher priority and detects a smaller number of feature points from areas of subjects with lower priority. 5. The image processing device according to item 4, (Item 6) The image processing device described in any one of items 1 to 5, characterized in that the detection means divides the image into a plurality of regions and detects feature points from the region including the subject with a density or number according to the tendency of the subject to change in appearance. (Item 7) 7. The image processing device according to item 6, wherein the detection means does not detect feature points from areas that do not include the subject. (Item 8) moreover, 8. The image processing device according to any one of items 1 to 7, further comprising an imaging means for imaging a subject. (Item 9) moreover, 9. The image processing device according to any one of items 1 to 8, further comprising a positioning unit that performs positioning of the image and a subsequent image following the image based on feature points related to the information output by the output unit. (Item 10) An image processing method performed by an image processing device, comprising: a detection step in which a detection means of the image processing device detects feature points from an area of ​​a subject in an image with a density or number according to the tendency of the subject to change in appearance; an output step in which an output means of the image processing device outputs information related to the feature points detected in the detection step; An image processing method comprising: (Item 11) A computer program for causing a computer to function as each of the means of the image processing device according to any one of items 1 to 9.

[0068] The invention is not limited to the above-described embodiments, and various modifications and variations are possible without departing from the spirit and scope of the invention. Accordingly, the following claims are appended to apprise the public of the scope of the invention. [Explanation of symbols]

[0069] 100: Image processing device 101: Object determination unit 102: Area division unit 103: Priority setting unit 104: Feature selection unit 105: Data storage unit 106: Image input unit 107: Imaging unit 108: Vibration correction unit 109: Feature point setting unit 110: Control unit

Claims

1. a detection means for detecting feature points from an area of ​​a subject in an image at a density or number corresponding to the tendency of the subject to change in appearance; an output means for outputting information relating to the feature points detected by the detection means; An image processing device comprising:

2. 2. The image processing device according to claim 1, wherein the detection means detects feature points from the region at a higher density the lower the susceptibility of the subject to changes in appearance, and detects feature points at a lower density the higher the susceptibility of the subject to changes in appearance.

3. A lower priority is assigned to objects that are more likely to change in appearance, and a higher priority is assigned to objects that are less likely to change in appearance; The detection means detects feature points at a higher density from an area of ​​a subject having a higher priority and detects feature points at a lower density from an area of ​​a subject having a lower priority.

3. The image processing device according to claim 2.

4. 2. The image processing device according to claim 1, wherein the detection means detects a greater number of feature points from the region when the subject's appearance is less likely to change, and detects a smaller number of feature points from the region when the subject's appearance is more likely to change.

5. A lower priority is assigned to objects that are more likely to change in appearance, and a higher priority is assigned to objects that are less likely to change in appearance; The detection means detects a greater number of feature points from areas of subjects with higher priority and detects a smaller number of feature points from areas of subjects with lower priority.

5. The image processing device according to claim 4.

6. 2. The image processing apparatus according to claim 1, wherein said detection means divides said image into a plurality of regions, and detects feature points from the region including said subject with a density or number according to the tendency of the subject to change in appearance.

7. 7. The image processing apparatus according to claim 6, wherein said detection means does not detect feature points from an area that does not include the subject.

8. moreover, 2. The image processing apparatus according to claim 1, further comprising an image capturing unit for capturing an image of a subject.

9. moreover, 2. The image processing apparatus according to claim 1, further comprising a positioning unit that performs positioning of the image and a subsequent image that follows the image, based on feature points related to the information output by the output unit.

10. An image processing method performed by an image processing device, comprising: a detection step in which a detection means of the image processing device detects feature points from an area of ​​a subject in an image with a density or number according to the tendency of the subject to change in appearance; an output step in which an output means of the image processing device outputs information related to the feature points detected in the detection step; An image processing method comprising:

11. A computer program for causing a computer to function as each of the means of the image processing apparatus according to any one of claims 1 to 9.

Images