Image processing device, image processing method, and program

Abstract

The present invention detects that an operation designating a position within an image has been carried out, and displays an enlarged image of an area near said position. The present invention acquires information relating to an object depicted in the enlarged image and displays information relating to said object.

Description

[Technical field]

[0001] The present invention relates to an image processing device, an image processing method, program Regarding. [Background technology]

[0002] Many techniques for improving the user's operability for displayed information have been studied. Patent Document 1 discloses a technique for detecting an area of ​​an image to be enlarged and displayed in order to improve the operability for information input. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2000-250681 A Summary of the Invention [Problem to be solved by the invention]

[0004] In operations such as sending control instructions using displayed information, there is a demand for technology that improves operability in selecting an object shown in a captured image.

[0005] Therefore, the present invention provides an image processing device, an image processing method, program One of the aims is to provide [Means for solving the problem]

[0006] According to a first aspect of the present invention, an image processing device includes a detection means for detecting that a first operation for specifying a position within an image has been performed, a first display means for displaying an enlarged image of an area adjacent to the position, an acquisition means for acquiring information regarding an object appearing in the enlarged image, and a second display means for displaying information regarding the object.

[0007] According to a second aspect of the present invention, an image processing method detects that a first operation has been performed to specify a position within an image, displays an enlarged image of a range adjacent to the position, obtains information about an object appearing in the enlarged image, and displays the information about the object.

[0008] According to a third aspect of the present invention, a storage medium stores a program that causes a computer of an image processing device to function as a detection means for detecting that a first operation for specifying a position within an image has been performed, a first display means for displaying an enlarged image of an area adjacent to the position, an acquisition means for acquiring information about an object appearing in the enlarged image, and a second display means for displaying information about the object. Effect of the Invention

[0009] According to the present invention, it is possible to improve the operability in selecting an object shown in a captured image. [Brief description of the drawings]

[0010] [Figure 1] 1 is a diagram showing a schematic configuration of a control system including an image processing device according to an embodiment of the present invention. [Diagram 2] 1 is a functional block diagram of an image processing device according to an embodiment of the present invention. [Diagram 3] FIG. 11 is a diagram illustrating an example of display information. [Figure 4] 4 is a flowchart showing a flow of processing by the image processing device. [Diagram 5] FIG. 11 is a diagram showing another display example of an enlarged image. [Figure 6] 1 is a diagram illustrating a configuration of an image processing device according to an embodiment of the present invention. [Figure 7] 5 is a flowchart showing a processing flow by the image processing device according to the present embodiment. [Figure 8] FIG. 2 is a hardware configuration diagram of a control device according to the present embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] An image processing apparatus according to an embodiment of the present invention will now be described with reference to the drawings. 1 is a diagram showing a schematic configuration of a control system 100 including an image processing device according to this embodiment. The control system 100 includes an image processing device 1, a control target 2, and an imaging device 3. The image processing device 1 is communicatively connected to the control target 2 and the imaging device 3 via a communication network.

[0012] In the example shown in FIG. 1, a user operates an image processing device 1. The image processing device 1 controls a control target 2 based on the user's operation. The imaging device 3 is, for example, a camera, and captures the range of space in which the operation or state of the control target 2 is captured, the range of space in which the control target 2 operates, and the like. When the control target 2 is a robot arm, the imaging device 3 captures, for example, a range of an angle of view in which the moving state of an object grasped by the robot arm from the origin to the destination can be captured. The image processing device 1 acquires an image captured by the imaging device 3. The image processing device 1 is, for example, a device including a display having a display function, such as a tablet terminal, a personal computer, or a smartphone. The display may be, for example, a touch panel display.

[0013] FIG. 2 is a functional block diagram of the image processing device according to this embodiment. The image processing device 1 has the functions of a detection unit 11, a first display unit 12, an acquisition unit 13, a second display unit 14, and an identification unit 15. The image processing device 1 executes an image processing program. As a result, the image processing device 1 exerts the functions of the detection unit 11, the first display unit 12, the acquisition unit 13, the second display unit 14, and the identification unit 15.

[0014] The detection unit 11 detects that an operation of designating a position has been performed in a captured image acquired by the image processing device 1. For example, if the image processing device 1 has a touch panel display, the designation operation may be an operation of touching the touch panel display with a finger or the like to designate a position. Alternatively, if the image processing device 1 has a mouse as an input device, the designation operation may be an operation of moving a cursor displayed on the display of the captured image with the mouse and clicking on the position to be designated. The first display unit 12 displays an enlarged image of the area surrounding the specified position in the captured image. The acquisition unit 13 acquires information about a target shown in an enlarged image of a captured image (hereinafter, referred to as "target information"). The second display unit 14 displays the target information shown in the enlarged image. With the enlarged image displayed, the identification unit 15 identifies the specified position as the position selected in the captured image (hereinafter referred to as the "selected position"). For example, when the image processing device 1 is a touch panel display, the identification unit 15 may identify the position as the selected position by ending a touch action such as releasing a finger at a position to be specified with the enlarged image displayed. Also, when the image processing device 1 includes a mouse as an input device, with the enlarged image displayed, the cursor displayed on the display of the captured image may be moved with the mouse and clicked at the position to be specified, thereby identifying the position as the selected position.

[0015] The target information displayed by the second display unit 14 may be, for example, information indicating the distance from the imaging device 3 to the target shown in the captured image. Alternatively, the target information displayed by the second display unit 14 may be, for example, information regarding the normal direction of the surface of the target in the coordinate system of the space captured by the imaging device 3. The normal direction of the surface of the target is one aspect of information representing the surface. The target information may be information other than the normal direction representing the surface. Alternatively, the target information may be information regarding the temperature of the surface of the target. When the target information is information indicating the distance from the imaging device 3 to the target shown in the captured image, for example, the distance from the imaging device 3 to the target shown in the captured image may be detected by a TOF sensor or the like provided in the imaging device 3. When the target information is information regarding the normal direction of the surface of the target in the coordinate system of the space captured by the imaging device 3, for example, the normal direction of the surface of the target may be detected by a 3D modeling function provided in the imaging device 3. When the object information is information related to the temperature of a surface of the object, the temperature of the surface may be detected by a temperature sensor provided in the imaging device 3 or the like.

[0016] When the object information is information indicating the distance from the imaging device 3 to the object shown in the captured image, more specifically, the object information may be information obtained by dividing the enlarged image into a plurality of sections and indicating the distance between the imaging device 3 and the surface of the object on the pixel located at the center of each section. When the object information shown in the enlarged image is information regarding the normal direction of the surface of the object, more specifically, the object information may be information obtained by dividing the enlarged image into a plurality of sections and indicating the normal direction of the surface of the object on the pixel located at the center of each section. When the object information shown in the enlarged image is information regarding the temperature of the surface of the object, more specifically, the object information may be information obtained by dividing the enlarged image into a plurality of sections and indicating the temperature of the surface of the object on the pixel located at the center of each section. The division into a plurality of sections may be performed by pixels.

[0017] FIG. 3 is a diagram illustrating an example of the display information. At time T1, the image processing device 1 acquires a captured image D1 from the imaging device 3. The captured image D1 may be a moving image or a still image. The image processing device 1 displays the captured image D1 on a display. For convenience of explanation, the display is assumed to be a touch panel display, as an example. At time T2, the image processing device 1 detects that an operation (for example, a touch operation with a finger) is being performed at a certain position in the captured image D1. The image processing device 1 detects that position in the captured image D1. For example, while the image processing device 1 detects that an operation is being performed (for example, while a finger continues to touch the touch panel display), the image processing device 1 displays an enlarged image D2 in the vicinity (or surroundings) of the position where the operation is being performed. The enlarged image D2 is an image in which an area near the position where the operation is being performed in the captured image D1 displayed on the touch panel display is enlarged. It can also be said that the image processing device 1 displays the enlarged image D2 in association with the captured image D1. While displaying the enlarged image D2, the image processing device 1 detects, for example, that no operation is being performed (for example, a finger is removed from the touch panel display). The time at which this is detected is represented as "time T3." In this case, the image processing device 1 stops displaying the enlarged image D2. That is, the image processing device 1 deletes the enlarged image D2 from the display information of the display. Furthermore, in response to detecting, for example, that no operation is being performed, the image processing device 1 identifies the position at which the operation ended as the selected position.

[0018] According to such processing of the image processing device 1, the enlarged image D2 is displayed in association with the captured image D1, which improves operability in inputting instructions to an object shown in the captured image D1.

[0019] Next, the processing in the image processing device 1 will be described with reference to Fig. 4. Fig. 4 is a flowchart showing the flow of processing in the image processing device 1. The first display unit 12 displays the captured image D1 acquired from the imaging device 3 on the display (step S101). The detection unit 11 detects whether or not an operation (first operation) is being performed on the captured image D1 (step S102). When detecting that an operation is being performed on the captured image D1, the detection unit 11 identifies a position on the captured image D1 where the operation is being performed (step S103). The detection unit 11 calculates coordinates corresponding to the identified position in a coordinate system for the captured image D1. The detection unit 11 outputs coordinate information indicating the calculated coordinates to the first display unit 12.

[0020] The first display unit 12 inputs the coordinate information from the detection unit 11 and specifies a nearby range including the coordinates represented by the coordinate information (step S104). For example, the first display unit 12 specifies a predetermined range centered on the coordinates as the nearby range. The predetermined range is, for example, a range in which the size of a shape such as a rectangle, a circle, or an ellipse is determined in advance. The predetermined range may be expressed by the coordinates of the vertices of a rectangle, the radius of a circle, or the like. For convenience, the predetermined range is a circle having a predetermined size centered on the calculated coordinates. In this case, the first display unit 12 specifies the inside of the circle having a predetermined size centered on the calculated coordinates as the nearby range. The first display unit 12 generates an enlarged image D2 of the nearby range (step S105). The generated enlarged image D2 is displayed on the display (step S106). In this case, the first display unit 12 may display the enlarged image D2 on the display in a manner that the center of the enlarged image D2 is aligned with the specified position. After generating the enlarged image D2, the first display unit 12 may create coordinate information in which the coordinates of pixels in the enlarged image D2 are linked to the coordinates of pixels in the captured image D1.

[0021] In response to identifying the neighborhood range, the acquisition unit 13 acquires target information for a range including the neighborhood range (step S107). As described above, the target information is, for example, information such as the distance from the imaging device 3 to the target, the normal direction of the surface of the target, and the temperature of the surface of the target. The acquisition unit 13 acquires, for example, target information corresponding to each pixel included in the neighborhood range from the target information.

[0022] The process of acquiring the target information for the range including the neighboring range will be described in more detail. For example, the photographed image D1 has target information in which information at the coordinates is linked to the coordinates representing the position of each pixel of the photographed image D1. When a pixel represents an object, the target information includes information linking the position representing the pixel with information about the pixel's position (for example, the distance from the imaging device 3 to the target, temperature, etc.). As another method, the photographed image D1 may include information linking each of the sections to information about each section by dividing the photographed image D1 into a plurality of sections. For example, if the target information is information indicating the normal direction of the object, the distance from the imaging device 3 to the object may be acquired for the central pixel of each section and its surrounding pixels, the variation of the distance with respect to the variation of the pixel position may be calculated, and information indicating the normal direction may be calculated in advance and linked to the section of interest and stored in the photographed image D1. The acquisition unit 13 acquires information associated with the position of each pixel in the vicinity range from the target information. The acquisition unit 13 outputs the acquired target information to the second display unit 14. The target information may be acquired from the imaging device 3 separately from the captured image D1. Alternatively, the acquisition unit 13 may acquire the target information from a sensor measuring the target via a communication network. For convenience of explanation, the target information about the vicinity range is represented as "attention information." In other words, step S106, the acquisition unit 13 acquires attention information from the target information.

[0023] The acquisition unit 13 outputs the attention information to the second display unit 14. The second display unit 14 displays the acquired attention information in the enlarged image D2 (step S108). For example, it is assumed that the target information includes information representing the distance between the imaging device 3 and the target. In this case, the second display unit 14 acquires the attention information, which is information representing the distance, from the target information. The second display unit 14 displays the distance represented by the attention information in the pixels of the enlarged image D2, for example. The second display unit 14 may display the attention information in the form of a heat map, contour lines, or the like. Alternatively, the second display unit 14 may display the attention information in the form of a numerical value.

[0024] For example, the target information includes information representing the normal direction of a surface of the target. In this case, the second display unit 14 acquires attention information representing the normal direction from the target information and displays the acquired attention information. The second display unit 14 may display the normal direction by an arrow. Alternatively, the second display unit 14 may display the normal direction by a straight line.

[0025] For example, the target information includes information representing the temperature of a surface of the target. In this case, the second display unit 14 acquires attention information, which is information representing temperature, from the target information and displays the acquired attention information. The second display unit 14 may display the temperature in a manner represented by a numerical value or in a manner represented by a heat map.

[0026] For example, the second display unit 14 displays the enlarged image D2 and the attention information while detecting that an operation for specifying a position within the enlarged image is being performed.

[0027] When the detection unit 11 detects that no operation is being performed, it determines that the operation has ended. The detection unit 11 identifies the position when the operation ended (step S109). This process is an example of a process in which the identification unit 15 identifies information indicating the position specified by the second operation as a selected point in the image. Note that the second operation may be an operation of touching the enlarged image with a finger, or an operation of removing the finger from the touch panel display. The detection unit 11 outputs the identified position to the identification unit 15.

[0028] The identification unit 15 determines whether the identified position is within the range of the enlarged image D2 (step S110). When the identification unit 15 determines that the identified position is within the range of the enlarged image D2, the identification unit 15 identifies, as a selected point, coordinates linked to the coordinates of the position in the above coordinate information (i.e., coordinates in the captured image D1) (step S111). When the answer is No in step S110, the identification unit 15 identifies the coordinates representing the position as a selected point (step S112). Through the above processing, the processing for identifying the selected point selected by the user in the captured image D1 is completed.

[0029] The identification unit 15 may generate an instruction signal including coordinates indicating the selected point identified in the captured image D1. In this case, the identification unit 15 transmits the instruction signal to the control target 2. The control target 2 acquires the coordinates of the selected point in the captured image D1 included in the instruction signal. The control target 2 may execute a process of converting from the coordinate system in the captured image D1 to the spatial coordinate system of the control target 2. In this case, the control target 2 converts the coordinates of the selected point in the captured image D1 to coordinates in the spatial coordinate system of the control target 2.

[0030] According to the above-described processing of the image processing device 1, when the user inputs a selection point in the captured image D1, an enlarged image D2 of the vicinity of the position where the operation is being performed is displayed. Then, when the image processing device 1 detects that no operation is being performed, it recognizes the position where the operation is completed as the selection point. This allows the user to specify the selection point while viewing the enlarged image D2 of the vicinity of the position where the operation is being performed, improving operability. Furthermore, according to the above-described processing, attention information about the target is displayed in the enlarged image D2. This allows the user to specify the selection point while checking the attention information, improving operability.

[0031] Furthermore, the specification unit 15 may further include target information at the selection point as the instruction signal. For example, the target information may include information indicating the normal direction of the target. This allows the control target 2 to determine an appropriate approach to the target (for example, the hand angle when picking the target) using the information indicating the normal direction of the target. This allows the user to specify the selection point on the assumption that the target information will be notified to the control target 2, improving operability.

[0032] <Other embodiments> The second display unit 14 may specify a candidate position that is a candidate for the selection point based on the target information in the enlarged image D2, and display the specified candidate position in the enlarged image D2. For example, the second display unit 14 may specify at least a part of positions with little variation in the vicinity of the information representing the surface of the target as the candidate position. More specifically, the second display unit 14 specifies a range in which the difference in angle of the normal direction between adjacent pixels is less than a threshold value using the normal direction as a range with little variation in the normal direction, and specifies at least a part of this specified range as the candidate position. The second display unit 14 may create an enlarged image D2 that displays the color of the pixel corresponding to the candidate position in a manner different from the surroundings of the candidate position (i.e., a manner in which the candidate position can be identified), and display the created enlarged image D2. This makes it possible to detect a relatively flat position on the surface of the target. The candidate position is not limited to one position, and may be multiple positions. For example, when the target is an object to be operated (or grasped, etc.) by a robot such as a robot arm, and the image processing device 1 is an operation terminal that controls the operation of the robot, the process of identifying the candidate positions has the effect of enabling the robot to reliably perform the operation on the target, because the process of displaying the candidate positions can indicate locations where the robot can more reliably perform the operation.

[0033] The second display unit 14 may specify the candidate position based on the distance from the imaging device 3 to the target. For example, the second display unit 14 may specify the position of the target at the shortest distance from the target to be photographed as the candidate position. The second display unit 14 creates an enlarged image D2 that shows the candidate position in an identifiable manner, and displays the created enlarged image D2. For example, when the target is an object that is operated (or grasped, etc.) by a robot such as a robot arm, and the image processing device 1 is an operation terminal that controls the operation of the robot, the process of specifying the candidate position has the effect of allowing the robot to be operated so that the amount of movement of the robot is reduced. This is because the process of displaying the candidate position reduces the amount of movement of the robot when it moves to the candidate position. Also, for example, the second display unit 14 may specify a position with the smallest variation in distance to a nearby target as a candidate position. The second display unit 14 creates an enlarged image D2 that shows the candidate position in a distinguishable manner, and displays the created enlarged image D2. This allows a position with small variation in distance, i.e., a flat position, to be specified, and the process of specifying the candidate position allows a place where the robot can perform an operation more reliably to be indicated. For example, when the target is an object to be operated (or grasped, etc.) by a robot such as a robot arm, and the image processing device 1 is an operation terminal that controls the operation of the robot, the process of identifying the candidate position has the effect of allowing the robot to be operated so that the amount of movement of the robot is reduced. This is because the process of displaying the candidate position reduces the amount of movement of the robot when it moves to the candidate position.

[0034] The image processing device 1 may display warning information on the display based on the relationship between the position where the operation is completed and the candidate position. For example, the detection unit 11 calculates the distance between the position where the operation is no longer detectable and the candidate position. The detection unit 11 outputs the distance between the position where the operation is no longer detectable and the candidate position to the second display unit 14. When the second display unit 14 determines that the distance between the position where the operation is no longer detectable and the candidate position is equal to or greater than a predetermined distance threshold, it displays warning information indicating that the candidate position is far from the selection point. This enables the image processing device 1 to reduce operational errors when the user inputs a selection point in the captured image D1.

[0035] Next, another display example will be described with reference to Fig. 5. Fig. 5 is a diagram showing another display example of an enlarged image. In the above example, the second display unit 14 displays the enlarged image D2 on the display so that the center of the enlarged image D2 coincides with the specified selection point. However, the center of the enlarged image D2 does not have to coincide with the specified selection point. In this case, the second display unit 14 displays the enlarged image D2 so that, for example, the range in which the enlarged image D2 is displayed does not overlap with the selection point.

[0036] It is assumed that the image processing device 1 detects that an operation is being performed on the display while the enlarged image D2 is being displayed. The operation is assumed to be an operation of moving a position. In this case, the second display unit 14 may move the point p displayed in the enlarged image in accordance with the value of the operation being performed, according to the amount of movement and the direction of movement of the position where the operation is being performed. When the detection unit 11 detects that no operation is being performed, it detects the position in the enlarged image D2 where the point p is displayed in the enlarged image D2 at that timing. When it becomes impossible to detect that an operation is being performed, the detection unit 11 outputs the position in the enlarged image D2 where the point p is displayed in the enlarged image D2 to the identification unit 15. At that timing, the identification unit 15 determines that the coordinates of the captured image D1, which are associated with the coordinates of the position in the enlarged image D2 where the point is displayed in the enlarged image D2 and recorded in the above-mentioned coordinate information, are the selected point.

[0037] In the above description, the second display unit 14 performs processing to move the point p displayed in the enlarged image in accordance with the amount of movement and the direction of movement of the finger position in the enlarged image or the captured image in which the display position on the touch panel display does not move. However, the second display unit 14 may perform processing to move the enlarged image in accordance with the amount of movement and the direction of movement of the finger position. In other words, the second display unit 14 may perform processing to move the enlarged image in accordance with the movement of the finger position so that the point p is linked to the finger position and is located at the center of the enlarged image.

[0038] By such processing, the selection point can be specified while checking the position of the selection point in both the captured image D1 and the enlarged image D2.

[0039] Next, an image processing device will be described with reference to Fig. 6 and Fig. 7. Fig. 6 is a diagram showing the configuration of the image processing device according to this embodiment. Fig. 7 is a flowchart showing the processing flow by the image processing device according to this embodiment. The image processing device 1 includes a detection unit 11, a first display unit 12, an acquisition unit 13, and a second display unit . The detection unit 11 detects that an operation is being performed in the captured image D1 (step S201). The first display unit 12 creates an enlarged image D2 of the vicinity of the position pointed to on the image at which the operation is being performed, and displays the created enlarged image D2 (step S202). The acquisition unit 13 acquires attention information related to an object included in the enlarged image D2 (step S203). The second display unit 14 displays the acquired attention information (step S204). The detection unit 11 in Fig. 6 can be realized using a function similar to that of the detection unit 11 in Fig. 2. The first display unit 12 in Fig. 6 can be realized using a function similar to that of the first display unit 12 in Fig. 2. The acquisition unit 13 in Fig. 6 can be realized using a function similar to that of the acquisition unit 13 in Fig. 2. The second display unit 14 in Fig. 6 can be realized using a function similar to that of the second display unit 14 in Fig. 2.

[0040] (Hardware configuration example) An example of the configuration of hardware resources for implementing the image processing device 1 according to each embodiment of the present invention described above using one calculation processing device (information processing device, computer) will be described. However, the image processing device 1 may be physically or functionally implemented using at least two calculation processing devices. The image processing device 1 may also be implemented as a dedicated device.

[0041] 8 is a block diagram showing an example of a hardware configuration of a calculation processing device capable of realizing an image processing device according to each embodiment of the present invention. The calculation processing device 20 has a central processing unit (Central_Processing_Unit, hereinafter referred to as "CPU") 21, a volatile storage device 22, a disk 23, a non-volatile recording medium 24, and a communication interface (hereinafter referred to as "communication IF") 27. The calculation processing device 20 may be connectable to an input device 25 and an output device 26. The calculation processing device 20 can transmit and receive information to and from other calculation processing devices and communication devices via the communication IF 27.

[0042] The non-volatile recording medium 24 is a computer-readable medium, such as a compact disc or a digital versatile disc. The non-volatile recording medium 24 may also be a universal serial bus memory (USB memory), a solid state drive, or the like. The non-volatile recording medium 24 holds the program even without a power supply, and allows the program to be carried. The non-volatile recording medium 24 is not limited to the above-mentioned medium. Instead of the non-volatile recording medium 24, the program may be carried via the communication IF 27 and a communication network.

[0043] The volatile storage device 22 is computer-readable and can temporarily store data. The volatile storage device 22 is a memory such as a dynamic random access memory (DRAM) or a static random access memory (SRAM).

[0044] That is, when the CPU 21 executes a software program (computer program: hereinafter simply referred to as a "program") stored on the disk 23, it copies the program to the volatile storage device 22 and executes arithmetic processing. The CPU 21 reads data required for executing the program from the volatile storage device 22. When display is required, the CPU 21 displays the output result on the output device 26. When inputting a program from an external device such as another device connected for communication, the CPU 21 reads the program from the input device 25.

[0045] The CPU 21 interprets and executes the control programs (FIGS. 4 and 7) stored in the volatile storage device 22 that correspond to the functions (processing) represented by the units shown in FIG. 2 or FIG. 6. The CPU 21 executes the processing described in each of the above-mentioned embodiments of the present invention.

[0046] That is, in such a case, each embodiment of the present invention can be realized by the control program.Furthermore, each embodiment of the present invention can be realized by a computer-readable non-volatile recording medium on which the control program is recorded.

[0047] The program may be a program for implementing some of the above-mentioned functions, or may be a so-called differential file (differential program) that can implement the above-mentioned functions in combination with a program already recorded in the computer system. The present invention has been described above by taking the above-mentioned embodiment as an exemplary example. However, the present invention is not limited to the above-mentioned embodiment. That is, the present invention can be applied in various aspects that can be understood by a person skilled in the art within the scope of the present invention. [Explanation of symbols]

[0048] 1. Image processing device 2. Control target 3. Imaging device 100 Control system 11. Detection section 12...First display section 13... Acquisition part 14...Second display section 15...Specific section

Claims

1. A detection means for detecting that a first operation for specifying a position within an image has been performed; a first display means for displaying an enlarged image of a range in the vicinity of the position; an acquisition means for acquiring information representing a surface of an object at a predetermined position of the object as information regarding the object shown in the enlarged image; A second display means for displaying information representing a surface of the object; An image processing device comprising:

2. The second display means displays information about the object within the enlarged image. The image processing device according to claim 1 .

3. The acquisition means further acquires, as the information about the target, information indicating a distance from an imaging device that captures the image to a predetermined position of the target, The second display means displays information indicating the distance.

3. The image processing device according to claim 1 or 2.

4. The information representing the surface is information indicating a normal direction of the surface. The image processing device according to any one of claims 1 to 3.

5. The second display means displays information about the object while accepting a second operation for designating a position within the enlarged image. The image processing device according to any one of claims 1 to 4.

6. The second display means displays a candidate position for the second operation identified based on information about the target in the enlarged image. The image processing device according to claim 5 .

7. a specifying means for detecting the second operation and specifying information indicating a position designated by the second operation as a selected point in the image; Equipped with The specifying means further transmits an instruction signal including coordinates indicating the selected point; The indication signal further includes information regarding an object at the selection point. The image processing device according to claim 5 .

8. Detecting that a first operation has been performed to specify a location within the image; Displaying an enlarged image of a region in the vicinity of the position; acquiring information representing a surface of an object at a predetermined position of the object as information regarding the object shown in the enlarged image; Display information representing the surface of the object Image processing methods.

9. The computer of the image processing device A detection means for detecting a first operation of specifying a location within an image; a first display means for displaying an enlarged image of a range in the vicinity of the position; an acquisition means for acquiring information representing a surface of an object at a predetermined position of the object as information regarding the object shown in the enlarged image; A second display means for displaying information representing a surface of the object; A program that functions as a

Images