Image processing device, image processing method, and program

Abstract

To solve a problem such that intention of operation relative to a certain position image cannot be diverted to other position image.SOLUTION: An image processing device 1 includes: a response management unit 113 for storing two or more pieces of response information indicating correspondence between an operation sequence definition for defining a set of operation identifiers relative to two or more each operation and an intention identifier for identifying intention of the operation; a first output unit 141 for outputting a first image associated with position information; an operation sequence receiving unit 121 for receiving an operation sequence that is a set of operation identifiers for identifying each of two or more operation for the first image; an intention sequence acquisition unit 131 for acquiring an intention sequence corresponding to the operation sequence received by the operation sequence receiving unit 121 from the response management unit 113; a second output unit 142 for outputting a second image; and an operation unit 136 for performing operation on the intention sequence acquired by the intention sequence acquisition unit 131 on the second image. By the image processing device, intention of operation on the first image can be diverted to the second image, and as a result, the second image can be automatically operated according to intention of a user.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an image processing device or the like that performs processing on a position image, which is an image associated with position information such as a street view. [Background technology]

[0002] There have been technologies to improve the operability of Street View. More specifically, there have been Street View systems that improve operability by taking into consideration the landscape image to be displayed and the manner in which the mobile terminal device is used, and enable users to view landscape information from a desired viewpoint height when visiting a town on a field trip or the like (see Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 6390992 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in the conventional technology, the intention of an operation performed on a position image, which is an image associated with position information, cannot be applied to other position images. [Means for solving the problem]

[0005] The image processing device of the first invention is an image processing device including: a correspondence management unit in which two or more pieces of correspondence information indicating the correspondence between an operation sequence definition that defines a set of operation identifiers for each of two or more operations and an intention identifier that identifies the intention of the operation; a first output unit that outputs a first image that is an image corresponding to location information; an operation sequence receiving unit that receives an operation sequence that is a set of operation identifiers that identify each of two or more operations on the first image; an intention sequence acquisition unit that acquires from the correspondence management unit an intention sequence that is an intention identifier corresponding to each of one or more operation sequence definitions that match the operation sequence received by the operation sequence receiving unit; a second output unit that outputs a second image; and an operation unit that performs on the second image one or more operations that satisfy the operation sequence definition for the intention sequence acquired by the intention sequence acquisition unit.

[0006] With this configuration, the intention of the operation on the first image can be applied to the second image, so that the second image can be automatically operated according to the user's intention.

[0007] In addition, in the image processing device of the second invention, compared to the first invention, the correspondence management unit stores two or more pieces of first correspondence information indicating the correspondence between an operation sequence definition and a chunk identifier that identifies a chunk, which is a block of operations, and two or more pieces of second correspondence information indicating the correspondence between a chunk sequence definition, which is a definition of a chunk sequence, which is a set of one or more chunk identifiers, and an intention identifier, and the intention sequence acquisition unit acquires one or more chunk sequences, which are a set of one or more chunk identifiers corresponding to each of one or more operation sequence definitions that match the operation sequence accepted by the operation sequence receiving unit, from the two or more pieces of first correspondence information, and acquires one or more intention identifiers corresponding to the chunk sequence definition that matches the one or more chunk sequences from the two or more pieces of second correspondence information.

[0008] With this configuration, the intention of the operation on the first image can be applied to the second image, so that the second image can be automatically operated according to the user's intention.

[0009] In addition, compared to the second invention, the image processing device of the third invention further includes an attention feature determination unit that references a feature management unit in which two or more feature information pieces each having a feature identifier that identifies a feature, and two or more feature attribute values ​​including a feature category and location information of the feature, and determines an attention feature in the first image that satisfies an attention condition, using the location information corresponding to the first image; a category acquisition unit that acquires the category of the attention feature from the feature management unit; and a target feature determination unit that determines a target feature in the second image that is a feature of the same category as the category acquired by the category acquisition unit, and the image processing device is configured to perform, on the target feature, one or more operations that satisfy the operation sequence definition for each of the one or more intention identifiers acquired by the intention sequence acquisition unit.

[0010] With this configuration, the intention of the operation on the first image can be applied to the second image, so that the second image can be automatically operated according to the user's intention.

[0011] In addition, the image processing device of the fourth invention is an image processing device in which, compared to the third invention, the operation unit uses position information of the target feature to perform operations on the target feature so as to maintain a positional relationship with the target feature according to the intention identified by the intention identifier.

[0012] With this configuration, the intention of the operation on the first image can be applied to the second image, so that the second image can be automatically operated according to the user's intention.

[0013] In addition, the image processing device of the fifth invention is an image processing device in which, compared to the fourth invention, the operation sequence definition is one or more operation identifiers using regular expressions, the chunk sequence definition is one or more chunk identifiers using regular expressions, and the operation unit uses position information of the target feature to change the number of repetitions of the chunk identifier, or change the selection of the chunk identifier, or change the number of repetitions of the operation identifier, or change the selection of the operation identifier, so as to maintain a positional relationship with respect to the target feature according to the intention identified by the intention identifier, and performs an operation on the target feature.

[0014] With this configuration, the intention of the operation on the first image can be applied to the second image, so that the second image can be automatically operated according to the user's intention.

[0015] In addition, the image processing device of the sixth invention is an image processing device according to any one of the third to fifth inventions, further comprising an image determination unit which acquires a position image including a feature of the same category as the category acquired by the category acquisition unit from an image storage unit in which two or more position images are stored, and the second image is the position image acquired by the image determination unit.

[0016] With this configuration, the second image can be determined automatically.

[0017] Furthermore, the image processing device of the seventh invention is an image processing device in which, compared to the sixth invention, the image determination unit determines one or more second images which are one or more second images including a feature of the same category as the category acquired by the category acquisition unit, and which correspond to location information which satisfies a location condition for the location information corresponding to the first image, and the operation unit performs, on each of the one or more second images determined by the image determination unit, one or more operations which satisfy an operation sequence definition for each of the one or more intention identifiers acquired by the intention sequence acquisition unit.

[0018] With this configuration, the second image can be determined automatically.

[0019] In addition, the image processing device of the eighth invention is an image processing device including: a correspondence management unit in which two or more pieces of correspondence information indicating the correspondence between an operation sequence definition and an intention identifier are stored; an intention receiving unit that receives intention information including an intention sequence acquired in response to an operation on a first image; a second output unit that outputs a second image; and an operation unit that acquires an operation sequence definition for each of one or more intention identifiers contained in the intention sequence contained in the intention information accepted by the intention receiving unit, and performs one or more operations on the second image that satisfy each of the one or more operation sequence definitions.

[0020] With this configuration, it is possible to reuse the intention information that specifies the intention of the operation on the first image.

[0021] In addition, in the image processing device of the ninth invention, compared to the eighth invention, the intention information also includes a category of the feature of interest, and further includes a target feature determination unit that determines a target feature which is a feature in the second image and is of the same category as the category held by the intention information, and the operation unit is an image processing device that obtains an operation sequence definition for the intention sequence held by the intention information, and performs one or more operations on the target feature that satisfy each of one or more operation sequence definitions.

[0022] With this configuration, it is possible to reuse the intention information that specifies the intention of the operation on the first image.

[0023] Further, the image processing device of the tenth aspect of the present invention is an image processing device according to any one of the first to ninth aspects of the present invention, wherein the first image and the second image are different types of position images.

[0024] With this configuration, the intention of the operation for the first type of position image can be applied to the second type of position image.

[0025] In addition, the image processing device of the eleventh invention is an image processing device according to any one of the first to eighth inventions, wherein the first image and the second image are a street view image, a three-dimensional image, an indoor image, or a drive recorder image.

[0026] With this configuration, the intention of the operation on the first image can be applied to the second image, so that the second image can be automatically operated according to the user's intention. Effect of the Invention

[0027] According to the image processing device of the present invention, the intention of the operation on the first image can be applied to the second image, so that the second image can be automatically operated according to the user's intention. [Brief description of the drawings]

[0028] [Figure 1] Block diagram of image processing device 1 according to embodiment 1. [Diagram 2]1 is a flowchart illustrating a first operation example of the image processing device 1. [Diagram 3] A flowchart for explaining a first example of the intention sequence acquisition process [Figure 4] A flowchart for explaining a second example of the intention sequence acquisition process [Diagram 5] A flowchart for explaining an example of the attention feature determination process [Figure 6] A flowchart illustrating an example of the image determination process. [Figure 7] A flowchart illustrating an example of the image operation process. [Figure 8] A flowchart illustrating an example of the intention execution process. [Figure 9] 11 is a flowchart illustrating a second operation example of the image processing device 1. [Figure 10] A flowchart illustrating an example of the image operation process. [Figure 11] FIG. 1 is a diagram for explaining an example of the concept of the image processing device 1. [Figure 12] A diagram showing the correspondence table etc. [Figure 13] A diagram showing the screen image [Figure 14] A diagram explaining the process of acquiring the same intention sequence. [Figure 15] A diagram showing the correspondence table [Figure 16] A diagram showing the screen image [Figure 17] Block diagram of an image processing device 2 according to a second embodiment. [Figure 18] A flowchart for explaining an example of the operation of the image processing device 2. [Figure 19] A flowchart illustrating an example of the image operation process. [Figure 20] Overview of the computer system according to the above embodiment. [Figure 21] Block diagram of the computer system DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0029] Hereinafter, an embodiment of an image processing device and the like will be described with reference to the drawings. Note that, in the embodiments, components with the same reference numerals perform similar operations, so that repeated description may be omitted.

[0030] (Embodiment 1) In this embodiment, an image processing device is described that obtains an operation sequence, which is one or more operations on a first image that is an image associated with location information, obtains an intention sequence indicating a set of intentions of operations based on the operation sequence, and performs an operation on a second image according to the intention sequence. Note that the image processing device does not normally use the operation sequence on the first image as it is for the operation of the second image.

[0031] In addition, in this embodiment, an image processing device is described that obtains a chunk sequence indicating a set of one or more chunks from an operation sequence for a first image, obtains an intention sequence from the chunk sequence, and performs an operation on a second image according to the intention sequence.

[0032] In addition, in this embodiment, an image processing device will be described that acquires a feature of interest in a first image and performs an operation on a feature in a second image that has the same category as the feature of interest.

[0033] In addition, in this embodiment, an image processing device is described that manipulates the second image by making adjustments according to the attribute values ​​of the target features in the second image, rather than directly using the operation sequence for the first image to manipulate the second image.

[0034] In addition, in this embodiment, an image processing device that selects one or more second images and operates each of the one or more second images will be described. In this case, an image processing device that selects one or more second images that satisfy a position condition with respect to a first image will be described.

[0035] In addition, in this embodiment, an image processing device is described that selects one or more second images, compares each of the one or more second images with a first image, and operates each of the second images using an operation sequence for the first image.

[0036] Furthermore, in this embodiment, a case where the first image and the second image are different types of images will also be described.

[0037] In this specification, information X being associated with information Y means that information Y can be obtained from information X, or information X can be obtained from information Y, and the method of association is not important. Information X and information Y may be linked, may exist in the same buffer, information X may be included in information Y, or information Y may be included in information X, etc.

[0038] Furthermore, in this specification, determining information Z or selecting information Z means obtaining information Z, obtaining a pointer to information Z, obtaining an ID of information Z, setting a flag for information Z, etc., as long as information Z can be accessed.

[0039] 1 is a block diagram of an image processing device 1 according to the present embodiment. The image processing device 1 may be a standalone device, or may be a server that receives instructions and information from a terminal device (not shown) and transmits a position image to the terminal device. When the image processing device 1 is a standalone device, the image processing device 1 may be, for example, a personal computer, a smartphone, or a tablet terminal, but the type does not matter. When the image processing device 1 is a server, the image processing device 1 may be, for example, a cloud server or an ASP server, but the type does not matter.

[0040] The image processing device 1 includes a storage unit 11, a reception unit 12, a processing unit 13, and an output unit 14. The storage unit 11 includes a feature management unit 111, an image storage unit 112, and a correspondence management unit 113. The reception unit 12 includes an operation sequence reception unit 121. The processing unit 13 includes an intention sequence acquisition unit 131, a feature of interest determination unit 132, a category acquisition unit 133, an image determination unit 134, a target feature determination unit 135, and an operation unit 136. The output unit 14 includes a first output unit 141 and a second output unit 142.

[0041] Various types of information are stored in the storage unit 11 constituting the image processing device 1. The various types of information are, for example, feature information, a position image, and correspondence information, which will be described later. Note that the feature management unit 111, the image storage unit 112, and the correspondence management unit 113 may be present in an external server (not shown).

[0042] The feature management unit 111 stores two or more pieces of feature information. The feature information is information related to features. A feature is usually an object that exists on the ground, but may also include an object that exists indoors. A feature may also be called an object. Examples of features include buildings, parks, bridges, rivers, and mountains.

[0043] The feature information has a feature identifier and one or more feature attribute values. The feature identifier is information for identifying a feature. The feature identifier is, for example, the ID or name of a feature. The feature attribute value is an attribute value of the feature. It is preferable that the one or more feature attribute values ​​include a category and location information. The category is a type of feature. The categories are, for example, "pub", "school", "university", "conveyor belt sushi restaurant", "park", and "museum". It is preferable that the one or more feature attribute values ​​include two or more location information. It is preferable that the one or more feature attribute values ​​included in the feature information have area information for identifying an area of ​​the feature. The area information usually includes two or more location information. The location information is information for identifying a location. It is preferable that the location information is information for identifying a location in a three-dimensional space, but it may be information for identifying a location in two dimensions. The location information is, for example, (latitude, longitude, altitude), but may also be (latitude, longitude).

[0044] Two or more position images are stored in the image storage unit 112. A position image is an image associated with one or more pieces of position information. A position image is usually a photographed image. A position image is an image that is the target of an operation. The content of the operation does not matter.

[0045] The image associated with the location information may be an image including the location information. The location information is, for example, a street view, a three-dimensional image, an indoor image, or a drive recorder image. The three-dimensional image is, for example, an image of Google (registered trademark) Earth. The indoor image is an image taken inside a room. The indoor image is, for example, an image acquired by LiDAR. The drive recorder image is an image taken by a drive recorder installed in a moving object.

[0046] As described above, the image storage unit 112 may exist in an external server, not in the image processing device 1. The external server is, for example, a server that manages street view or a Google Earth server.

[0047] The correspondence management unit 113 stores two or more pieces of correspondence information. The correspondence information is information indicating the correspondence between an operation sequence and an intention identifier. The operation sequence is a set of one or more operation identifiers. The set of operation identifiers is a sequence of operation identifiers in time series.

[0048] The correspondence information has, for example, an operation sequence definition and an intention identifier. The operation sequence definition is information that defines an operation sequence. The operation sequence definition is information that defines a set of two or more operation identifiers. It is preferable that the operation sequence definition is a description of one or more operation identifiers using a regular expression. The intention identifier is information that identifies an intention. The intention identifier is, for example, an intention ID or an intention name. The intention is the intention of the operation. It is preferable that the two or more pieces of correspondence information correspond to an image type identifier. The image type identifier is information that identifies the type of image. The image type identifier is, for example, "street view", "three-dimensional image", "indoor image", and "dashcam image".

[0049] The correspondence information includes, for example, first correspondence information and second correspondence information. The first correspondence information is information indicating the correspondence between an operation sequence definition and a chunk identifier. A chunk identifier is information for identifying a chunk. A chunk is a group of operations. A chunk is a collection of one or more operations. A chunk identifier is a chunk ID or chunk name. The second correspondence information is information indicating the correspondence between a chunk sequence definition and an intention identifier. A chunk sequence definition is a definition of a chunk sequence, which is a collection of one or more chunk identifiers. It is preferable that the chunk sequence definition is a description of one or more chunk identifiers using regular expressions. It is preferable that the first correspondence information and the second correspondence information correspond to an image type identifier.

[0050] The reception unit 12 receives various instructions and information. The various instructions and information are, for example, an operation sequence for a first image, an output instruction, an operation end instruction, and corresponding information. The operation sequence is a set of operation identifiers that identify two or more operations. The first image is a position image of a target of operation by a user.

[0051] The output instruction is an instruction to output the first image. The output instruction usually has an image identifier. The image identifier is an identifier of the location image. The image identifier is, for example, a URL or a file name.

[0052] The operation end instruction is an instruction to end the operation on the first image, for example, an instruction to hide the first image.

[0053] Here, acceptance refers to, for example, receiving information transmitted from a user's terminal device (not shown) via a wired or wireless communication line, but it may also be a concept that includes the acceptance of information input from an input device such as a keyboard, mouse, or touch panel, and the acceptance of information read from a recording medium such as an optical disk, magnetic disk, or semiconductor memory.

[0054] The means for inputting various instructions and information may be any means, such as a touch panel, a keyboard, a mouse, or a menu screen.

[0055] The operation sequence receiving unit 121 receives an operation sequence, which is a set of operation identifiers that identify two or more operations on the first image. The operations on the first image are user operations. The operation sequence receiving unit 121 receives an operation sequence from a user, for example, or receives an operation sequence from a terminal device (not shown) of the user. The operation is, for example, one of "move," "turn left," "turn right," "turn up," "turn down," "zoom in," and "reduce." However, the operation is not limited.

[0056] The processing unit 13 performs various types of processing. The various types of processing are, for example, processing performed by an intention sequence acquisition unit 131, a focus feature determination unit 132, a category acquisition unit 133, an image determination unit 134, a target feature determination unit 135, or an operation unit 136.

[0057] The processing unit 13 acquires a first image corresponding to the output instruction. The processing unit 13 acquires the first image from, for example, an image server (not shown). The processing unit 13 acquires the first image from, for example, the image storage unit 112.

[0058] The processing unit 13 accumulates the correspondence information accepted by the accepting unit 12 in the correspondence management unit 113 .

[0059] The intention sequence acquisition unit 131 acquires an intention sequence, which is one or more intention identifiers corresponding to one or more operation sequence definitions matching the operation sequence accepted by the operation sequence acceptance unit 121, from the correspondence management unit 113. The intention sequence acquisition unit 131 acquires, for example, an image type identifier of the image to be operated, and acquires the intention sequence by using correspondence information corresponding to the image type identifier.

[0060] The intention sequence acquisition unit 131 acquires, for example, from two or more pieces of first correspondence information, a chunk sequence which is a set of one or more chunk identifiers corresponding to one or more operation sequence definitions matching the operation sequence accepted by the operation sequence acceptance unit 121. Next, the intention sequence acquisition unit 131 acquires, for example, from two or more pieces of second correspondence information, one or more intention identifiers corresponding to the chunk sequence definition matching the chunk sequence.

[0061] The attention feature determination unit 132 refers to the feature management unit 111 and uses position information corresponding to the first image to determine an attention feature in the first image that matches the attention condition. The attention feature determination unit 132 usually determines one attention feature, but may determine two or more attention features.

[0062] The attention condition is a condition for determining the attention feature. The attention condition is usually a condition related to the position information of the feature. For example, the attention condition is that the feature exists in the direction in which the user was facing for the longest time. For example, the attention condition is that the feature exists in a central area within a range of a threshold value from the center point of the first image for the longest time. For example, the attention condition is that the feature exists in the last first image in which all operations were performed and exists in a central area within a range of a threshold value from the center point of the first image. For example, the attention condition is that the feature exists in the last first image in which all operations were performed and exists in the direction in which the user was facing. The direction in which the user was facing is, for example, the direction in which the focus point (for example, the cursor position, the center point) in the first image is moving.

[0063] The category acquisition unit 133 acquires, from the feature management unit 111, a category that is an attribute value of the feature of interest determined by the feature of interest determination unit 132.

[0064] The image determination unit 134 determines one or more second images. The image determination unit 134 obtains, for example, from the image storage unit 112, one or more position images including features of the same category as the category obtained by the category acquisition unit 133. The features included in the position image are features whose position specified by the position information of the features is included in the area specified by the area information corresponding to the position image. The position image obtained by the image determination unit 134 is the second image.

[0065] When the first image and the second image are different types of position images, the image determination unit 134 determines, for example, one or more second images that include the same feature as the feature of interest in the first image.

[0066] The image determination unit 134 determines, for example, one or more second images including features of the same category as the category acquired by the category acquisition unit 133, and corresponding to location information that satisfies a location condition for the location information associated with the first image. The image determination unit 134 determines, for example, one or more second images including features of the same category as the category acquired by the category acquisition unit 133, and corresponding to location information that satisfies a location condition for the location information associated with the first image. The location condition is, for example, that the closeness is within the top N or less (N is a natural number equal to or greater than 1), or that the distance is within a threshold or less than the threshold.

[0067] The target feature determining unit 135 determines a target feature in each of the one or more second images determined by the image determining unit 134. The target feature is a feature in the same category as the category acquired by the category acquiring unit 133.

[0068] The operation unit 136 performs one or more operations that satisfy the operation sequence definition for the intention sequence acquired by the intention sequence acquisition unit 131, on each of the one or more second images determined by the image determination unit 134. For example, the operation unit 136 acquires an image type identifier for each of the one or more second images, acquires an operation sequence definition using two or more pieces of correspondence information (which may be called a correspondence table) that correspond to the image type identifier, and performs one or more operations that satisfy the operation sequence definition on each of the one or more second images determined by the image determination unit 134.

[0069] The operation unit 136 performs, for example, one or more operations on the target feature that satisfy the operation sequence definition for each of the one or more intention identifiers acquired by the intention sequence acquisition unit 131. Performing an operation on the target feature is performing an operation on the second image having the target feature. For example, the operation unit 136 performs an operation on the target feature using the position information of the target feature determined by the target feature determination unit 135 so as to maintain a positional relationship with respect to the target feature according to the intention identified by the intention identifier. For example, the operation unit 136 performs an operation on the target feature using the position information of the target feature to change the number of repetitions of the chunk identifier, or change the selection of the chunk identifier, or change the number of repetitions of the operation identifier, or change the selection of the operation identifier, so as to maintain a positional relationship with respect to the target feature according to the intention identified by the intention identifier. Note that the position information of the target feature here is usually two or more pieces of position information, and is area information that specifies the area in which the target feature exists.

[0070] The operation unit 136 performs one or more operations that satisfy the operation sequence definition for each of the one or more intention identifiers acquired by the intention sequence acquisition unit 131, on each of the one or more second images determined by the image determination unit 134.

[0071] When the operation unit 136 applies the intention to the target feature, the operation sequence that specifies the user's operation on the first image may not be applied as is due to differences in the topography between the first image and the second image. Therefore, the operation unit 136 needs to make adjustments based on the operation performed by the user depending on the feature. For example, in order to avoid continuing to perform an operation such as "proceed" even when the road in the second image becomes a dead end, the operation unit 136 does not perform operations beyond the limit when the operation cannot be performed any more on the second image. Also, for example, if the intention identifier is "appearance" described later, the distance to proceed and the angle to turn at an intersection may differ between the first image and the second image, so the operation unit 136 changes the operation sequence for the first image as a result and performs an operation on the second image using the changed operation sequence. For example, if the intention identifier is "appearance" and the distance to proceed is different, the operation unit 136 adds or deletes the operation identifier "proceed" so as to move to both ends of the target feature. In addition, when the turning angle is different between the road in the first image and the road in the second image, the operation unit 136 adjusts the number and angle of the operations of "turn left" or "turn right" in the second image so that it is parallel to the road beyond the turn.

[0072] The output unit 14 outputs various types of information, such as a first image and one or more second images.

[0073] Here, output means, for example, transmission to a user's terminal device (not shown), but is a concept that also includes display on a display, projection using a projector, printing on a printer, transmission to other external devices, storage on a recording medium, and handing over the processing results to other processing devices or other programs.

[0074] The first output unit 141 outputs a first image corresponding to the position information. There is no limitation on the trigger for the first output unit 141 to output the first image. For example, the first output unit 141 outputs the first image when an output instruction is accepted.

[0075] The second output unit 142 outputs the second image. The second output unit 142 outputs, for example, one or more second images determined by the image determination unit 134. Furthermore, the second output unit 142 normally outputs a second image that changes in response to an operation performed by the operation unit 136. Note that it may be considered that the output of the second image is performed by the operation unit 136.

[0076] The storage unit 11, the feature management unit 111, the image storage unit 112, and the correspondence management unit 113 are preferably non-volatile recording media, but may also be realized using volatile recording media.

[0077] There is no restriction on the process by which information is stored in the storage unit 11, etc. For example, information may be stored in the storage unit 11, etc. via a recording medium, information transmitted via a communication line, etc. may be stored in the storage unit 11, etc., or information inputted via an input device may be stored in the storage unit 11, etc.

[0078] It is preferable that the reception unit 12 and the operation sequence reception unit 121 are realized by wireless or wired communication means, but they may also be realized by a means for receiving broadcasts, a device driver for an input means such as a touch panel or a keyboard, or control software for a menu screen.

[0079] The processing unit 13, the intention sequence acquisition unit 131, the attention feature determination unit 132, the category acquisition unit 133, the image determination unit 134, the target feature determination unit 135, and the operation unit 136 can usually be realized by a processor, a memory, etc. The processing procedure of the processing unit 13, etc. is usually realized by software, and the software is recorded in a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit). The processor may be a CPU, MPU, GPU, etc., and the type is not important.

[0080] The output unit 14, the first output unit 141, and the second output unit 142 are typically realized by wireless or wired communication means, but may also be realized by driver software for an output device such as a display or speaker, or by a combination of driver software for an output device and an output device, etc.

[0081] Next, a first operation example of the image processing device 1 will be described with reference to the flowchart of FIG.

[0082] (S201) The reception unit 12 judges whether or not an output instruction has been received. If an output instruction has been received, the process proceeds to S202, and if not, the process proceeds to S216.

[0083] (S202) The processing unit 13 obtains a first image corresponding to the output instruction received in S201.

[0084] (S203) The first output unit 141 outputs the first image acquired in S202.

[0085] (S204) Operation sequence receiving unit 121 determines whether or not an operation for the first image being output has been received. If an operation has been received, the process proceeds to S205, and if not, the process proceeds to S208.

[0086] (S205) Operation unit 136 acquires the changed first image in accordance with the operation accepted in S204. Second output unit 142 outputs the first image.

[0087] (S206) The operation unit 136 acquires an operation identifier that identifies the operation accepted in S204, and temporarily stores the operation identifier in a buffer (not shown). For example, the operation identifiers that identify each of the operations "move", "turn left", "turn right", "turn up", "turn down", "zoom in", and "reduce" are "m", "l", "r", "u", "d", "i", and "o".

[0088] (S207) The processing unit 13 temporarily stores the frame, which is the first image changed after the operation, in a buffer (not shown). Return to S204. The frame corresponds to one or more pieces of position information.

[0089] (S208) The reception unit 11 judges whether or not an instruction to end the operation has been received. If an instruction to end the operation has been received, the process proceeds to S209, and if not, the process returns to S204.

[0090] (S209) The intention sequence acquisition unit 131 acquires an operation sequence, which is a set of one or more operation identifiers, from a buffer (not shown).

[0091] (S210) The intention sequence acquisition unit 131 acquires an intention sequence corresponding to the operation sequence acquired in S209. An example of such intention sequence acquisition processing will be described with reference to the flowchart of FIG.

[0092] (S211) The noteworthy feature determination unit 132 determines a noteworthy feature in the first image. An example of such noteworthy feature determination processing will be described with reference to the flowchart in FIG.

[0093] (S212) The category acquisition unit 133 acquires from the feature management unit 111 a category in the feature information of the feature of interest determined in S211.

[0094] (S213) The intention sequence acquisition unit 131 accumulates the intention sequence acquired in S209 and the category acquired in S212 in a buffer (not shown).

[0095] (S214) The image determination unit 134 determines one or more second images. An example of such image determination processing will be described with reference to the flowchart in FIG.

[0096] (S215) The operation unit 136 performs an operation on two or more images including the one or more second images determined in S214 according to the intention sequence accumulated in S213. An example of such image operation processing will be described with reference to the flowchart in FIG.

[0097] (S216) The accepting unit 12 judges whether the corresponding information has been accepted. If the corresponding information has been accepted, the process proceeds to S217, and if the corresponding information has not been accepted, the process returns to S201.

[0098] (S217) The processing unit 13 accumulates the correspondence information received in S216 in the correspondence management unit 113. The process returns to S201.

[0099] In the flowchart of FIG. 2, the process ends when the power is turned off or an interrupt occurs to end the process.

[0100] Next, a first example of the intention sequence acquisition process of S210 will be described with reference to the flowchart of FIG.

[0101] (S301) The intention sequence acquisition unit 131 assigns 1 to a counter i and a counter j.

[0102] (S302) The intention sequence acquisition unit 131 determines whether or not the i-th first correspondence information exists in the correspondence management unit 113. If the i-th first correspondence information exists, the process proceeds to S303; if not, the process proceeds to S311.

[0103] (S303) The intention sequence acquisition unit 131 acquires, from the correspondence management unit 113, an operation sequence definition contained in the i-th first correspondence information.

[0104] (S304) The intention sequence acquisition unit 131 judges whether or not a partial operation sequence that matches the operation sequence definition acquired in S303 exists in the partial operation sequence, which is a set of one or more consecutive operation identifiers after the j-th operation identifier. If the partial operation sequence exists, the process proceeds to S305, and if not, the process proceeds to S310.

[0105] (S305) The intention sequence acquiring unit 131 acquires a chunk identifier paired with the operation sequence definition acquired in S303 from the i-th first correspondence information, and temporarily accumulates the chunk identifier in a buffer (not shown).

[0106] (S306) The intention sequence acquisition unit 131 advances j to the next operation identifier of the partial operation sequence of the maximum length that matches the operation sequence definition in S304.

[0107] (S307) The intention sequence acquisition unit 131 judges whether j advanced in S306 is the next position of the last operation identifier in the operation sequence acquired in S209 (whether the operation sequence has ended). If the operation sequence has ended, the process proceeds to S308, and if not, the process proceeds to S309.

[0108] (S308) The intention sequence acquisition unit 131 acquires a chunk sequence, which is one or more chunk identifiers, from a buffer (not shown).

[0109] (S309) The intention sequence acquisition unit 131 assigns 1 to the counter i. The process returns to S302.

[0110] (S310) The intention sequence acquisition unit 131 increments the counter i by 1. The process returns to S302.

[0111] (S311) The intention sequence acquisition unit 131 increments the counter j by 1. The process returns to S302.

[0112] (S312) The intention sequence acquisition unit 131 assigns 1 to counter i and counter j.

[0113] (S313) The intention sequence acquisition unit 131 determines whether or not the i-th second correspondence information exists in the correspondence management unit 113. If the i-th second correspondence information exists, the process proceeds to S314; if not, the process proceeds to S322.

[0114] (S314) The intention sequence acquisition unit 131 acquires the chunk sequence definition contained in the i-th second correspondence information from the correspondence management unit 113.

[0115] (S315) The intent sequence acquisition unit 131 determines whether or not a partial chunk sequence that matches the chunk sequence definition acquired in S314 exists among the partial chunk sequence, which is a set of one or more consecutive chunk identifiers from the j-th chunk identifier onward. If the partial chunk sequence exists, the process proceeds to S316; if not, the process proceeds to S321.

[0116] (S316) The intention sequence acquisition unit 131 acquires an intention identifier paired with the chunk sequence definition acquired in S315 from the i-th second correspondence information, and temporarily accumulates the intention identifier in a buffer (not shown).

[0117] (S317) The intention sequence acquisition unit 131 advances j to the next chunk identifier of the partial chunk sequence of the maximum length that matches the chunk sequence definition in S315.

[0118] (S318) The intention sequence acquisition unit 131 judges whether j advanced in S317 is the position next to the last chunk identifier of the chunk sequence acquired in S308 (whether the chunk sequence has ended). If the chunk sequence has ended, the process proceeds to S319, and if not, the process proceeds to S320.

[0119] (S319) The intention sequence acquisition unit 131 acquires an intention sequence, which is one or more intention identifiers, from a buffer (not shown), and then returns to the upper-level process.

[0120] (S320) The intention sequence acquisition unit 131 assigns 1 to the counter i. Then, the process returns to S313.

[0121] (S321) The intention sequence acquisition unit 131 increments the counter i by 1. The process returns to S313.

[0122] (S322) The intention sequence acquisition unit 131 increments the counter j by 1. The process returns to S313.

[0123] Next, a second example of the intention sequence acquisition process of S210 will be described with reference to the flowchart of FIG.

[0124] (S401) The intention sequence acquisition unit 131 assigns 1 to a counter i and a counter j.

[0125] (S402) The intention sequence acquisition unit 131 judges whether or not the i-th correspondence information exists in the correspondence management unit 113. If the i-th correspondence information exists, the process proceeds to S403, and if not, the process proceeds to S411.

[0126] (S403) The intention sequence acquisition unit 131 acquires an operation sequence definition contained in the i-th correspondence information from the correspondence management unit 113.

[0127] (S404) The intention sequence acquisition unit 131 judges whether or not a partial operation sequence that matches the operation sequence definition acquired in S403 exists in the partial operation sequence, which is a set of one or more consecutive operation identifiers after the j-th operation identifier. If the partial operation sequence exists, the process proceeds to S405, and if not, the process proceeds to S410.

[0128] (S405) The intention sequence acquisition unit 131 acquires an intention identifier paired with the operation sequence definition acquired in S403 from the i-th correspondence information, and temporarily stores the intention identifier in a buffer (not shown).

[0129] (S406) The intention sequence acquisition unit 131 advances j to the next operation identifier of the partial operation sequence of the maximum length that matches the operation sequence definition in S404.

[0130] (S407) The intention sequence acquisition unit 131 judges whether j advanced in S406 is the next position of the last operation identifier in the operation sequence acquired in S209 (whether the operation sequence has ended). If the operation sequence has ended, the process proceeds to S408, and if not, the process proceeds to S409.

[0131] (S408) The intention sequence acquisition unit 131 acquires an intention sequence, which is one or more intention identifiers, from a buffer (not shown), and returns to the upper-level process.

[0132] (S409) The intention sequence acquisition unit 131 assigns 1 to the counter i. The process returns to S402.

[0133] (S410) The intention sequence acquisition unit 131 increments the counter i by 1. The process returns to S402.

[0134] (S411) The intention sequence acquisition unit 131 increments the counter j by 1. The process returns to S402.

[0135] Next, an example of the attention feature determination process in S211 will be described with reference to the flowchart in FIG.

[0136] (S501) The interest feature determination unit 132 assigns 1 to a counter i.

[0137] (S502) The feature of interest determination unit 132 judges whether or not the i-th frame exists among the frames accumulated in the buffer (not shown) in S207. If the i-th frame exists, the process proceeds to S503, and if not, the process proceeds to S514.

[0138] (S503) The feature of interest determination unit 132 acquires range information that specifies the range of the area captured in the i-th frame. The feature of interest determination unit 132 acquires the range information from, for example, position information (position information of the representative point) associated with the i-th frame and the frame resolution. The feature of interest determination unit 132 acquires the range information associated with the i-th frame. The range information is, for example, information indicating a range of three-dimensional space, such as "(x1, y1, h1) (x2, y2, h2)".

[0139] (S504) The feature of interest determination unit 132 acquires positional information of interest corresponding to the i-th frame. The positional information of interest is information for identifying the position of a point of interest in a frame, or positional information of a focus point, for example (x, y, h).

[0140] (S505) The attention feature determination unit 132 acquires direction information corresponding to the i-th frame. The direction information is information indicating a direction, for example, an angle from true north, and is a numerical value between 0 and 360. The direction information is information indicating a direction that can be acquired from the difference between the attention position information of each of two consecutive frames.

[0141] (S506) The interest feature determination unit 132 assigns 1 to a counter j.

[0142] (S507) The attention feature determination unit 132 judges whether or not the j-th feature information exists in the feature management unit 111. If the j-th feature information exists, the process proceeds to S508, and if not, the process proceeds to S513.

[0143] (S508) The noteworthy feature determination unit 132 acquires the position information contained in the j-th feature information from the feature management unit 111.

[0144] (S509) The attention feature determination unit 132 uses the area information acquired in S503 and the position information acquired in S508 to determine whether the jth feature is present in the area captured by the i-th frame. If the jth feature is present in the area of ​​the i-th frame, the process proceeds to S510; if not, the process proceeds to S512.

[0145] (S510) The noteworthy feature determination unit 132 judges whether or not the position information of the j-th feature acquired in S508 matches the noteworthy condition. If the noteworthy condition is met, the process proceeds to S511, and if the noteworthy condition is not met, the process proceeds to S512.

[0146] (S511) The noteworthy feature determination unit 132 increments the counter corresponding to the j-th feature information by 1. Note that the initial value of the counter corresponding to each feature information is "0".

[0147] (S512) The interest feature determination unit 132 increments the counter j by 1. The process returns to S507.

[0148] (S513) The interest feature determination unit 132 increments the counter i by 1. The process returns to S502.

[0149] (S514) The attention feature determination unit 132 acquires the feature information associated with the counter with the maximum value, and returns to the upper level process.

[0150] 5, only one piece of feature information is acquired, but two or more pieces of feature information may be acquired. In such a case, for example, in S514, the attention feature determination unit 132 acquires feature information whose counter value is equal to or greater than a threshold value or whose counter value is in the top N (N is a natural number equal to or greater than 1).

[0151] Next, an example of the image determination process in S214 will be described with reference to the flowchart in FIG.

[0152] (S601) The image determination unit 134 acquires position information of a first image.

[0153] (S602) The image determination unit 134 assigns 1 to a counter i.

[0154] (S603) The image determination unit 134 judges whether or not there is an i-th candidate image that can be the second image. If there is an i-th candidate image, the process proceeds to S604. If there is no i-th candidate image, the process proceeds to S613. The i-th candidate image is, for example, a position image stored in the image storage unit 112 or an image server (not shown).

[0155] (S604) The image determination unit 134 obtains area information that identifies the area captured by the i-th candidate image.

[0156] (S605) The image determination unit 134 assigns 1 to a counter j.

[0157] (S606) The image determination unit 134 judges whether or not the j-th feature information exists in the feature management unit 111. If the j-th feature information exists, the process proceeds to S607, and if not, the process proceeds to S611.

[0158] (S607) The image determination unit 134 acquires the category of the j-th feature information from the feature information.

[0159] (S608) The image determination unit 134 judges whether or not the category acquired in S607 is the same as the category of the feature of interest in the first image. If the two categories are the same, the process proceeds to S609, and if not, the process proceeds to S612.

[0160] (S609) The image determination unit 134 obtains the distance between the two features using the position information of the j-th feature information and the position information of the feature of interest, and associates the distance with the j-th feature information.

[0161] (S610) Image determination unit 134 temporarily stores the identifier of the i-th candidate image in a buffer (not shown) in pairs with the distance.

[0162] (S611) The image determination unit 134 increments the counter i by 1. The process returns to S603.

[0163] (S612) The image determination unit 134 increments the counter j1. Then, the process returns to S606.

[0164] (S613) The image determination unit 134 determines one or more candidate images whose distances match the distance condition to be paired with each of the one or more candidate images accumulated in S610, and acquires one or more second images which are the one or more candidate images. The process returns to the upper level process. The distance condition is, for example, "top three from shortest distance" or "distance is within or less than a threshold value."

[0165] In addition, in the flowchart of Figure 6, if the first image and the second image are different types of position images, the image determination unit 134 may acquire one or more second images that include the same feature as the feature of interest in the first image.

[0166] Next, an example of the image manipulation process in S215 will be described with reference to the flowchart in FIG.

[0167] (S701) The operation unit 136 acquires the number of images on which an operation is to be performed using the acquired intention sequence. The images on which the operation is to be performed are usually one or more second images and the first image acquired in S613. That is, the number of images is usually "the number of images acquired in S613 + 1". However, the images on which the operation is to be performed may be only one or more second images acquired in S613.

[0168] (S702) The operation unit 136 generates windows for the number of images acquired in S701.

[0169] (S703) Operation unit 136 assigns 1 to counter i.

[0170] (S704) Operation unit 136 judges whether or not the i-th image to be operated exists. If the i-th image exists, the process proceeds to S705, and if not, the process proceeds to S707.

[0171] (S705) Operation unit 136 obtains the i-th image to be operated on, and outputs the image to the i-th window.

[0172] (S706) The operation unit 136 increments the counter i by 1. The process returns to S704.

[0173] (S707) Operation unit 136 assigns 1 to counter j.

[0174] (S708) The operation unit 136 judges whether or not the j-th intention identifier exists in the acquired intention sequence. If the j-th intention identifier exists, the process proceeds to S709, and if not, the process returns to the upper processing.

[0175] (S709) The operation unit 136 refers to the correspondence management unit 113 and acquires an operation sequence definition corresponding to the j-th intention identifier.

[0176] (S710) Operation unit 136 assigns 1 to counter k.

[0177] (S711) Operation unit 136 determines whether or not the kth image to be operated exists. If the kth image exists, the process proceeds to S712, and if not, the process proceeds to S714.

[0178] (S712) The operation unit 136 performs an operation corresponding to the jth intention identifier on the kth image by using the operation sequence definition acquired in S709. An example of such intention execution processing will be described with reference to the flowchart in FIG.

[0179] (S713) The operation unit 136 increments the counter k by 1. The process returns to S711.

[0180] (S714) The operation unit 136 increments the counter j by 1. The process returns to S708.

[0181] Next, an example of the intention execution process in S712 will be described with reference to the flowchart in FIG.

[0182] (S801) The operation unit 136 acquires area information of a target feature in a second image that is an operation target.

[0183] (S802) Operation unit 136 assigns 1 to counter i.

[0184] (S803) The operation unit 136 judges whether or not the i-th operation identifier in the operation sequence definition exists. If the i-th operation identifier exists, the process proceeds to S804, and if not, the process returns to the upper process.

[0185] (S804) Operation unit 136 judges whether or not the i-th operation identifier is “movement.” If it is “movement,” the process proceeds to S805, and if it is not “movement,” the process proceeds to S811.

[0186] (S805) Operation unit 136 places a focus point on the road in the image of the operation target. Note that operation unit 136 sets the focus point to a point on the road that is closest to the center position of the image of the operation target, for example.

[0187] (S806) Operation unit 136 acquires the distance between the target feature and the focus point.

[0188] (S807) Operation unit 136 determines whether the distance acquired in S806 satisfies a condition. If the condition is satisfied, the process proceeds to S808, and if the condition is not satisfied, the process proceeds to S810. Note that the condition here is, for example, "first threshold value<=distance<=second threshold value." In other words, if the focus point gets too close to or too far away from the target feature during movement, it is determined to be inappropriate, and the movement is stopped.

[0189] (S808) Operation unit 136 acquires an image with the focus point moved. Note that the amount of movement is, for example, a predetermined unit amount. The image with the focus point moved is an image advanced by the amount of movement.

[0190] (S809) The operation unit 136 increments the counter i by 1. The process returns to S803.

[0191] (S810) The operation unit 136 advances i until the operation identifier indicated by i becomes the operation identifier next to the operation identifier of “move” in the operation sequence definition. Then, the process returns to S803.

[0192] (S811) Operation unit 136 determines whether or not the i-th operation identifier is “enlargement.” If it is “enlargement,” the process proceeds to S812, and if it is not “enlargement,” the process proceeds to S816.

[0193] (S812) Operation unit 136 acquires position information of the target feature and position information of the focus point, and acquires the distance between the target feature and the focus point using the two pieces of position information.

[0194] (S813) The operation unit 136 determines whether the distance acquired in S812 satisfies a condition. If the condition is satisfied, the process proceeds to S814, and if the condition is not satisfied, the process proceeds to S815. The condition is, for example, "third threshold value<=distance<=fourth threshold value." In other words, if the focus point gets too close to or too far away from the target feature during enlargement (zooming in), it is determined to be inappropriate, and the enlargement is stopped.

[0195] (S814) Operation unit 136 enlarges the image to be operated. The process proceeds to S809. Note that the degree of enlargement is usually a predetermined unit amount.

[0196] (S815) The operation unit 136 advances i until the operation identifier indicated by i becomes the position of the operation identifier next to the operation identifier of “Enlarge” in the operation sequence definition. Then, the process returns to S803.

[0197] (S816) Operation unit 136 determines whether or not the i-th operation identifier is “reduction.” If it is “reduction,” the process proceeds to S817, and if it is not “reduction,” the process proceeds to S821.

[0198] (S817) Operation unit 136 acquires position information of the target feature and position information of the focus point, and acquires the distance between the target feature and the focus point using the two pieces of position information.

[0199] (S818) The operation unit 136 determines whether the distance acquired in S817 satisfies a condition. If the condition is satisfied, the process proceeds to S819, and if the condition is not satisfied, the process proceeds to S820. The condition is, for example, "fifth threshold value<=distance<=sixth threshold value." In other words, if the focus point gets too close to or too far away from the target feature during reduction (zooming out), it is determined to be inappropriate, and the reduction is stopped.

[0200] (S819) Operation unit 136 reduces the image to be operated on. The process proceeds to S809. The degree of reduction is usually a predetermined unit amount.

[0201] (S820) The operation unit 136 advances i until the operation identifier indicated by i becomes the position of the operation identifier next to the operation identifier of “reduce” in the operation sequence definition. Then, the process returns to S803.

[0202] (S821) The operation unit 136 executes the operation identified by the operation identifier. Return to S809. Note that such an operation is, for example, "turn left," "turn right," "turn up," or "turn down."

[0203] Next, a second operation example of the image processing device 1 will be described with reference to the flowchart of Fig. 9. In the flowchart of Fig. 9, the description of the same steps as those in the flowchart of Fig. 9 will be omitted.

[0204] (S901) The intention sequence acquisition unit 131 determines whether or not an intention identifier has been acquired in S210. If an intention identifier has been acquired, the process proceeds to S211, and if an intention identifier has not been acquired, the process returns to S204. Note that in S210 here, a maximum of one intention identifier is acquired using an operation sequence that is usually one or more operation identifiers.

[0205] (S902) The operation unit 136 performs an operation on the one or more second images determined in S214 according to the intention identifier accumulated in S210. Return to S201. An example of such image operation processing will be described with reference to the flowchart of FIG.

[0206] In the flowchart of FIG. 9, while the user is manipulating the first image, one or more second images are automatically output, and each of the one or more second images is automatically manipulated according to the user's operational intention.

[0207] In the flowchart of FIG. 9, the process ends when the power is turned off or an interrupt occurs to end the process.

[0208] Next, an example of the image manipulation process in S902 will be described with reference to the flowchart in FIG.

[0209] (S1001) The second output unit 142 determines whether or not a window of the second image exists. If a window of the second image exists, the process proceeds to S1004, and if not, the process proceeds to S1002.

[0210] (S1002) Second output unit 142 acquires the number of acquired second images. Second output unit 142 generates windows equal to the number of the acquired second images.

[0211] (S1003) Second output unit 142 outputs each of the one or more acquired second images to each window.

[0212] (S1004) The operation unit 136 refers to the correspondence management unit 113 and acquires an operation sequence definition corresponding to the intention identifier acquired in S210.

[0213] (S1005) Operation unit 136 assigns 1 to counter i.

[0214] (S1006) Operation unit 136 determines whether or not the i-th second image to be operated exists. If the i-th second image exists, the process proceeds to S1007, and if not, the process returns to the upper process.

[0215] (S1007) The operation unit 136 performs, on the i-th second image, an operation corresponding to the intention identifier acquired in S210, by using the operation sequence definition acquired in S1004. An example of such intention execution processing has been described with reference to the flowchart in FIG.

[0216] (S1008) Operation unit 136 increments counter i by 1. The process returns to S1006.

[0217] A specific example of the operation of the image processing device 1 according to the present embodiment will now be described.

[0218] FIG. 11 is a diagram for explaining an example of the concept of the image processing device 1. In the image processing device 1, one or more chunks are determined from one or more operations on the first image of the user, an intention is determined from a chunk string, and an operation is performed to reflect the intention in the second image. As described above, in the image processing device 1, an intention may be determined from one or more operations on the first image of the user, and an operation to reflect the intention in the second image may be performed. In other words, it is not essential for the image processing device 1 to take chunks into consideration. Two specific examples will be described below. Specific Example 1 is a case where the first image and the second image are the same type of position image. In Specific Example 1, the first image and the second image are street views. Specific Example 2 is a case where the first image and the second image are different types of position images. In Specific Example 2, the first image is a street view, and the second image is a Google Earth image.

[0219] (Example 1) The first correspondence table of Fig. 12(b) and the second correspondence table of Fig. 12(c) are stored in the correspondence management unit 113 of the image processing device 1. Fig. 12(a) shows operation identifiers in the street view.

[0220] The first correspondence table (Figure 12(b)) has multiple records each having a "chunk," "chunk identifier," and "operation sequence definition." Here, the chunks are "move," "look left / right," "look up / down," and "zoom," and their respective chunk identifiers are "M," "P," "T," and "Z." The operation sequence definition corresponding to each chunk identifier is a regular expression using one or more operation identifiers from among "m," "l," "r," "u," "d," "i," and "o." In the regular expression, [] indicates that one of them is to be adopted, + indicates a positive closure (one or more times), and * indicates a closure (zero or more times).

[0221] A screen image in this case is shown in Fig. 13. First, it is assumed that the user inputs an instruction to output a street view including a store A, which is an izakaya, to the image processing device 1. In response, the image processing device 1 outputs a street view 1302 including store A.

[0222] Next, the user uses screen operation buttons 1301 in FIG. 13 to operate as follows, as shown in FIG. 14: "i", "o", "o", "u", "u", "d", "m", "m", "r", "l", "l", "r", "m", "m", "m", "i", "i", "o".

[0223] Then, the operation sequence receiving unit 121 sequentially receives such operations. Next, the intention sequence acquiring unit 131 acquires chunk identifiers "Z" from the partial operation sequence "i", "o", "o", acquires chunk identifiers "T" from the partial operation sequence "u", "u", "u", "d", acquires chunk identifiers "M" from the partial operation sequence "m", "m", acquires chunk identifiers "P" from the partial operation sequence "r", "l", "l", "r", acquires chunk identifiers "M" from the partial operation sequence "m", "m", "m", and acquires chunk identifiers "Z" from the partial operation sequence "i", "i", "i", "o". That is, the intention sequence acquiring unit 131 acquires chunk sequences "Z", "T", "M", "P", "M", and "Z" from the operation sequence "i", "o", "o", "u", "u", "d", "m", "m", "r", "l", "l", "r", "m", "m", "m", "i", "i", "o", by referring to the first correspondence table (FIG. 12(b)), from the operation sequence "i", "o", "o", "u", "u", "d", "m", "m", "r", "l", "l", "r", "m", "m", "i", "i", "o".

[0224] Next, the intention sequence acquisition unit 131 acquires the intention identifier “details” from the chunk sequences “Z” and “T”, The intention identifier "periphery" is acquired from the chunk sequences "M", "P", and "M". That is, the intention sequence acquisition unit 131 acquires the intention sequences "details" and "periphery" from the acquired chunk sequence by referring to the second correspondence table (FIG. 12(c)).

[0225] Next, the feature of interest determination unit 132 determines the feature of interest "Store A" in the first image 1302 using the above-mentioned algorithm.

[0226] Next, the category acquisition unit 133 acquires the category “pub” in the feature information of the feature of interest “store A” from the feature management unit 111. Then, the intention sequence acquisition unit 131 accumulates the acquired intention sequences “details” and “surroundings” and the category “pub” in a buffer (not shown).

[0227] Next, the image determination unit 134 determines a street view that includes the izakaya, and includes the target features of the category "izakaya" that correspond to the location information close to the location information of "store A," and the features of the top three stores ("store B," "store C," and "store D") that are closest to "store A," and obtains the three street views from an image server (not shown). Such second images are 1303, 1304, and 1305 in FIG. 13.

[0228] Next, the operation unit 136 performs operations corresponding to the intention strings "details" and "surroundings" on each of the three street views (1303 to 1305) using the above-mentioned algorithm.

[0229] As a result, users can see three automatically-operated Street View images, allowing them to check the "details" and "surrounding area" of "Store B," "Store C," and "Store D."

[0230] (Example 2) The correspondence management unit 113 of the image processing device 1 stores a correspondence table shown in Fig. 15. The correspondence table has a plurality of records each having an "intention identifier" and an "operation sequence definition". This correspondence table is for operations on Google Earth.

[0231] In Specific Example 2, the user performs the same operation as in Specific Example 1, and as a result, the image processing device 1 outputs a street view 1302 including store A, and the intention sequence acquisition unit 131 acquires the intention sequences "details" and "surroundings." In addition, the attention feature determination unit 132 determines the attention feature "Store A" in the first image 1302 by the above-mentioned algorithm. The processing up to this point is the same as in Specific Example 1.

[0232] Next, the intention sequence acquisition unit 131 accumulates the acquired intention sequence "details" and "surroundings" and the target feature "store A" in a buffer (not shown).

[0233] Next, the image determination unit 134 inputs the target feature "Store A" to the Google Earth image, and outputs the Google Earth image including "Store A." Such an output image is 1601 in FIG.

[0234] Next, the operation unit 136 selects the operation sequence definition “Zoom in” that is paired with the intention identifier “Details.” + from the correspondence table (FIG. 15). In addition, the operation unit 136 acquires the operation sequence definition [zoom in | zoom out | left | right | up | down] that is paired with the intention identifier "periphery". + is obtained from the correspondence table (Figure 15).

[0235] Then, the operation unit 136 inputs the operation sequence definition “Zoom in” to the Google Earth image 1601. + In accordance with the above, zooming in is repeated until the position information of the focus point in the Google Earth image 1601 is at a predetermined distance from the position information of "Store A". Next, the operation unit 136 performs an operation sequence definition [Zoom in | Zoom out | Left | Right | Up | Down] based on the intention identifier "surroundings". + In accordance with this, for example, during the time that the first screen 1302 is being operated, one of the operations from zooming in, zooming out, left, right, up, or down is randomly selected, the selected operation is performed, and the "surrounding area" of "Store A" is presented to the user using Google Earth images.

[0236] In addition, in a specific example, the correspondence table corresponding to each type of position image (first image, second image) may be one correspondence table as in Fig. 15, or two correspondence tables as in Fig. 12(b) and (c). In other words, the correspondence table may be composed of correspondence information, or may be composed of first correspondence information and second correspondence information.

[0237] As described above, according to the present embodiment, the intention of an operation on a first image can be applied to one or more second images, and as a result, one or more second images can be automatically operated in accordance with the user's intention.

[0238] In addition, according to this embodiment, the intention of the operation on the first image can be reused on one or more second images of a type different from the first image, so that the one or more second images can be automatically operated in accordance with the user's intention.

[0239] Moreover, according to this embodiment, the second image can be automatically determined.

[0240] Furthermore, according to the present embodiment, it is possible to reuse the intention information that specifies the intention of the operation on the first image.

[0241] The processing in this embodiment may be realized by software. This software may be distributed by software download or the like. This software may be recorded on a recording medium such as a CD-ROM and distributed. This also applies to other embodiments in this specification. The software for realizing the image processing device 1 in this embodiment is a program as follows. That is, this program is a program for making a computer that can access a correspondence management unit in which two or more pieces of correspondence information indicating the correspondence between an operation sequence definition that defines a set of operation identifiers for each of two or more operations and an intention identifier that identifies the intention of the operation be stored function as a first output unit that outputs a first image that is an image corresponding to position information, an operation sequence receiving unit that receives an operation sequence that is a set of operation identifiers that identify each of two or more operations on the first image, an intention sequence acquisition unit that acquires from the correspondence management unit an intention sequence that is an intention identifier corresponding to each of one or more operation sequence definitions that match the operation sequence received by the operation sequence receiving unit, a second output unit that outputs a second image, and an operation unit that performs one or more operations on the second image that satisfy the operation sequence definition for the intention sequence acquired by the intention sequence acquisition unit.

[0242] (Embodiment 2) In this embodiment, an image processing device 2 that distributes an intention sequence acquired by operating a first image and uses the intention sequence for operating a second image will be described.

[0243] 17 is a block diagram of the image processing device 2 in this embodiment. The image processing device 2 may be a standalone device, or may be a server that receives instructions and information from a terminal device (not shown) and transmits a position image to the terminal device. When the image processing device 2 is a standalone device, the image processing device 2 is, for example, a so-called personal computer, a smartphone, or a tablet terminal, but the type does not matter. When the image processing device 2 is a server, the image processing device 2 is, for example, a cloud server or an ASP server, but the type does not matter.

[0244] The image processing device 2 includes a storage unit 11, a reception unit 22, a processing unit 23, and an output unit 24. The reception unit 22 includes an intention reception unit 221. The processing unit 23 includes a focus feature determination unit 132, a category acquisition unit 133, an image acquisition unit 234, a target feature determination unit 135, and an operation unit 136. The output unit 24 includes a second output unit 142.

[0245] The receiving unit 22 receives various instructions and information, such as intention information, an automatic operation instruction, and an intention sequence, which will be described later.

[0246] The automatic operation instruction is an instruction to automatically operate the second image. The automatic operation instruction includes, for example, an identifier of a position image of an operation target. The position image of the operation target is the second image. The automatic operation instruction includes, for example, information for identifying an intention sequence.

[0247] The intention sequence etc. may be only the intention sequence, may be the intention sequence and a category, or may be the intention sequence and a target feature identifier. The category is the category of the target feature. The target feature identifier is information for identifying the target feature. The target feature identifier is, for example, the name of the feature to be operated or the ID of the feature.

[0248] The intention receiving unit 221 receives intention information. The intention information is information that specifies the user's intention with respect to the position image. The intention information includes an intention sequence. Such an intention sequence is information acquired in response to an operation on the first image. It is preferable that the intention information also includes an intention sequence and a category. The intention information may have an intention sequence and a target feature identifier.

[0249] The processing unit 23 performs various types of processing. The various types of processing are, for example, processing performed by a focus feature determination unit 132, a category acquisition unit 133, an image acquisition unit 234, a target feature determination unit 135, and an operation unit 136.

[0250] The image acquisition unit 234 acquires a second image of the target to be automatically operated. For example, the image acquisition unit 234 acquires the second image by using an identifier of a position image included in the automatic operation instruction. For example, the image acquisition unit 234 acquires the second image from the image storage unit 112 or an image server (not shown). Note that the second image and the first image may be the same or different in type.

[0251] The output unit 24 outputs various types of information, such as the second image acquired by the image acquisition unit 234.

[0252] Here, output means, for example, transmission to a user's terminal device (not shown), but is a concept that also includes display on a display, projection using a projector, printing on a printer, transmission to other external devices, storage on a recording medium, and handing over the processing results to other processing devices or other programs.

[0253] The reception unit 22 and the intention reception unit 221 are preferably realized by wireless or wired communication means, but may also be realized by a means for receiving broadcasts, a device driver for an input means such as a touch panel or keyboard, or control software for a menu screen.

[0254] The processing unit 23 and the image acquisition unit 234 can usually be realized by a processor, a memory, etc. The processing procedure of the processing unit 23, etc. is usually realized by software, and the software is recorded in a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit). The processor may be a CPU, MPU, GPU, etc., and the type is not important.

[0255] The output unit 24 is usually realized by a wireless or wired communication means, but may also be realized by driver software for an output device such as a display or speaker, or by a combination of driver software for an output device and the output device.

[0256] Next, an example of the operation of the image processing device 2 will be described with reference to the flowchart of FIG.

[0257] (S1801) The reception unit 22 judges whether or not an automatic operation instruction has been received. If an automatic operation instruction has been received, the process proceeds to S1802, and if not, the process proceeds to S1806.

[0258] (S1802) The image acquisition unit 234 acquires a second image by using an identifier of the position image included in the automatic operation instruction. The second image is a position image of the automatic operation target.

[0259] (S1803) The second output unit 142 outputs the second image acquired in S1802.

[0260] (S1804) The intention receiving unit 221 acquires intention information. The intention receiving unit 221 acquires, for example, the intention information from the storage unit 11. The intention receiving unit 221 acquires, for example, the intention information included in the automatic operation instruction. The intention receiving unit 221 acquires, for example, the intention information identified by an identifier of the intention information included in the automatic operation instruction.

[0261] (S1805) The operation unit 136 operates the second image output in S1803 by using the intention information acquired in S1804. An example of such an image operation will be described with reference to the flowchart in FIG.

[0262] (S1806) The intention receiving unit 221 determines whether or not it has received an intention sequence, etc. If it has received an intention sequence, etc., the process proceeds to S1807, and if it has not been received, the process returns to S1801.

[0263] (S1807) The processing unit 23 accumulates the intention sequence etc. accepted in S1806 in the storage unit 11. Return to S1801. Note that the intention sequence etc. is intention information. The intention sequence etc. is, for example, an intention sequence and a category, an intention sequence and a feature identifier, or only an intention sequence.

[0264] In the flowchart of FIG. 18, the process ends when the power is turned off or an interrupt occurs to end the process.

[0265] Next, an example of the image manipulation in S1805 will be described with reference to the flowchart in FIG.

[0266] (S1901) Operation unit 136 assigns 1 to counter i.

[0267] (S1902) The operation unit 136 judges whether or not the i-th intention identifier is present in the acquired intention sequence. If the i-th intention identifier is present, the process proceeds to S1903, and if not, the process returns to the upper processing.

[0268] (S1903) The operation unit 136 refers to the correspondence management unit 113 and acquires an operation sequence definition corresponding to the i-th intention identifier.

[0269] (S1904) The operation unit 136 performs an operation corresponding to the i-th intention identifier on the output second image by using the operation sequence definition acquired in S1903. An example of such intention execution processing has been described with reference to the flowchart in FIG.

[0270] (S1905) The operation unit 136 increments the counter i by 1. The process returns to S1902.

[0271] A specific operation example of the image processing device 2 in this embodiment will be described below. Assume that the storage unit 11 of the image processing device 2 stores an intention sequence identifier “intention X” and an intention sequence paired with the intention sequence “details” and “periphery”.

[0272] In this situation, it is assumed that the user inputs an automatic operation instruction including a target feature “Store A” and an intention string identifier “Intention X” into the image processing device 2.

[0273] Then, the reception unit 22 receives the automatic operation instruction "<interesting feature> store A <intention string identifier> intention X".

[0274] Next, the image acquisition unit 234 acquires a street view including "Store A" using "<attention feature> Store A" included in the automatic operation instruction. The category acquisition unit 133 also acquires the category "Pub" for "Store A". Next, the target feature determination unit 135 determines a predetermined number (here, three) of features "Store B", "Store C", and "Store D" that are features of the category "Pub" and are close to "Store A". Next, the image acquisition unit 234 acquires three street views including the features "Store B", "Store C", and "Store D" from an image server not shown.

[0275] Next, the output unit 24 outputs four street views each including "Store A", "Store B", "Store C", and "Store D".

[0276] Next, the operation unit 136 applies the intention strings "details" and "surroundings" paired with the intention string identifier "intention X" to the four street views, with "store A", "store B", "store C", and "store D" as target features, respectively, and automatically operates the four street views. Such an operation has been described in the specific example of the first embodiment.

[0277] Then, the output unit 24 outputs the street view to each of the windows 1302 to 1305 in FIG. 13, for example.

[0278] As described above, according to the present embodiment, it is possible to reuse the intention information that specifies the intention of the operation on the first image. In other words, according to the present embodiment, it is possible to reuse the intention of the operation on the first image for one or more second images, and as a result, it is possible to automatically operate one or more second images according to the user's intention.

[0279] Furthermore, according to this embodiment, the second image can be determined automatically.

[0280] Note that software for realizing the image processing device 2 in this embodiment is a program as follows. That is, this program is a program for causing a computer that can access a correspondence management unit in which two or more pieces of correspondence information indicating the correspondence between an operation sequence definition and an intention identifier are stored to function as an intention receiving unit that receives intention information including an intention sequence acquired in response to an operation on a first image, a second output unit that outputs a second image, and an operation unit that acquires an operation sequence definition for each of one or more intention identifiers included in the intention sequence included in the intention information accepted by the intention receiving unit and performs one or more operations on the second image that satisfy each of the one or more operation sequence definitions.

[0281] Fig. 20 shows the appearance of a computer that executes the program described in this specification to realize the image processing device 1 or the image processing device 2 of the various embodiments described above. The above-mentioned embodiments can be realized by computer hardware and a computer program executed thereon. Fig. 20 is an overview of this computer system 300, and Fig. 21 is a block diagram of the system 300.

[0282] In FIG. 20, a computer system 300 includes a computer 301 including a CD-ROM drive, a keyboard 302, a mouse 303, and a monitor 304.

[0283] 21, computer 301 includes, in addition to CD-ROM drive 3012, MPU 3013, bus 3014 connected to CD-ROM drive 3012 etc., ROM 3015 for storing programs such as a boot-up program, RAM 3016 connected to MPU 3013 for temporarily storing instructions of application programs and providing temporary storage space, and hard disk 3017 for storing application programs, system programs, and data. Although not shown here, computer 301 may further include a network card for providing connection to a LAN.

[0284] A program for causing computer system 300 to execute the functions of image processing device 1 and the like of the above-mentioned embodiment may be stored on CD-ROM 3101, inserted into CD-ROM drive 3012, and further transferred to hard disk 3017. Alternatively, the program may be transmitted to computer 301 via a network (not shown) and stored on hard disk 3017. The program is loaded into RAM 3016 at the time of execution. The program may also be loaded directly from CD-ROM 3101 or the network.

[0285] The program does not necessarily include an operating system (OS) or a third-party program that causes the computer 301 to execute the functions of the image processing device 1 of the above-mentioned embodiment. The program only needs to include an instruction portion that calls appropriate functions (modules) in a controlled manner to obtain a desired result. How the computer system 300 operates is well known, and a detailed description will be omitted.

[0286] In addition, in the above program, the steps of transmitting information and receiving information do not include processing performed by hardware, such as processing performed by a modem or interface card in the transmitting step (processing that is performed only by hardware).

[0287] The program may be executed by a single computer or a plurality of computers, that is, centralized processing or distributed processing may be performed.

[0288] Furthermore, in each of the above embodiments, it goes without saying that two or more communication means present in one device may be physically realized by one medium.

[0289] In each of the above embodiments, each process may be realized by centralized processing in a single device, or may be realized by distributed processing in a plurality of devices.

[0290] The present invention is not limited to the above-described embodiment, and various modifications are possible, and it goes without saying that these modifications are also included within the scope of the present invention. [Industrial Applicability]

[0291] As described above, the image processing device 1 of the present invention has the advantage that the intention of the operation on the first image can be reused on the second image, and therefore the second image can be automatically operated in accordance with the user's intention, making it useful as a positional image viewer, etc. [Explanation of symbols]

[0292] 1, 2 Image processing device 11 Storage area 12, 22 Reception 13, 23 Processing section 14, 24 Output section 111 Feature Management Department 112 Image storage section 113 Response Management Department 121 Operation queue reception section 131 Intention sequence acquisition unit 132 Attention Feature Determination Unit 133 Category Acquisition Department 134 Image Determination Unit 135 Target Feature Determination Division 136 Operation section 141 First output section 142 Second output section 221 Intention Reception Department 234 Image Acquisition Unit

Claims

1. a correspondence management unit in which two or more pieces of correspondence information are stored that indicate a correspondence between an operation sequence definition that defines a set of operation identifiers for each of the two or more operations and an intention identifier that identifies the intention of the operation; a first output unit that outputs a first image corresponding to the position information; an operation sequence receiving unit that receives an operation sequence that is a set of operation identifiers that identify two or more operations on the first image; an intention sequence acquisition unit that acquires, from the correspondence management unit, an intention sequence that is an intention identifier corresponding to one or more operation sequence definitions that match the operation sequence accepted by the operation sequence acceptance unit; a second output unit that outputs the second image; an operation unit that performs, on the second image, one or more operations that satisfy an operation sequence definition for the intention sequence acquired by the intention sequence acquisition unit.

2. The response management unit includes: two or more pieces of first correspondence information indicating a correspondence between an operation sequence definition and a chunk identifier that identifies a chunk, which is a collection of operations, and two or more pieces of second correspondence information indicating a correspondence between a chunk sequence definition, which is a definition of a chunk sequence, which is a set of one or more chunk identifiers, and an intent identifier are stored; The intention sequence acquisition unit The image processing device of claim 1, further comprising: obtaining from the two or more pieces of first correspondence information one or more chunk sequences which are a set of one or more chunk identifiers corresponding to one or more operation sequence definitions that match the operation sequence received by the operation sequence receiving unit; and obtaining from the two or more pieces of second correspondence information one or more intention identifiers corresponding to the chunk sequence definitions that match the one or more chunk sequences.

3. an attention feature determination unit that refers to a feature management unit in which two or more pieces of feature information are stored, each having a feature identifier for identifying a feature and two or more feature attribute values ​​including a category of the feature and location information of the feature, and determines an attention feature in the first image that satisfies an attention condition, using the location information corresponding to the first image; a category acquisition unit that acquires a category of the feature of interest from the feature management unit; a target feature determination unit that determines a target feature that is a feature in the second image and that is in the same category as the category acquired by the category acquisition unit, The operation unit includes: The image processing apparatus according to claim 2 , wherein one or more operations that satisfy an operation sequence definition for each of the one or more intention identifiers acquired by the intention sequence acquisition unit are performed on the target feature.

4. The operation unit includes: The image processing apparatus according to claim 3 , further comprising: an image processing device that performs an operation on the target feature using position information of the target feature so as to maintain a positional relationship with respect to the target feature according to an intention identified by the intention identifier.

5. The operation sequence definition is one or more operation identifiers using regular expressions, the chunk sequence definition is one or more chunk identifiers using regular expressions; The operation unit includes: An image processing device as described in claim 4, which uses the position information of the target feature to change the number of repetitions of a chunk identifier, or change the selection of a chunk identifier, or change the number of repetitions of an operation identifier, or change the selection of an operation identifier, so as to maintain a positional relationship with respect to the target feature according to the intention identified by the intention identifier, and performs an operation on the target feature.

6. The apparatus further includes an image determination unit that acquires a location image including a feature of the same category as the category acquired by the category acquisition unit from an image storage unit in which two or more location images are stored, The image processing device according to claim 3 , wherein the second image is the position image acquired by the image determination unit.

7. The image determination unit determining one or more second images corresponding to location information that includes a feature of the same category as the category acquired by the category acquisition unit and that satisfies a location condition for the location information associated with the first image; The operation unit includes: The image processing device according to claim 6 , wherein one or more operations that satisfy an operation sequence definition for each of the one or more intention identifiers acquired by the intention sequence acquisition unit are performed on each of the one or more second images determined by the image determination unit.

8. a correspondence management unit in which two or more pieces of correspondence information indicating the correspondence between an operation sequence definition and an intention identifier are stored; an intention receiving unit that receives intention information including an intention sequence acquired in response to an operation on a first image; a second output unit that outputs the second image; an operation unit that acquires an operation sequence definition for each of one or more intention identifiers contained in the intention sequence contained in the intention information received by the intention receiving unit, and performs one or more operations on the second image that satisfy each of the one or more operation sequence definitions.

9. The intent information also includes a category of the feature of interest; a target feature determining unit that determines a target feature that is a feature in the second image and is a feature of the same category as the category included in the intention information, The operation unit includes: The image processing apparatus according to claim 8 , wherein an operation sequence definition for the intention sequence contained in the intention information is obtained, and one or more operations that satisfy each of the one or more operation sequence definitions are performed on the target feature.

10. The image processing device according to claim 1 , wherein the first image and the second image are different types of position images.

11. The image processing device according to claim 1 , wherein the first image and the second image are a street view image, a three-dimensional image, an interior image, or a drive recorder image.

12. An image processing method implemented by a correspondence management unit storing two or more pieces of correspondence information indicating a correspondence between an operation sequence definition that defines a set of operation identifiers for each of two or more operations and an intention identifier that identifies an intention of the operation, a first output unit, an operation sequence receiving unit, an intention sequence acquisition unit, a second output unit, and an operation unit, a first output step in which the first output unit outputs a first image that is an image associated with position information; an operation sequence receiving step in which the operation sequence receiving unit receives an operation sequence that is a set of operation identifiers that identify two or more operations on the first image; an intention sequence acquisition step in which the intention sequence acquisition unit acquires, from the correspondence management unit, an intention sequence that is an intention identifier corresponding to one or more operation sequence definitions that match the operation sequence accepted by the operation sequence acceptance unit; a second output step in which the second output unit outputs a second image; and an operation step in which the operation unit performs, on the second image, one or more operations that satisfy an operation sequence definition for the intention sequence acquired by the intention sequence acquisition unit.

13. An image processing method implemented by a correspondence management unit in which two or more pieces of correspondence information indicating correspondence between an operation sequence definition and an intention identifier are stored, an intention reception unit, a second output unit, and an operation unit, an intention receiving step in which the intention receiving unit receives intention information including an intention sequence acquired in response to an operation on a first image; a second output step in which the second output unit outputs a second image; an operation step in which the operation unit acquires an operation sequence definition for each of one or more intention identifiers contained in the intention sequence contained in the intention information accepted by the intention accepting unit, and performs one or more operations on the second image that satisfy each of the one or more operation sequence definitions.

14. a computer that can access a correspondence management unit that stores two or more pieces of correspondence information indicating a correspondence between an operation sequence definition that defines a set of operation identifiers for each of two or more operations and an intention identifier that identifies the intention of the operation, a first output unit that outputs a first image corresponding to the position information; an operation sequence receiving unit that receives an operation sequence that is a set of operation identifiers that identify two or more operations on the first image; an intention sequence acquisition unit that acquires, from the correspondence management unit, an intention sequence that is an intention identifier corresponding to one or more operation sequence definitions that match the operation sequence accepted by the operation sequence acceptance unit; a second output unit that outputs the second image; a program for causing the intention sequence acquisition unit to function as an operation unit that performs, on the second image, one or more operations that satisfy an operation sequence definition for the intention sequence acquired by the intention sequence acquisition unit.

15. A computer that can access a correspondence management unit that stores two or more pieces of correspondence information indicating the correspondence between an operation sequence definition and an intention identifier, an intention receiving unit that receives intention information including an intention sequence acquired in response to an operation on a first image; a second output unit that outputs the second image; A program for functioning as an operation unit that acquires an operation sequence definition for one or more intention identifiers contained in the intention sequence contained in the intention information received by the intention receiving unit, and performs one or more operations on the second image that satisfy each of the one or more operation sequence definitions.

Images