Information Processing Apparatus, Information Processing Method, and Computer Program

Abstract

To acquire recognition environment information affecting recognition accuracy of a recognition engine.SOLUTION: An information processing apparatus 100 according to the present invention has a detection unit 101, and an environment acquiring unit 102. The detection unit 101 detects a marker disposed within a range of a recognition target for information acquisition from an image captured by an imaging apparatus for imaging an object positioned within the range of the recognition target. The marker is another object different from a subject or a background in the captured image. The environment acquiring unit 102 acquires recognition environment information based on image information of the detected marker. The recognition environment information is information representing how the recognition object is viewed in the captured image by the imaging apparatus when the imaging apparatus captures the recognition object positioned within the range of the recognition target.SELECTED DRAWING: Figure 14

Description

Technical Field

[0001] The present invention relates to a support technology for image recognition.

Background Art

[0002] With image recognition technology, a computer can recognize various subjects such as people, faces, products, animals, vehicles, obstacles, characters, and two-dimensional codes from images. Various devices have been made to improve the recognition accuracy of such recognition processing. For example, Patent Document 1 proposes a method of adjusting the exposure amounts of a plurality of cameras so that the number of parallax values calculated for a three-dimensional object to be recognized increases in order to improve the recognition accuracy of the three-dimensional object. Further, Patent Document 2 describes devising a feature pattern to be registered in a dictionary in order to improve the accuracy of personal recognition of a face. Specifically, in the configuration shown in Patent Document 2, a feature pattern combining the feature amounts of the target parts in the face area of the subject and the feature amounts of the parts other than the target parts is registered in the dictionary as a feature pattern for identifying the subject.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0004] In order to obtain a recognition engine that realizes high recognition accuracy, various methods have been proposed as described above. However, even if a high-precision recognition engine is delivered to the customer and the operator adjusts the recognition engine to suit the environment of the delivery destination, there are cases where the recognition accuracy of the specifications cannot be achieved. This is because the object to be recognized in the image processed by the recognition engine at the delivery destination has an appearance (reflection) that was not assumed before delivery.

[0005] In this specification, the appearance (reflection) of an object to be recognized in an image processed by a recognition engine is referred to as a recognition environment. The recognition environment is affected by various factors such as the installation position and angle (lens orientation) of the camera, the number and specifications, the lighting conditions within the field of view of the camera, and the position and orientation of the object to be recognized. Depending on such a recognition environment, the recognition engine may not be able to achieve the specified accuracy, which may reduce the reliability of the recognition engine.

[0006] The present invention was conceived to solve the problem that the accuracy of the recognition engine deteriorates due to the recognition environment. That is, the main object of the present invention is to provide a technique for acquiring information on the recognition environment that affects the recognition accuracy of the recognition engine.

Means for Solving the Problem

[0007] To achieve the above object, the information processing apparatus of the present invention includes detection means for detecting a marker arranged within the range of the object to be recognized from an image captured by an imaging device that captures an object located within the range of the object to be recognized, for information acquisition; and environment acquisition means for acquiring recognition environment information, which is information representing the appearance of the object to be recognized in the captured image by the imaging device when the object to be recognized located within the range of the object to be recognized is captured by the imaging device, based on the image information of the detected marker. It is provided with.

[0008] Further, the recognition support method of the present invention a computer detects a marker arranged within the range of the object to be recognized from an image captured by an imaging device that captures an object located within the range of the object to be recognized, for information acquisition, and a computer acquires recognition environment information, which is information representing the appearance of the object to be recognized in the captured image by the imaging device when the object to be recognized located within the range of the object to be recognized is captured by the imaging device, based on the image information of the detected marker.

[0009] Furthermore, the program storage medium of the present invention stores a computer program that causes a computer to execute: a process of detecting a marker arranged within the range of the object to be recognized from an image captured by an imaging device that captures an object located within the range of the object to be recognized, for information acquisition; and a process of acquiring recognition environment information, which is information representing how an object to be recognized located within the range of the object to be recognized appears in a captured image by the imaging device when the object to be recognized is captured by the imaging device, based on the image information of the detected marker. Note that the main object of the present invention can also be achieved by the recognition support method of the present invention corresponding to the information processing device of the present invention. Further, the main object of the present invention can also be achieved by the information processing device of the present invention, a computer program corresponding to the recognition support method of the present invention, and a program storage medium storing the same.

[0010]

Advantages of the Invention

Effects of the Invention

[0011] The information processing device and the recognition support method of the present invention can acquire information on the recognition environment that affects the recognition accuracy of the recognition engine. Therefore, for example, an operator who installs the recognition engine at a customer site can perform adjustments related to the image processing of the recognition engine using the acquired information on the recognition environment. Further, the operator can proceed with the preparation for adjustments related to the image processing of the recognition engine by acquiring the information on the recognition environment in advance. Thereby, the information processing device and the recognition support method of the present invention can suppress the deterioration of the accuracy of the recognition engine and prevent the decrease in reliability of the recognition engine.

Brief Description of the Drawings

[0012]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Embodiments for Carrying Out the Invention

[0013] Hereinafter, embodiments of the present invention will be described with reference to the drawings. Note that each of the embodiments described below is an example, and the present invention is not limited to the configurations of the embodiments described below.

[0014] <First Embodiment> ---Device Configuration--- FIG. 1 conceptually shows the hardware configuration of the information processing apparatus according to the first embodiment of the present invention. The information processing apparatus (hereinafter abbreviated as the support apparatus) 10 is a computer, and as shown in FIG. 1, includes a CPU (Central Processing Unit) 1, a memory 2, an input / output interface I / F (InterFace) 3, and a communication unit 4. These are mutually connected by a bus. Note that this hardware configuration is an example, and the hardware configuration of the support apparatus 10 is not limited to the configuration shown in FIG. 1.

[0015] The CPU 1 is an arithmetic unit, and in addition to a general CPU, the CPU 1 also includes an application specific integrated circuit (ASIC), a digital signal processor (DSP), a graphics processing unit (GPU), etc.

[0016] The memory 2 is a storage device that stores computer programs (hereinafter also referred to as programs) and data. For example, as the memory 2, a random access memory (RAM), a read only memory (ROM), an auxiliary storage device (e.g., a hard disk device), etc. are incorporated in the support apparatus 10.

[0017] The input / output I / F 3 can be connected to a user interface device provided in peripheral devices of the support apparatus 10, such as a display device 5 and an input device 6, and has a function of enabling information communication between the support apparatus 10 and the peripheral devices (display device 5, input device 6). Note that the input / output I / F 3 can also be connected to a portable storage medium or an external storage device.

[0018] The display device 5 is a device that displays drawing data processed by the CPU 1 etc. on a screen. Specific examples of the display device 5 include, for example, a liquid crystal display (LCD) and a cathode ray tube (CRT) display.

[0019] The input device 6 is a device that receives information input by a user operation. Specific examples of the input device 6 include, for example, a keyboard and a mouse.

[0020] Note that the display device 5 and the input device 6 may be integrated. An example of a device integrating the display device 5 and the input device 6 is, for example, a touch panel.

[0021] The communication unit 4 has a function of exchanging information (signals) with other computers and other devices via an information communication network (not shown).

[0022] The support device 10 shown in FIG. 1 has the above-described hardware configuration, but the support device 10 may include hardware elements not shown in FIG. 1. That is, the hardware configuration of the support device 10 is not limited to the configuration of FIG. 1.

[0023] The support device 10 is connected to the camera 7 either via an information communication network or directly. The camera 7 is an imaging device and has a function of transmitting information (video signal) of the captured video (moving image) to the support device 10. In the first embodiment, the orientation (orientation of the lens), height, etc. of the camera 7 are adjusted and installed so that the camera 7 can capture a preset shooting range. Note that the camera 7 may capture a still image and transmit information (video signal) of the still image to the support device 10. Also, the number of cameras 7 may be one or more, and is set as appropriate.

[0024] ---Processing Configuration--- FIG. 2 is a block diagram conceptually showing the control configuration in the support device 10 of the first embodiment. Note that in FIG. 2, the direction of the arrow in the drawing shows an example and does not limit the direction of the signal between the blocks.

[0025] The support device 10 includes an image acquisition unit 11, a detection unit 12, a storage unit 13, an environment acquisition unit 14, a display processing unit 15, and a data generation unit 16. The functional units including the image acquisition unit 11, the detection unit 12, the environment acquisition unit 14, the display processing unit 15, and the data generation unit 16 are realized by, for example, the CPU 1. That is, by executing the program stored in the memory 2 by the CPU 1, the functions of the image acquisition unit 11, the detection unit 12, the environment acquisition unit 14, the display processing unit 15, and the data generation unit 16 are realized. The program is stored in the memory 2 by being imported into the support device 10 from a portable storage medium (for example, a CD (Compact Disc) or a memory card) or another computer connected via an information communication network.

[0026] The storage unit 13 has a function of storing (memorizing) data, and is realized by a storage device such as a RAM or a hard disk device, for example. Information respectively acquired by the image acquisition unit 11, the detection unit 12, and the environment acquisition unit 14 is stored in the storage unit 13. Also, various information used by functional units such as the detection unit 12 and the environment acquisition unit 14 in processing is stored in the storage unit 13.

[0027] The image acquisition unit 11 is provided with a function of acquiring an image photographed by the camera 7. For example, the image acquisition unit 11 captures (imports, and saves the imported video signal in, for example, the storage unit 13) the video signal transmitted from the camera 7 at a preset timing, thereby sequentially acquiring images. The timing for capturing the images is, for example, at predetermined time intervals. Further, the image acquisition unit 11 further has a function of importing the video signal transmitted from the camera 7 and transmitting the video signal to the display device 5. When transmitting the video signal, the image acquisition unit 11 transmits a control signal instructing to display the video signal to the display device 5. Thereby, the display device 5 displays a video (a moving image or a still image) based on the video signal.

[0028] The detection unit 12 has a function of detecting a marker from the image acquired by the image acquisition unit 11 through image processing. Here, the marker is an object placed within the shooting range captured by the camera 7, and has a pattern or color that can be distinguished from the background and subjects other than the marker in the image captured by the camera 7. For example, the storage unit 13 stores in advance identification information of patterns and colors used to distinguish the marker from the background and other subjects. The detection unit 12 uses this information to detect the marker from the image (the captured image by the camera 7) acquired by the image acquisition unit 11 through image processing. Various processes can be considered for the image processing performed by the detection unit 12 to detect the marker. Here, in consideration of the processing capacity of the support device 10 and the color, shape, etc. of the marker to be detected, the image processing executed by the detection unit 12 is appropriately set.

[0029] Here, a specific example of the marker will be described with reference to FIG. 3. Note that the pattern of the marker shown in this FIG. 3 uses one square region partitioned by the dotted line BL as one unit, and dots are formed by coloring the square region. Hereinafter, the pattern of the marker will also be referred to as a dot pattern. Note that the dotted line BL and the dashed-dotted lines DL1, DL2, DL3 shown in FIG. 3 are auxiliary lines shown for convenience of explanation and may not be actually shown. Note that the dot pattern of the marker is not limited to the example of FIG. 3.

[0030] The marker MK shown in FIG. 3 is a pattern (dot pattern) formed within a region of 13 dots in the vertical direction and 9 dots in the horizontal direction, and the dot pattern that is the marker MK is printed on the paper MB. The outer region of the marker MK on the paper MB is the margin of the paper MB.

[0031] The marker MK has dot patterns PT1, PT2 of square shapes (square shapes) with different sizes from each other and a plurality of dot patterns PT3 each consisting of one dot. That is, the marker MK is a pattern formed by a plurality of dot patterns.

[0032] The dot patterns PT1 and PT2 are arranged at intervals along a straight line DL1 connecting their respective center points. Also, the dot patterns PT1 and PT2 are formed by a rectangular black frame pattern formed by black dots, a black square pattern formed at the inner center of this black frame, and a group of white dots formed between the black frame pattern and the black square pattern.

[0033] The plurality of dot patterns PT3 here have different colors from each other. That is, on the marker MK, as dot patterns PT3, a green dot PT3(G), two black dots PT3(K), a blue dot PT3(B), a yellow dot PT3(Y), and a red dot PT3(R) are formed. These six dot patterns PT3 are divided into two arrays of three each. One array has the three dot patterns PT3(Y), PT3(K), PT3(R) arranged with their center positions aligned along a straight line DL2 parallel to the straight line DL1. The other array has the three dot patterns PT3(G), PT3(K), PT3(B) arranged with their center positions aligned along a straight line DL3 parallel to the straight line DL1.

[0034] The storage unit 13 stores information on the marker MK as described above, and the detection unit 12 uses this information to detect the marker from the image.

[0035] The environment acquisition unit 14 acquires recognition environment information based on the image information of the marker detected by the detection unit 12. The recognition environment information is information representing the appearance (how it looks) of the object to be recognized (the object to be detected (recognized) from the captured image) in the captured image by the camera 7. The recognition environment information includes, for example, the number of pixels representing the object to be recognized in the image (also referred to as resolution), the degree of blur, information regarding brightness, color tone information, information such as the tilt angle of the object to be recognized with respect to the direction from the object to be recognized toward the camera 7. The information regarding brightness includes brightness balance, contrast ratio, luminance of white and black, etc. Note that the recognition environment information is not limited to these examples.

[0036] Here, the relationship between the marker and the recognition environment information will be described. That is, the configuration conditions such as the shape and size of the dot pattern constituting the marker are set based on the recognition environment information to be obtained. For example, when it is desired to obtain the number of pixels (resolution) representing the recognition object in the image as the recognition environment information, a dot pattern that satisfies the constraint conditions determined based on the information on the minimum number of pixels that the recognition engine can recognize the recognition object is set as the marker. The information on the minimum number of pixels is, for example, the information on the minimum number of pixels in the vertical and horizontal directions. Alternatively, the information on the minimum number of pixels may be information in which the pixel ratio between the vertical and horizontal directions and the minimum number of pixels in the vertical or horizontal direction are combined. The constraint conditions based on such information on the minimum number of pixels are, for example, the condition that the number of dots in the vertical and horizontal directions in the dot pattern does not exceed the minimum number of pixels in the vertical and horizontal directions. Alternatively, the constraint condition may be the condition that the number of dots in the vertical or horizontal direction in the dot pattern does not exceed the corresponding minimum number of pixels in the vertical or horizontal direction. The dot pattern of the marker is designed so as to satisfy such constraint conditions.

[0037] The reason will be described below using a specific example. FIG. 4 is a diagram for explaining the constraint conditions based on the minimum number of pixels. In this specific example, the recognition object is the human head H. The size of the marker is the same as the size of the recognition object. Also, assume that the minimum number of pixels in the vertical direction that the support device 10 equipped with the recognition engine can recognize the human head H is 20 pixels. In such a case, in order for the support device 10 to recognize the marker from the captured image by the camera 7 by the recognition engine and further obtain the recognition environment information, it is necessary to satisfy the constraint condition that the number of dots N in the vertical direction of the marker (todd pattern) is less than 20 dots.

[0038] That is, here, assume that the marker MK1 in FIG. 4 has a dot pattern with 13 dots in the vertical dot count N. Assume that the marker MK2 has a dot pattern with 35 dots in the vertical dot count N. Assume that such markers MK1 and MK2 are reflected in the captured image by the camera 7 with an image size of 20 pixels or more. In this case, since the 13 dots in the vertical direction in the marker MK1 are reflected in the captured image with a size of at least 20 pixels, in the captured image by the camera 7, 1 dot of the marker MK1 is reflected with 1 pixel or more. Therefore, the marker MK1 can be recognized by the support device 10. On the other hand, since the 35 dots in the vertical direction in the marker MK2 are reflected in the captured image with a size of 20 pixels, in the captured image by the camera 7, 1 dot of the marker MK2 is reflected with less than 1 pixel. Therefore, the marker MK2 is difficult to be recognized by the support device 10. Thus, in this example, based on the constraint condition that the vertical dot count N is 20 or less, the dot pattern of the marker is set.

[0039] Note that even for the same type of object to be recognized (for example, a human head), strictly speaking, there are individual differences. Therefore, as the size of the object to be recognized used when determining the size of the marker, a general size may be used.

[0040] Also, the size of the marker does not have to be the same as the size of the object to be recognized. FIG. 5 is a diagram showing another example of the relationship between the size of the marker and the object to be recognized. In the example of FIG. 5, the entire human body (human) MN is the object to be recognized. The size of the marker MK is the same as the size of the human head. Also, assume that the minimum number of vertical pixels that the recognition engine can recognize the entire human body MN is 42 pixels, and the ratio of the size MK_S of the marker MK to the size MN_S of the entire human body MN is set to 1:6. If, hypothetically, the marker MK is set to the same size as the entire human body MN, then the dot pattern of the marker MK is set such that the vertical dot count is 42 or less. On the other hand, as shown in FIG. 5, when the ratio of the size MK_S of the marker MK to the size MN_S of the entire human body MN is 1:6, the dot pattern of the marker MK is set such that the vertical dot count is 7 (=42 / 6) or less.

[0041] Information on the size ratio between such a marker and the object to be recognized may be stored in advance in the storage unit 13, for example, or may be stored in the storage unit 13 by being input to the support device 10 when the user operates the input device 6 based on an input screen or the like. Further, the information on the size ratio may be stored in the storage unit 13 by being acquired from a portable storage medium or from another computer or the like via the communication unit 4 to the support device 10.

[0042] The environment acquisition unit 14 acquires, as recognition environment information, the number of pixels of the object to be recognized in the captured image by the camera 7, for example, using the number of pixels included in the image area of the detected marker.

[0043] By the way, in the examples of FIGS. 4 and 5, the dot pattern is designed based on a constraint condition regarding the number of vertical dots and the number of horizontal dots is not considered. For this reason, for example, the environment acquisition unit 14 acquires, as recognition environment information, the number of vertical pixels of the object to be recognized based on the number of vertical pixels included in the image area of the detected marker.

[0044] As shown in the example of FIG. 4, when the size of the marker MK is the same as the size of the object to be recognized, the environment acquisition unit 14 acquires the number of vertical pixels included in the image area of the detected marker as the number of vertical pixels (recognition environment information) of the object to be recognized as it is. On the other hand, as shown in FIG. 5, the size of the marker may be different from the object to be recognized. In this case, the environment acquisition unit 14 uses the size ratio between the marker and the object to be recognized to convert the number of vertical pixels included in the image area of the detected marker into the number of pixels corresponding to the size of the object to be recognized. Then, the environment acquisition unit 14 acquires the converted number of pixels as the number of vertical pixels (recognition environment information) of the object to be recognized.

[0045] Note that the constraint conditions regarding the design of the dot pattern may take into account not only the number of vertical dots but also the number of horizontal dots. In this case, for example, the environment acquisition unit 14 may acquire the number of pixels in the vertical and horizontal directions (i.e., recognition environment information) of the object to be recognized based on the number of pixels in the vertical and horizontal directions included in the image area of the marker. Also, the constraint conditions regarding the design of the dot pattern may consider the number of horizontal dots instead of the number of vertical dots. In this case, for example, the environment acquisition unit 14 acquires the number of pixels in the horizontal direction (i.e., recognition environment information) of the object to be recognized based on the number of horizontal pixels included in the image area of the marker. Thus, the environment acquisition unit 14 acquires, as recognition environment information, the number of pixels in at least one of the vertical and horizontal directions of the object to be recognized according to the constraint conditions regarding the design of the dot pattern.

[0046] When information regarding the degree of blur or brightness is acquired as recognition environment information, it is desirable that the marker be formed by a dot pattern including a white dot group and a black dot group. The reason is that the difference in luminance between white and black is large, and it prominently represents the degree of blur and brightness information for each environment. In such a case, the environment acquisition unit 14 acquires, as recognition environment information, information regarding the degree of blur and brightness at the recognition target position in the captured image by the camera 7 based on the image information of the white dots and black dots included in the detected marker.

[0047] For example, the environment acquisition unit 14 acquires the edge intensity from a plurality of portions in the image where black dots and white dots are adjacent. For such acquisition of edge intensity, for example, a Sobel filter, a Prewitt filter, or the like is used. The environment acquisition unit 14 calculates the degree of blur based on the average of the edge intensities acquired in this way and acquires the calculated degree of blur as recognition environment information. Note that the environment acquisition unit 14 may calculate the ratio of the number of edge intensities exceeding the threshold value to the total number of acquired edge intensities as the degree of blur and acquire the calculated degree of blur as recognition environment information. Thus, there are various methods for calculating the degree of blur, and it is not limited to such methods.

[0048] The environment acquisition unit 14 can calculate the brightness balance, contrast ratio, and average luminance of white and black based on the image of the marker as information regarding brightness. For example, the environment acquisition unit 14 calculates the luminance for each white dot and each black dot in the image of the marker, and calculates the average luminance of white and the average luminance of black.

[0049] Also, the environment acquisition unit 14 can calculate the ratio of the average luminance of white to the average luminance of black in the image of the marker as the contrast ratio. Further, the environment acquisition unit 14 calculates the sum of the average luminance (average brightness) of white and the average luminance (average brightness) of black in the image of the marker. Here, assume that the luminance (brightness) is represented using numbers from 0 to 255, for example, and black with the minimum luminance is "0", and white with the maximum luminance is "255". In this case, the environment acquisition unit 14 can calculate the numerical value obtained by subtracting "255" from the sum of the calculated average luminance of white and black as the brightness balance. When it is clear that the black dots are black and the white dots are white, the brightness balance is a value of 0 or in the vicinity thereof, and when the image is too dark or too bright, it is a positive or negative numerical value corresponding to the degree. Note that the specific information regarding brightness and its calculation method are not limited to the above example.

[0050] The environment acquisition unit 14 may acquire, as recognition environment information, the inclination angle of the front direction of the pattern (the normal direction of the surface on which the pattern is formed (printed)) with respect to the direction from the object to be recognized toward the camera 7 (hereinafter, also referred to as the angle information of the object to be recognized). In this case, the marker has a shape capable of acquiring information on the direction from a reference point set in, for example, a rectangular coordinate system in a three-dimensional space toward the marker. One of the markers (dot patterns) having such a shape is, for example, the dot patterns PT1 and PT2 shown in FIG. 3. That is, the dot patterns PT1 and PT2 are rectangular shapes with four right-angled vertices. Also, the dot patterns PT1 and PT2 are formed on a plane. The environment acquisition unit 14 calculates a homography transformation matrix, for example, using the positional relationship of the four vertices of the dot pattern PT1 (or PT2) in the captured image and the actual positional relationship of those four vertices. Then, the environment acquisition unit 14 calculates the angle information of the object to be recognized using the homography transformation matrix and the position information where the marker is projected in the captured image.

[0051] When color information is acquired as recognition environment information, the marker is formed by a dot pattern including red dots, green dots, and blue dots. In the example of FIG. 3, the marker includes a green dot PT3(G), a blue dot PT3(B), a yellow dot PT3(Y), and a red dot PT3(R) as the dot pattern PT3. The environment acquisition unit 14 acquires, as recognition environment information, the color information at the recognition target position of the captured image based on the image information of the red dots, green dots, and blue dots included in the detected marker. Specifically, for example, the environment acquisition unit 14 acquires the actual RGB values of the red dots, green dots, and blue dots included in the marker. Then, the environment acquisition unit 14 uses the deviation amount between the actual RGB values and the respective RGB values included in the marker in the image as the color information and acquires the color information as recognition environment information. Note that the calculation method of the color information is not limited.

[0052] As described above, by using the dot pattern shown in FIG. 3, the environment acquisition unit 14 can acquire at least the number of pixels of the object to be recognized, the degree of blurring, brightness information, angle information of the object to be recognized, and color information in the captured image as recognition environment information. Note that the environment acquisition unit 14 may acquire recognition environment information other than these.

[0053] The display processing unit 15 has a function of controlling the display device 5. For example, the display processing unit 15 causes the display device 5 to display the captured image on the screen of the display device 5 by transmitting the image acquired from the camera 7 by the image acquisition unit 11 to the display device 5. At the same time, the display processing unit 15 causes the display device 5 to display the recognition environment information acquired by the environment acquisition unit 14 on the screen of the display device 5 in a state where the recognition environment information is superimposed on the captured image. The display position of the recognition environment information is set based on, for example, the arrangement position of the marker in the captured image. For example, the recognition environment information is displayed so as to at least partially overlap the image of the marker. Or, the recognition environment information is displayed at a position in the vicinity with an interval from the image of the marker. In this way, by superimposing the recognition environment information on the image of the marker or displaying it in the vicinity of the image of the marker, the correspondence relationship between the marker and the recognition environment information becomes visible.

[0054] Note that the display processing unit 15 may also cause the display device 5 to display the lead line on the captured image displayed on the display device 5 by transmitting the information of the lead line that connects the image of the marker and the displayed recognition environment information to the display device 5. Further, the display processing unit 15 may display the identification number in the vicinity of the image of the marker and display the recognition environment information together with the identification number in the image area away from the image of the marker. That is, here, as long as it is visible from the display image of the display device 5 that the marker and the recognition environment information acquired from the marker are related, the display position of the recognition environment information is not limited.

[0055] The display processing unit 15 may represent the recognition environment information to be displayed on the display device 5 in color. For example, the display processing unit 15 displays a semi-transparent color mark corresponding to the numerical value of the recognition environment information on the screen of the display device 5. Further, when a plurality of markers are reflected in the captured image by the camera 7, the display processing unit 15 can generate a map (heat map) on the image representing the recognition environment information by displaying the recognition environment information associated with each marker as a color mark.

[0056] FIG. 6 is a diagram showing an example of a display screen with the recognition environment information superimposed. In the example of FIG. 6, the object to be recognized is a human head, and a marker MK having the same size as the human head is arranged at the same position as the human head. The number of pixels (45 pixels) of the object to be recognized is superimposed and displayed in the vicinity of the marker MK as the recognition environment information RE. In the example of FIG. 6, one piece of recognition environment information is displayed, but a plurality of pieces of recognition environment information may be displayed.

[0057] Furthermore, the display processing unit 15 has a function of displaying a virtual image on the display device 5. The virtual image is an image obtained by performing image processing on a model image of the object to be recognized based on the recognition environment information acquired by the environment acquisition unit 14. The model image of the object to be recognized is, for example, an image pre-stored in the storage unit 13, and is, for example, a replica image such as a photographic image of the object to be recognized, a painting representing the object to be recognized, or a CG (Computer Graphics) image. The display processing unit 15 generates a virtual image by performing image processing that clarifies what the image will look like (be photographed) when the acquired recognition environment information is reflected in the model image. All of the acquired recognition environment information may be reflected in the virtual image, or a part of the recognition environment information may be reflected. By the display processing unit 15 displaying such a virtual image on the display device 5, the support device 10 can present the recognition environment information to the operator (user) in a state where it is easy to grasp. Note that the display processing unit 15 can display the virtual image in an easy-to-understand manner by displaying the virtual image in a predetermined size regardless of the size of the object to be recognized in the captured image.

[0058] FIG. 7 is a diagram showing a specific example of a pseudo image. In the example of FIG. 7, the object to be recognized is a human head. The CG image showing the human head is the model image of the object to be recognized (denoted as the original image in FIG. 7). The display processing unit 15 generates a pseudo image by performing image processing on the model image based on the recognition environment information.

[0059] For example, the display processing unit 15 performs image processing on the model image to reduce the resolution (number of pixels) of the model image to the resolution (number of pixels) of the object to be recognized acquired as the recognition environment information while maintaining the image size of the model image. By such image processing, a pseudo image such as image A in FIG. 7 is generated.

[0060] In addition, the display processing unit 15 performs image processing on the model image to make the degree of blur of the model image the same as the degree of blur acquired as the recognition environment information. By such image processing, a pseudo image such as image B in FIG. 7 is generated. Also, the display processing unit 15 performs image processing on the model image (image of the head) to rotate it at the angle of the object to be recognized acquired as the recognition environment information. By such image processing, a pseudo image such as image C in FIG. 7 is generated. Furthermore, the display processing unit 15 performs image processing on the model image to change the contrast of the model image according to the contrast ratio acquired as the recognition environment information. By such image processing, a pseudo image such as image D in FIG. 7 is generated. There are various methods for such image processing performed on the model image, and an appropriate method is adopted as appropriate.

[0061] The data generation unit 16 has a function of generating print data for printing a marker on a sheet. The storage unit 13 stores the image information of the marker, and the data generation unit 16 extracts the image information of the marker from the storage unit 13 and generates print data based on this image information. This print data may be a data file for printing the image of the marker on an application such as word processing software or slide creation software. Also, the print data may be unique data without using such an application. As shown in the examples of FIGS. 4 and 5, when the marker is set to a size corresponding to a human head, the data generation unit 16 generates, for example, print data for printing the marker on a vertically oriented A4-sized sheet.

[0062] ---Operation Example / Recognition Support Method--- Hereinafter, the recognition support method in the first embodiment will be described with reference to FIGS. 8 and 9.

[0063] FIG. 8 is a diagram showing an example of using a marker. FIG. 9 is a flowchart showing an operation example of the support device 10 in the first embodiment. This flowchart represents an example of the processing procedure of a computer program executed by the CPU 1 of the support device 10.

[0064] Before the support device 10 performs the operations described below, as shown in FIG. 8, the camera 7 included in the recognition system is installed at an installation location based on the specifications. Also, a marker MK is prepared in consideration of the size of the recognition object, the recognition accuracy of the recognition engine, etc. In the example of FIG. 8, the recognition object is a human head, and the marker MK has the same size as a human head. The operator OP holds the marker MK at the height position of the head, which is the recognition object, and stays at a location within the recognition target range that the recognition system is targeting. That is, the operator OP repeats actions such as moving and stopping within the recognition target range of the recognition system while holding the marker MK in order to investigate the difference in the recognition environment due to the location within the recognition target range of the recognition system. The camera 7 continuously or intermittently captures the actions of the operator OP within the field of view FV.

[0065] The support device 10 acquires an image (video or still image) by the image acquisition unit 11 capturing a video signal representing a captured image from the camera 7 (S81). Note that the video signal from the camera 7 may be directly transmitted to the support device 10, or may be transmitted via an information communication network or a storage device such as a portable storage medium.

[0066] The detection unit 12 of the support device 10 detects a marker from the acquired image (S82).

[0067] The environment acquisition unit 14 of the support device 10 acquires recognition environment information of a recognition target (human head) at the position PS within the field of view FV of the camera 7 based on the image of the detected marker (S83). That is, the environment acquisition unit 14 of the support device 10 acquires, for example, the number of pixels (resolution) representing the recognition target, the degree of blur, information regarding brightness, color information, the angle of the recognition target, etc. as recognition environment information.

[0068] The display processing unit 15 of the support device 10 causes the screen of the display device 5 to display an image in which the recognition environment information acquired by the environment acquisition unit 14 is superimposed on the captured image of the camera 7 (S84). The recognition environment information to be displayed is all or part of the acquired recognition environment information.

[0069] Furthermore, the display processing unit 15 of the support device 10 generates a pseudo-image representing the appearance of the recognition target by performing image processing on a model image using the recognition environment information acquired by the environment acquisition unit 14, and causes the display device 5 to display this pseudo-image (S85).

[0070] FIG. 10 is a diagram showing an example of displaying recognition environment information. In the example of FIG. 10, the number of pixels (resolution) of the recognition target is displayed by numerical values and a pseudo-image. As information regarding brightness, the balance of lightness, the contrast ratio, and the white and black luminance (lightness) are represented, and these are displayed by numerical values and a graph.

[0071] In the example of FIG. 10, the average luminance of black is calculated as "77.6", and the average luminance of white is calculated as "246.2". The brightness balance is calculated by subtracting 255 from the sum of the average luminance of black "77.6" and the average luminance of white "246.2" so that the minimum value is "-255" and the maximum value is "255". This calculated "68.8" is displayed on the screen of the display device 5 as the balance. Also, the contrast ratio is calculated by obtaining the ratio of the average luminance of white to the average luminance of black, and is displayed as "3.17". The degree of blur is calculated based on the average of the edge strengths of a plurality of portions where black dots and white dots are adjacent in the dot pattern of the marker. Here, the average of the edge strengths is normalized to a numerical value from 0 to 1, and the degree of blur is calculated by subtracting the normalized numerical value from the numerical value "1" so that the numerical value "1" represents blurring. In the example of FIG. 10, the degree of blur is displayed as "0.551".

[0072] For each of the red, green, and blue colors in the recognized marker, the amount of deviation from the original RGB value of the marker is calculated, and the amount of deviation is displayed by a numerical value (RGB value) respectively.

[0073] The support device 10 acquires (detects) the recognition environment information based on the captured image of the camera 7 as described above, and displays the acquired recognition environment information. Note that the detection method of the recognition environment information is not limited to the method described above. Also, the display method of the recognition environment information is not limited to the example of FIG. 10. Further, in FIG. 9, a plurality of processes (steps) S81 to S85 are shown in order, but some of these processes may be executed in parallel or in a different order. For example, process S84 and process S85 may be executed in parallel. Also, process S84 and process S85 may be executed in the reverse order. Further, one of process S84 and process S85 may be omitted.

[0074] ---Effects in the First Embodiment--- As described above, the support device 10 of the first embodiment detects a marker from an image captured by the camera 7, and acquires recognition environment information within the recognition target range by the recognition system based on the image of the detected marker. The recognition environment information is information representing the appearance of the recognition target object in the captured image by the camera 7, and is information related to the performance of the camera 7 and the brightness within the recognition target range, etc., which affects the recognition accuracy of the recognition engine.

[0075] In other words, the support device 10 of the first embodiment can acquire the recognition environment information within the recognition target range by a simple operation of posting a marker within the recognition target range by the recognition system without using the recognition engine. Also, the support device 10 can acquire quite detailed recognition environment information such as the resolution of the recognition target object, information regarding brightness, degree of blurring, color information, etc. For this reason, the support device 10 enables an appropriate pre-evaluation of the recognition engine of the recognition system before constructing the recognition system. Furthermore, by confirming the recognition environment information, the support device 10 enables adjustment of the number of cameras 7 constituting the recognition system, the installation position, the orientation, and parameters such as the shutter speed, aperture, and white balance of the cameras 7. Thereby, the adjustment work after constructing the recognition system is reduced, and the recognition system can be incorporated smoothly.

[0076] Also, the size and the way of using (posting method) the marker are set based on the size and height position of the recognition target object. Furthermore, the shape and color of the dot pattern constituting the marker are set based on the content of the required recognition environment information and the minimum number of pixels that the recognition engine can recognize the recognition target object. In this way, since the configuration of the marker to be used can be set generally, the support device 10 is applicable to the acquisition of recognition environment information in various systems using image recognition.

[0077] In the above description, the recognition target object was a person, but the recognition target object is not limited to a person, such as a vehicle, a product, an animal, letters, symbols, etc. Also, the recognition algorithm, recognition method, etc. of the recognition engine are not limited.

[0078] Furthermore, the support device 10 of the first embodiment can superimpose the recognition environment information on the captured image by the camera 7 and display it on the screen in a state where the association between the marker detected from the captured image by the camera 7 and the recognition environment information obtained from the marker is visible. Thereby, the support device 10 can easily let the operator grasp where in the captured image by the camera 7 the acquired recognition environment information is located. Furthermore, the captured image of the camera 7 displayed on the display device 5 may be a real-time video. In this case, the support device 10 can easily let the operator grasp the association between the marker and the recognition environment information by displaying the recognition environment information obtained from the image in real time on the video (image).

[0079] Also, the support device 10 can generate a virtual image by performing image processing on a model image of the recognition object using the acquired recognition object information, and display the generated virtual image on the display device 5. Thereby, the support device 10 can easily represent how the recognition object actually looks like by the virtual image. That is, the support device 10 can let the operator sensuously grasp the recognition environment information through vision.

[0080] <Second Embodiment> Hereinafter, a second embodiment according to the present invention will be described. In the description of this second embodiment, the same reference numerals are given to the same-named parts as those in the first embodiment, and the overlapping description of the common parts is omitted.

[0081] In this second embodiment, the support device 10 has a function of acquiring the relative position of the marker with respect to the position of the camera 7 from the captured image by the camera 7, associating this relative position with the recognition environment information, and displaying it on the display device 5. The details will be described below.

[0082] ---Processing Configuration--- FIG. 11 is a block diagram conceptually showing the control configuration of the support device (information processing device) 10 according to the second embodiment. In FIG. 11, the direction of the arrow in the drawing is shown as an example and does not limit the direction of the signal between the blocks.

[0083] In addition to the configuration of the first embodiment, the support device 10 of this second embodiment further includes a position acquisition unit 17. This position acquisition unit 17 is also realized by, for example, the CPU 1.

[0084] The position acquisition unit 17 acquires position information representing the relative position of the marker with respect to the position of the camera 7. That is, the position acquisition unit 17 can calculate the relative position information of the marker based on the image information of the marker detected by the detection unit 12, the height information of the marker, and the parameters of the camera 7. The height information of the marker is information representing the height from the floor surface to the displayed marker. For example, the height (length) from the floor surface to the center point of the marker is the height information of the marker. Such height information may be stored in advance in the storage unit 13, or height information input by, for example, an operator (user) using the input device 6 may be stored. The parameters of the camera 7 are information representing the position, posture (direction of the lens), etc. of the camera 7. Such parameters may be stored in advance in the storage unit 13, or parameters acquired by communication from the camera 7 may be stored. Further, the storage unit 13 may store the parameters of the camera input by the operation of the operator using the input device 6.

[0085] The position acquisition unit 17 acquires the center coordinates of the detected marker based on the image of the marker. Then, the position acquisition unit 17 uses the center coordinates, the height information of the marker, and the parameters of the camera to calculate, for example, position information in three-dimensional coordinates representing the relative position of the marker with respect to the position of the camera 7 by planar projective transformation. Alternatively, the position acquisition unit 17 may project the position information in the three-dimensional coordinates calculated as such onto a two-dimensional plane parallel to the floor surface, and use the position information in this two-dimensional plane as the relative position information of the marker.

[0086] In addition, the position acquisition unit 17 can also acquire the relative position information of the marker by other methods. For example, the position acquisition unit 17 can also acquire the above position information by the input operation of the operator using the input device 6. For example, the position acquisition unit 17 causes the display device 5 to display a list in which the recognition environment information stored in the storage unit 13 is associated with the image from which the recognition environment information is extracted. Then, the position acquisition unit 17 causes the operator to input the relative position information of the marker corresponding to the recognition environment information using the input device 6. In this case, in order to obtain the relative position information of the marker, the height information of the marker and the information of the camera parameters do not need to be used and thus do not have to be acquired.

[0087] The relative position information of the marker thus acquired (calculated) is stored in the storage unit 13 in a state associated with the recognition environment information acquired from the image of the marker. In other words, the storage unit 13 stores the recognition environment information together with the relative position information of the marker.

[0088] The display processing unit 15 maps the position of the marker (the relative position of the marker with reference to the position of the camera 7) on the screen of the display device 5 (also referred to as a map screen) on which two-dimensional or three-dimensional coordinates based on the position of the camera 7 are displayed. In addition, the display processing unit 15 displays the recognition environment information associated with the marker in association with the mapping position of the marker. That is, the display processing unit 15 displays how the recognition environment information differs depending on the position based on the camera 7. Note that the displayed position information and recognition environment information of the marker may be all the acquired information or a part thereof.

[0089] FIG. 12 is a diagram showing an example of the display of the map screen. In the example of FIG. 12, the positions of the markers are plotted as points on a graph of two-dimensional coordinates in a plane parallel to the floor surface. And around the points representing the positions of the markers, the recognition environment information corresponding to the positions (in this example, the number of pixels (resolution) of the object to be recognized) is displayed numerically. Further, in the example of FIG. 12, the position CP and the field of view range FV of the camera 7 are displayed. Note that FIG. 12 is an example, and either one or both of the position CP and the field of view range FV of the camera 7 may not be displayed. Also, in the example of FIG. 12, the positions of the markers are represented by points and the recognition environment information is represented numerically. Instead of this, the recognition environment information may be displayed by a heat map display in which the positions of the markers associated with the recognition environment information are colored with colors corresponding to the numerical values of the recognition environment information.

[0090] ---Operation Example / Recognition Support Method--- Hereinafter, an operation example of the support device 10 of the second embodiment will be described with reference to FIG. 13. FIG. 13 is a flowchart showing an operation example of the support device 10 in the second embodiment. This flowchart represents the processing procedure executed by the CPU 1 of the support device 10.

[0091] The processes S131, S132, S134, S135, and S136 in FIG. 13 are the same processes as the processes S81, S82, S83, S84, and S85 in FIG. 9, and the description thereof will be omitted.

[0092] In process S133, the position acquisition unit 17 of the support device 10 acquires the position information of the marker based on the image information of the marker detected by the detection unit 12, the height information of the marker, and the parameters of the camera 7 (such as the position and the orientation of the lens). The acquired position information of the marker is information representing the relative position of the marker with respect to the position of the camera 7.

[0093] In process S137, the display processing unit 15 of the support device 10 causes the display device 5 to display a map screen based on the correspondence between the position information acquired by the position acquisition unit 17 and the recognition environment information acquired by the environment acquisition unit 14. Three-dimensional coordinates or two-dimensional coordinates are represented on this map screen.

[0094] In FIG. 13, a plurality of processes (steps) are shown in order, but the processes executed in the second embodiment and the execution order of those processes are not limited to the example of FIG. 13. For example, processes S135, S136, and S137 may be executed in parallel or in another order. Also, any one or two of processes S135, S136, and S137 may be omitted.

[0095] ---Effects in the Second Embodiment--- As described above, the support device 10 of the second embodiment can display (map) a map screen in which the recognition environment information is displayed at the position of the marker associated with the recognition environment information.

[0096] Therefore, the support device 10 of the second embodiment can make it easier to grasp the positional relationship of the recognition environment information with respect to the camera 7 and can clearly understand the recognizable range based on the camera.

[0097] ---Modification Example--- In addition to the configurations of the first or second embodiment, the support device 10 may be configured to add information on temporal changes to the recognition environment information. For example, an operator holds up a marker at the same position at different times or installs it fixedly. Thereby, based on the images of the marker at the same position in each image taken at different times, the environment acquisition unit 14 acquires the recognition environment information at each time (time of day or date). In this case, the storage unit 13 stores the shooting time of the image of the marker, and the shooting time of the image of the marker that is the basis is associated with the recognition environment information stored in the storage unit 13. Further, when the display processing unit 15 causes the display device 5 to display the recognition environment information, the shooting time of the image of the marker that is the basis may also be displayed. In this way, by presenting the temporal change of the recognition environment at the same position, the support device 10 can acquire information on the temporal change of the recognition environment associated with the temporal change of external light such as sunlight or shadows.

[0098] <Third Embodiment> The third embodiment according to the present invention will be described below.

[0099] FIG. 14 is a block diagram conceptually showing the control configuration of the information processing apparatus in the third embodiment. Note that in FIG. 14, the direction of the arrow in the drawing shows an example and does not limit the direction of the signal between the blocks.

[0100] The information processing apparatus 100 in the third embodiment includes a detection unit 101 and an environment acquisition unit 102. The information processing apparatus 100 has, for example, the same hardware configuration (see FIG. 1) as the support device 10 in the first or second embodiment. The detection unit 101 and the environment acquisition unit 102 are realized, for example, when the CPU 1 executes a computer program. Note that the camera 7, the display device 5, and the input device 6 may not be connected to the information processing apparatus 100.

[0101] The detection unit 101 has a function of detecting marker image information from a captured image obtained by an imaging device (camera 7), for example, by image processing. The captured image (video data or still image data) may be configured to be input directly from the camera 7 to the information processing device 100, or may be configured to be input to the information processing device 100 via another device or a portable storage medium.

[0102] The environment acquisition unit 102 has a function of acquiring recognition environment information based on the marker image information detected by the detection unit 101. The recognition environment information is information indicating in what state an object to be recognized is captured when the object to be recognized located at the position of the marker is captured by the imaging device. For example, the recognition environment information is various types of information as described in the first and second embodiments.

[0103] FIG. 15 is a flowchart showing an operation example of the information processing device 100 in the third embodiment. Note that this flowchart shows an example of the processing procedure of a computer program executed by the CPU 1 of the information processing device 100.

[0104] For example, the information processing device 100 (detection unit 101) detects marker image information from a captured image obtained by the imaging device (S151). Then, the information processing device 100 (environment acquisition unit 102) acquires recognition environment information based on the detected marker image information (S152).

[0105] Note that the information processing device 100 can display the acquired recognition environment information on the screen of the display device by the same method as in the first and second embodiments. Further, the information processing device 100 may cause the acquired recognition environment information to be printed by a printing device, or may store it as a file in a storage device such as a portable storage medium. Furthermore, the information processing device 100 may transmit the acquired recognition environment information to another computer by communication. When the recognition environment information is output (displayed, printed, presented) in this way, the extraction source image may be associated with the recognition environment information, and further, the position information of the marker may also be associated therewith.

[0106] The information processing apparatus 100 according to the third embodiment can achieve the same effects as those of the first and second embodiments.

[0107] The present invention has been described by taking the above-described embodiments as exemplary examples. However, the present invention is not limited to the above-described embodiments. That is, within the scope of the present invention, various aspects understandable by those skilled in the art can be applied.

[0108] This application claims priority based on Japanese Patent Application No. 2014-173111 filed on August 27, 2014, and incorporates all of its disclosures herein.

[0109] Some or all of the above-described embodiments may be described as follows in the appended claims, but are not limited thereto.

[0110] (Appended Claim 1) Detection means for detecting a marker existing at a certain position within the field of view of the imaging device from an image acquired from the imaging device, Environment acquisition means for acquiring recognition environment information indicating how a recognition target at the position appears in an image photographed by the imaging device, based on the image information of the detected marker, An information processing apparatus comprising the same.

[0111] (Appended Claim 2) The marker is formed by a dot pattern that satisfies a dot number constraint based on the minimum number of pixels by which a recognition engine can recognize a recognition target, The environment acquisition means acquires, as the recognition environment information, the number of pixels of the recognition target at the position, using the number of pixels included in the image area of the detected marker. The information processing apparatus according to Appended Claim 1.

[0112] (Appended Claim 3) The environment acquisition means, when the size of the marker is the same as that of the recognition target, acquires the number of pixels included in the image area of the detected marker as the number of pixels of the recognition target at the position. When the size of the marker is different from that of the object to be recognized, the number of pixels included in the image area of the detected marker is converted by the size ratio between the marker and the object to be recognized, thereby obtaining the number of pixels of the object to be recognized at the position. The information processing apparatus according to Supplementary Note 2.

[0113] (Supplementary Note 4) The dot pattern of the marker includes a white dot group and a black dot group. Based on the image information of the white dots and black dots included in the detected marker, the environment acquisition means acquires information regarding the degree of blurring and brightness at the position as the recognition environment information. The information processing apparatus according to Supplementary Note 2 or Supplementary Note 3.

[0114] (Supplementary Note 5) The dot pattern of the marker represents a shape from which three mutually orthogonal directions from a certain reference point can be constantly acquired. Based on the image information of the dot pattern of the detected marker, the environment acquisition means acquires the angle of the object to be recognized with respect to the shooting direction of the imaging device as the recognition environment information. The information processing apparatus according to any one of Supplementary Notes 2 to 4.

[0115] (Supplementary Note 6) The dot pattern of the marker includes red dots, green dots, and blue dots. Based on the image information of the red dots, green dots, and blue dots included in the detected marker, the environment acquisition means acquires the color information at the position as the recognition environment information. The information processing apparatus according to any one of Supplementary Notes 2 to 5.

[0116] (Supplementary Note 7) The dot pattern of the marker includes a square first partial dot pattern, a square second partial dot pattern smaller than the first partial dot pattern, and a plurality of square third partial dot patterns each formed by dots of one color. The first partial dot pattern and the second partial dot pattern are arranged vertically such that a first straight line connecting their respective center points is perpendicular to each side. The plurality of third partial dot patterns are divided into a first group arranged on the right side with the second partial dot pattern in between and a second group arranged on the left side. In each of the first and second groups, the third partial dot patterns are arranged vertically such that a straight line connecting their respective center points is parallel to the first straight line. The first partial dot pattern and the second partial dot pattern are formed from a square black frame formed by black dots, a square white frame formed by white dots adjacent to the inside of the black frame, and a group of square black dots adjacent to the inside of the white frame. The plurality of third partial dot patterns include at least red dots, green dots, blue dots, and yellow dots. The information processing apparatus according to any one of Appendices 2 to 6.

[0117] (Appendix 8) When the object to be recognized is a human head or face, it further includes data generation means for generating data for printing the marker on a vertically oriented A4 paper. The information processing apparatus according to any one of Appendices 1 to 7.

[0118] (Appendix 9) It further includes display processing means for causing a display unit to display a display screen in which the recognition environment information is superimposed on an image acquired from the imaging device in a state where the correspondence with the detected marker is visible. The information processing apparatus according to any one of Appendices 1 to 8.

[0119] (Appendix 10) Position acquisition means for acquiring position information indicating the relative position of the marker with respect to the position of the imaging device, Based on a plurality of pieces of recognition environment information acquired for a plurality of positions within the field of view of the imaging device, display processing means for causing a display unit to display a display screen on which the plurality of pieces of recognition environment information are drawn in coordinates representing a relative positional relationship with reference to the position of the imaging device is further provided. The information processing apparatus according to any one of Appendices 1 to 9.

[0120] (Appendix 11) The display processing means represents at least one of the position and the field of view of the imaging device in the coordinates. The information processing apparatus according to Appendix 10.

[0121] (Appendix 12) The position acquisition means acquires the position information indicating a position on a plane parallel to the floor surface corresponding to the position where the marker exists, based on the image information of the detected marker, the height information of the marker, and the camera parameters of the imaging device. The information processing apparatus according to Appendix 10 or 11.

[0122] (Appendix 13) Further provided is display processing means for causing a display unit to display a pseudo-image obtained by applying image processing to an original image showing a recognition object in a normal state so that the recognition object in the original image appears as shown by the acquired recognition environment information. The information processing apparatus according to any one of Appendices 1 to 12.

[0123] (Appendix 14) Further provided is display processing means for associating a plurality of pieces of recognition environment information acquired for the same position within the field of view of the imaging device from each image taken at different times with the shooting time of each image and causing the display unit to display the information. The information processing apparatus according to any one of Appendices 1 to 12.

[0124] (Appendix 15) In a recognition support method executed by a computer, Detect a marker existing at a certain position within the field of view of the imaging device from an image acquired from the imaging device. Based on the image information of the detected marker, obtain recognition environment information indicating how the object to be recognized appears in the image captured by the imaging device at the position. A recognition support method including this.

[0125] (Appendix 16) The marker is formed by a dot pattern that satisfies the dot number constraint based on the minimum number of pixels by which the recognition engine can recognize the object to be recognized. The acquisition of the recognition environment information includes acquiring the number of pixels of the object to be recognized at the position by using the number of pixels included in the image area of the detected marker. The recognition support method according to Appendix 15.

[0126] (Appendix 17) The acquisition of the recognition environment information When the size of the marker is the same as that of the object to be recognized, acquire the number of pixels included in the image area of the detected marker as the number of pixels of the object to be recognized at the position. When the size of the marker is different from that of the object to be recognized, acquire the number of pixels of the object to be recognized at the position by converting the number of pixels included in the image area of the detected marker by the size ratio between the marker and the object to be recognized. The recognition support method according to Appendix 16 including this.

[0127] (Appendix 18) The dot pattern of the marker includes a white dot group and a black dot group. The acquisition of the recognition environment information includes acquiring information regarding the degree of blur and brightness at the position based on the image information of the white dots and black dots included in the detected marker. The recognition support method according to Appendix 16 or Appendix 17.

[0128] (Appendix 19) The dot pattern of the marker represents a shape from which three mutually orthogonal directions from a certain reference point can be constantly obtained. The acquisition of the recognition environment information includes acquiring the angle of the object to be recognized with respect to the shooting direction of the imaging device based on the image information of the dot pattern of the detected marker. The recognition support method according to any one of Appendices 16 to 18.

[0129] (Appendix 20) The dot pattern of the marker includes red dots, green dots, and blue dots. The acquisition of the recognition environment information includes acquiring the color information at the position based on the image information of the red dots, green dots, and blue dots included in the detected marker. The recognition support method according to any one of Appendices 16 to 19.

[0130] (Appendix 21) The dot pattern of the marker includes a square first partial dot pattern, a square second partial dot pattern smaller than the first partial dot pattern, and a plurality of square third partial dot patterns each formed by dots of one color. The first partial dot pattern and the second partial dot pattern are arranged vertically such that a first straight line connecting their respective center points is orthogonal to each side. The plurality of third partial dot patterns are divided into a first group arranged on the right side with the second partial dot pattern in between and a second group arranged on the left side. In each of the first and second groups, each third partial dot pattern is arranged vertically such that a straight line connecting their respective center points is parallel to the first straight line. The first partial dot pattern and the second partial dot pattern are formed by a black frame in the shape of a square formed by black dots, a white frame in the shape of a square formed by white dots adjacent to the inside of the black frame, and a group of black dots in the shape of a square adjacent to the inside of the white frame. The plurality of third partial dot patterns include at least red dots, green dots, blue dots, and yellow dots. The recognition support method according to any one of Appendices 16 to 20.

[0131] (Appendix 22) When the object to be recognized is a human head or face, generate data for printing the marker on a vertically oriented A4 sheet of paper. The recognition support method according to any one of Appendices 15 to 21, further comprising this.

[0132] (Appendix 23) Cause the display unit to display a display screen in which the recognition environment information is superimposed on the image acquired from the imaging device in a state where the correspondence with the detected marker is visible. The recognition support method according to any one of Appendices 15 to 22, further comprising this.

[0133] (Appendix 24) Obtain position information indicating the relative position of the marker with respect to the position of the imaging device. Based on a plurality of pieces of recognition environment information acquired for a plurality of positions within the field of view of the imaging device, cause the display unit to display a display screen on which the plurality of pieces of recognition environment information are drawn on coordinates representing the relative positional relationship with respect to the position of the imaging device. The recognition support method according to any one of Appendices 15 to 22, further comprising this.

[0134] (Appendix 25) At least one of the position and the field of view of the imaging device is further drawn on the coordinates. The recognition support method according to Appendix 24.

[0135] (Appendix 26) The acquisition of the position information is to acquire the position information indicating a position on a plane parallel to the floor surface corresponding to the position where the marker exists, based on the image information of the detected marker, the height information of the marker, and the camera parameters of the imaging device. The recognition support method according to Appendix 24 or Appendix 25.

[0136] (Appendix 27) Causing a display unit to display a virtual image obtained by applying image processing to an original image showing an object to be recognized in a normal state so that the object to be recognized in the original image appears as indicated by the acquired recognition environment information. The recognition support method according to any one of Appendices 15 to 26, further including this.

[0137] (Appendix 28) Causing a display unit to display a plurality of pieces of recognition environment information acquired for the same position within the field of view of the imaging device from each image captured at different times, in association with the capture time of each image. The recognition support method according to any one of Appendices 15 to 27, further including this.

[0138] (Appendix 29) A program storage medium storing a program for causing a computer to execute the recognition support method according to any one of Appendices 15 to 28.

Explanation of Signs

[0139] 5 Display device 7 Camera 10 Information processing device (support device) 11 Image acquisition unit 12, 101 Detection unit 13 Storage unit 14, 102 Environment acquisition unit 15 Display processing unit 16 Data generation unit 17 Position acquisition unit 100 Information processing device

Claims

1. detection means for detecting a marker from a captured image; display control means for causing a second image that changes based on the recognition accuracy of the marker as an image in a target area where the marker exists to be displayed on a display screen together with the captured image; An information processing apparatus comprising the second image including an image of a generated subject.

2. The information processing apparatus according to claim 1, wherein the second image includes numerical values.

3. The information processing apparatus according to claim 1 or 2, wherein the generated subject image is a virtual image generated based on the recognition accuracy.

4. The display control means receives the recognition accuracy at different times based on a plurality of the captured images captured at different times, and controls the display screen based on the received recognition accuracy. The information processing apparatus according to any one of claims 1 to 3.

5. The display control means controls the display screen so as to display different second images for each of the different times based on the recognition accuracy at the different times. The information processing apparatus according to claim 4.

6. A computer, detects a marker from a captured image, causes a second image that changes based on the recognition accuracy of the marker as an image in a target area where the marker exists to be displayed on a display screen together with the captured image, The second image includes an image of a generated subject. An information processing method.

7. The information processing method according to claim 6, wherein the second image includes numerical values.

8. The information processing method according to claim 6 or 7, wherein the generated subject image is a virtual image generated based on the recognition accuracy.

9. In the display, the computer receives the recognition accuracy at different times based on a plurality of the captured images captured at different times, and controls the display screen based on the received recognition accuracy. The information processing method according to any one of claims 6 to 8.

10. In the display, the computer controls the display screen so as to display different second images for each of the different times based on the recognition accuracy at the different times. The information processing method according to claim 9.

11. On a computer, a detection process for detecting a marker from a captured image; a display control process for causing a second image that changes based on the recognition accuracy of the marker as an image in a target area where the marker exists to be displayed on a display screen together with the captured image; A computer program that causes the above to be executed, and the second image includes an image of the generated subject. **Claim 12** The computer program according to claim 11, wherein the second image includes numerical values. **Claim 13** The computer program according to claim 11 or 12, wherein the image of the generated subject is a pseudo-image generated based on the recognition accuracy. **Claim 14** In the display control process, based on a plurality of the captured images captured at different times, receive the recognition accuracy at the different times, and control the display screen based on the received recognition accuracy. The computer program according to any one of claims 11 to 13. **Claim 15** In the display control process, based on the recognition accuracy at the different times, control the display screen to display different second images for each of the different times. The computer program according to claim 14.

Images