Information processing device and program

Abstract

To provide an information processor, a program, and a method for positioning which can suppress the number of markers and can derive a highly accurate result of positioning.SOLUTION: An information processor 10 has a processor 100 which acquires a plurality of image outside shape positions in an image coordinate system from the outside shape of an identifier 6 by image processing on the basis of an image 7 including the identifier 6 provided in a space imaged by the imaging unit 2 and determines the position of the imaging unit 2 on the basis of the image outside shape position and a world known outside shape position showing a known three-dimensional position of the outside shape of the identifier 6 in the world coordinate system.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an information processing device, a program, and a positioning method. [Background technology]

[0002] 2. Description of the Related Art Conventionally, there is known a technique for estimating the position of a moving object or the like by using a captured image.

[0003] Patent Document 1 relates to a method for identifying a self-location of a moving body that travels autonomously in an environment where multiple markers are installed indoors and above. In the method for identifying a self-location of a moving body in Patent Document 1, the positions of multiple markers installed on a ceiling or the like are preregistered. To identify the self-location, marker candidate points are extracted from a ceiling image captured from the moving body, and two-dimensional candidate point coordinates are calculated. In addition, multiple arbitrary virtual points are set within the moving body travel area, and the two-dimensional coordinates of the markers on the image obtained when the moving body is present at the virtual points are derived from the three-dimensional registered positions, and the candidate point coordinates and the two-dimensional coordinates are compared. Then, the virtual point of the moving body that corresponds to the two-dimensional coordinates that are most similar to the candidate point coordinates is estimated as the self-location. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2005-315746 A Summary of the Invention [Problem to be solved by the invention]

[0005] The more markers there are, the higher the accuracy of position measurement. For example, in the technology described in Patent Document 1, the number of markers installed in the movement range is directly related to the position accuracy. However, if you try to realize positioning at multiple locations, it becomes necessary to install many markers in places where movement is expected.

[0006] An object of the present invention is to provide an information processing device, a program, and a positioning method that reduce the number of markers and derive highly accurate positioning results. [Means for solving the problem]

[0007] In order to achieve the above object, an information processing device according to one aspect of the present invention comprises: Exist in a given space Imaged by the imaging unit The predetermined Installed in the space Has an asymmetrical shape Based on a captured image including an identifier, the outer shape of the identifier is determined by image processing. Multiple vertices of From the image coordinate system Each of the vertices and acquiring an image outer shape position of the identifier in the world coordinate system. Supports multiple vertices The position of the imaging unit is determined based on a known global outer shape position indicating a known three-dimensional position. ,child It is characterized by: In order to achieve the above-mentioned object, an information processing device of another aspect of the present invention is characterized in that, based on an captured image including an identifier located within a specified space captured by an imaging unit present in the specified space, multiple image outer shape positions in an image coordinate system are obtained from the outer shape of the identifier by image processing, and a position of the imaging unit is determined based on the image outer shape position of the identifier whose size in the captured image is larger than a specified value and a world known outer shape position indicating a known three-dimensional position of the outer shape of the identifier in a world coordinate system. In order to achieve the above-mentioned object, another aspect of the information processing device of the present invention is characterized in that, based on an captured image including an identifier located within a specified space captured by an imaging unit present in the specified space, multiple image outer shape positions in an image coordinate system are obtained from the outer shape of the identifier by image processing, a primary world imaging position of the imaging unit is obtained based on the image outer shape positions and a world known outer shape position indicating a known three-dimensional position of the outer shape of the identifier in the world coordinate system, an image identifier position indicating the position of the identifier on the image and the world known identifier position are associated based on the primary world imaging position and a world known identifier position indicating the known three-dimensional position of the identifier in the world coordinate system, and a position of the imaging unit is determined based on the image identifier position and the world known identifier position. Effect of the Invention

[0008] According to the present invention, it is possible to provide an information processing device, a program, and a positioning method that reduce the number of markers and derive a positioning result with high position accuracy. [Brief description of the drawings]

[0009] [Figure 1] 1 is a schematic diagram showing an overall configuration of a positioning system including an information processing device according to an embodiment of the present invention; [Diagram 2] 1 is a schematic diagram of a captured image of a ceiling including a marker and an identifier acquired by an information processing device according to the present embodiment. [Diagram 3] 2 is a block diagram showing the hardware configuration of an information processing device and an imaging unit according to the embodiment. FIG. [Figure 4] 1 is a functional block diagram showing a functional configuration of an information processing device according to an embodiment of the present invention. [Diagram 5] 4 is a schematic diagram showing an example of the positions of a marker and an identifier according to the embodiment; FIG. [Figure 6] FIG. 11 is a diagram showing, in table form, an example of the positions of markers and identifiers according to the present embodiment. [Figure 7] 10 is a flowchart illustrating the flow of a positioning process executed by an information processing device according to the present embodiment. [Figure 8] 5A to 5C are schematic diagrams illustrating a process of extracting a high-luminance area from a captured image according to the present embodiment. [Figure 9] 11 is a schematic diagram illustrating a process of identifying an identifier to be used for positioning from an extracted area according to the embodiment; FIG. [Figure 10] 10A and 10B are schematic diagrams illustrating a process of identifying a marker from a captured image according to the present embodiment. [Figure 11] FIG. 13 is a diagram showing a list of image marker positions, image identifier positions, and world known marker positions according to the present embodiment. [Figure 12] FIG. 13 is a diagram showing a table of the relationship between world known identifier positions and calculated identifier positions according to the present embodiment. [Figure 13] 11 is a diagram showing the relationship between a calculated identifier position and an image identifier position on an image coordinate system according to the present embodiment. FIG. [Figure 14] FIG. 13 is a diagram showing an example of a table that associates image identifier positions with world known identifier positions based on image identifier positions and calculated identifier positions according to the present embodiment. [Figure 15] 11 is a schematic diagram showing an example in which the identifier according to the embodiment is a rectangle. FIG. [Figure 16] 11 is a schematic diagram showing an example of assigning identifier positions to vertices of a polygon identifier according to the present embodiment. FIG. [Figure 17] 10 is a flowchart illustrating a process for acquiring an image identifier position when the identifier is a polygon according to the present embodiment. [Figure 18] 1A is a schematic diagram showing an example of a case in which the size of an identifier according to the present embodiment is smaller than a predetermined value, and FIG. 1B is a schematic diagram showing an example of a case in which the boundary portion of an identifier is unclear. [Figure 19] 11 is a diagram showing an example of a table of world known identifier positions when the identifier according to the present embodiment is a rectangle. FIG. [Figure 20]11 is a conceptual diagram showing an example in which an identifier in a captured image according to the present embodiment is identified in association with a world known identifier position, and the identified world known identifier position is used as a marker. FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0011] <Positioning System 1> First, an overview of a positioning system 1 will be described with reference to Fig. 1. Fig. 1 is a schematic diagram showing an overall configuration of a positioning system 1 including an information processing device 10. Fig. 2 is a schematic diagram of a captured image 7 of a ceiling 4 including a marker 5 and an identifier 6 acquired by the information processing device 10.

[0012] As shown in FIG. 1, the positioning system 1 includes an imaging unit 2 mounted on the side of a moving object 3 rather than on the side of a building 9 including the ceiling, a marker 5 installed inside the building 9 through which the moving object 3 moves, and an information processing device 10 that locates the position of the moving object 3 based on an image 7 captured by the imaging unit 2.

[0013] The imaging unit 2 acquires captured images 7 while mounted on the moving body 3. The imaging unit 2 continuously captures images of the ceiling 4 above at a predetermined frame rate, and acquires a plurality of captured images 7 of the ceiling 4 that are continuous in time.

[0014] The imaging unit 2 is also connected to a communication device 28 for communicating with the information processing device 10. The captured image 7 acquired by the imaging unit 2 or various information based on the captured image 7 is transmitted to the information processing device 10 via the communication device 28. In this example, the moving object 3 on which the imaging unit 2 is mounted is a work vehicle such as a forklift.

[0015] The marker 5 is an object including a device or a sign that can obtain three-dimensional position information in the world coordinate system 70 where the marker 5 is located from the database 19 by either transmitting data from the marker 5 itself or by having the imaging unit 2 image the marker 5 and processing the image.

[0016] The marker 5 is a light-emitting device capable of controlling the light emission manner, such as the color and timing of visible light. The marker 5 optically communicates identification information, for example, by changing color or blinking according to a predetermined pattern. The light emitted by the marker 5 may be near-infrared light instead of visible light. In other words, the wavelength of the light emitted by the marker 5 may be in a light waveband that can be captured by a camera.

[0017] A plurality of markers 5 are installed to identify the position and orientation of the imaging unit 2. In the example shown in Fig. 1, the markers 5 are installed at two locations on the ceiling 4. Note that the markers 5 do not necessarily have to be installed on the ceiling 4, but may be installed inside the building 9 in which the mobile object 3 that can be imaged by the imaging unit 2 moves.

[0018] The identifier 6 is, for example, a light source or lighting installed in multiple locations on the ceiling 4. In this specification, the identifier 6 means an area that can be identified by image processing from the captured image 7. For example, brightness or chromaticity can be used to identify the area.

[0019] A window 8 is installed in the building 9, and when the window 8 is captured in the captured image 7, the luminance condition may be met depending on the time of day, but image processing is performed to control not to process the window 8 area as the identifier 6. Note that the identifier 6 is not limited to a self-luminous object. A white reflective object that reflects light, or an area painted green on a brown plane, etc. may be identified as the identifier 6 from the captured image 7 by image processing.

[0020] The information processing device 10 measures the position of the imaging unit 2 based on the known positions of the marker 5 and the identifier 6, and the image marker position 51 of the marker 5 and the image identifier position 61 of the identifier 6 included in the captured image 7. The known positions of the marker 5 and the identifier 6 may be specified in advance from design drawings, for example, or may be specified by another positioning device. In this specification, the position of the marker 5 or the identifier 6 is expressed in a three-dimensional world coordinate system 70.

[0021] The information processing device 10 is connected to a communication device 18 for communicating with the imaging unit 2. The information processing device 10 acquires the captured image 7 from the imaging unit 2 by establishing communication between the communication device 18 on the information processing device 10 side and the communication device 28 on the imaging unit 2 side.

[0022] <Hardware configuration> FIG. 3 is a block diagram showing the hardware configuration of the information processing device 10 and the imaging unit 2. As shown in FIG.

[0023] In the example of FIG. 3, the information processing device 10 includes a processor 11, a read only memory (ROM) 12, a random access memory (RAM) 13, an input / output unit 14, a communication unit 15, and a storage unit 16.

[0024] The processor 11 performs various calculations and processes. The processor 11 is, for example, a central processing unit (CPU), a micro processing unit (MPU), a system on a chip (SoC), a digital signal processor (DSP), a graphics processing unit (GPU), an application specific integrated circuit (ASIC), a programmable logic device (PLD), or a field-programmable gate array (FPGA). Alternatively, the processor 11 is a combination of a plurality of these. The processor 11 may also be a combination of these with a hardware accelerator or the like.

[0025] The processor 11, the ROM 12, and the RAM 13 are connected to each other via a bus. The processor 11 executes various processes according to a program recorded in the ROM 12 or a program loaded into the RAM 13. A part or the whole of the program may be incorporated in the circuitry of the processor 11.

[0026] The input / output unit 14 is composed of a keyboard, various buttons, a microphone, etc., and inputs various information in response to user's instruction operations. The input / output unit 14 is also composed of a display, a speaker, etc., and outputs the captured image 7 and sound. An information terminal with a touch panel is included. The communication unit 15 is a network interface for communicating with the imaging unit 2 etc. via the communication device 28. The storage unit 16 is an area for storing various information such as the captured image 7 and known information.

[0027] Next, a description will be given of an example of a hardware configuration of the imaging unit 2. The imaging unit 2 includes an optical lens unit 21 and an image sensor 22.

[0028] The optical lens unit 21 is composed of lenses that collect light in order to capture an image of a subject, such as a focus lens or a zoom lens. The focus lens is a lens that forms an image of a subject on the light receiving surface of the image sensor 22. The zoom lens is a lens that can freely change the focal length within a certain range. The optical lens unit 21 is also provided with peripheral circuits that adjust setting parameters such as focus, exposure, and white balance as necessary.

[0029] The image sensor 22 is composed of a photoelectric conversion element, an AFE (Analog Front End), and the like. The photoelectric conversion element is composed of, for example, a CMOS (Complementary Metal Oxide Semiconductor) type photoelectric conversion element. An object image is incident on the photoelectric conversion element from the optical lens unit 21. The photoelectric conversion element photoelectrically converts (captures) the object image, accumulates the image signal for a certain period of time, and sequentially supplies the accumulated image signal as an analog signal to the AFE. The AFE performs various signal processing such as A / D (Analog / Digital) conversion processing on this analog image signal. A digital signal is generated by this various signal processing, and the captured image 7 is output as an output signal of the imaging unit 2.

[0030] <Functional configuration of information processing device> Fig. 4 is a functional block diagram showing the functional configuration of the information processing device 10. Fig. 4 shows the functional configuration as described above, and also shows the relationship of the processing flow.

[0031] The information processing device 10 performs various controls of the information processing device 10 using a processing unit 100 that is realized by the processor 11 executing arithmetic processing based on a predetermined program.

[0032] The processing unit 100 has an image processing unit 101, a marker identification unit 102, a known information acquisition unit 103, a primary positioning unit 104, a calculation unit 105, an identifier identification unit 106, a final positioning unit 107, and an output unit 108, each of which has its own function.

[0033] The image processing unit 101 acquires the captured image 7 and performs pre-processing such as distortion processing. The marker identification unit 102 executes processing to identify the marker 5 from the captured image 7, thereby acquiring the image marker position 51.

[0034] The known information acquisition unit 103 executes a process of acquiring pre-set known information such as the imaging unit internal parameters 25, the imaging unit installation parameters 26, the world known marker positions 52, and the world known identifier positions 62 from the database 19. The database 19 may be constructed in the storage unit 16 of the information processing device 10, or may be constructed in an external server of the information processing device 10.

[0035] The primary positioning unit 104 executes a process of primarily locating the position of the imaging unit 2 based on the image marker positions 51 acquired from the captured image 7 and the world known marker positions 52 acquired from the database 19, and acquiring the primary world imaging position 23. This primary world imaging position 23 includes the position of the imaging unit 2 and the direction in which the imaging unit 2 is facing.

[0036] The calculation unit 105 executes a projection calculation process that calculates, by projecting, from the world known identifier position 62, a position on the captured image 7 obtained when the imaging unit 2 captures an image. The identifier identification unit 106 executes a process of associating and identifying the image identifier position 61 of the identifier 6 acquired from the captured image 7 by image processing with the world known identifier position 62. In this embodiment, this association process is executed, and the position of the moving object 3 is calculated as a PnP problem including the image identifier position 61 and the world known identifier position 62 in addition to the image marker position 51 and the world known marker position 52. As a result, even if only a small number of markers 5, for example two, are installed, the position of the moving object 3 can be calculated with high accuracy by utilizing the identifier 6 of existing lighting, etc.

[0037] The confirmed positioning unit 107 executes a process of acquiring a confirmed world imaging unit position 24, which is the final position of the imaging unit 2, based on the primary position of the imaging unit 2, the world known marker position 52, the world known identifier position 62, etc. This confirmed world imaging unit position 24 includes the position of the imaging unit 2 and the direction in which the imaging unit 2 is facing. The output unit 108 executes a process of outputting the positioning result.

[0038] In this example, the information processing device 10 is located at a location away from the imaging unit 2, but is not limited to this configuration. For example, the information processing device 10 may be configured to be mounted on the moving object 3 or the imaging unit 2.

[0039] Fig. 5 is a schematic diagram showing the planar installation of the markers 5 and the identifiers 6 according to this embodiment. Fig. 5 shows the positional relationship between the markers 5 and the identifiers 6 installed on the ceiling 4 in a plan view. In this example, the two markers 5 are given identification numbers 5a and 5b and are each treated as individual markers. In this embodiment, the identifiers 6, which are 16 lights, are similarly given identification numbers 6a to 6p and are each treated as individual markers.

[0040] The identifier 6 in this embodiment is a rectangular ceiling light such as a square or a rectangle. For each identifier 6, a world known identifier position 62 indicating the center position of the identifier 6 and a plurality of world known outer shape positions 65 indicating the outer shape positions of the identifier 6 are registered in the database 19 as known information. The world known identifier position 62 is set to a position such as the center of gravity position of the identifier 6 or an intersection point connecting the world known outer shape positions 65. The world known outer shape position 65 is a position indicating an outer shape feature arranged along the contour of the identifier 6. The world known outer shape position 65 in this embodiment is set to a position corresponding to a vertex of a polygon. As described above, the positions of the world known identifier position 62 and the world known outer shape position 65 can be specified from the design drawings of the building 9, etc.

[0041] Here, an example of the world known identifier position 62 will be described with reference to FIG. 6. FIG. 6 is a diagram showing an example of the positions of the marker 5 and the identifier 6 in the world coordinate system 70 in a table format. FIG. 6 shows the world coordinates of the center positions as three-dimensional positions for each of the identification numbers 5a and 5b of the marker 5 and the identification numbers 6a to 6p of the identifier 6. Here, for convenience of explanation, one of the corners of the ceiling 4 is set as the origin, the x-axis and the y-axis are set in a planar view, the vertical direction is set as the z-axis, and the floor surface is set as the origin of the Z-axis. With the three-dimensional positions of the marker 5 and the identifier 6 registered in advance in the database 19 as known information, a process of identifying the position of the imaging unit 2 is executed based on the captured image 7 acquired by capturing an image of the ceiling 4 by the imaging unit 2.

[0042] <Positioning processing> Next, a description will be given of a positioning process using the captured image 7. Fig. 7 is a flowchart illustrating the flow of the positioning process executed by the information processing device 10 according to this embodiment.

[0043] When the positioning process starts, the processing unit 100 acquires the captured image 7 captured by the imaging unit 2 and performs distortion correction processing (step S101). The distortion correction processing corrects, for example, barrel aberration of the captured image 7 captured by a wide-angle lens.

[0044] After the process of step S101, the processing unit 100 performs a process of identifying the position of the identifier 6 (step S102).

[0045] An example of the process of identifying the identifier position will be described with reference to Figs. 8 and 9. Fig. 8 is a schematic diagram for explaining the process of extracting a high brightness area from a captured image 7. As shown in Fig. 8, in order to identify the identifier 6 from the captured image 7, the image processing unit 101 of the processing unit 100 performs preprocessing by binarizing the captured image 7 using a predetermined threshold value, making only the high brightness areas white and the rest black. As a result, a high brightness area such as a saturated area is extracted. However, in this state, the position of the window 8 may also be extracted as a high brightness area depending on the time of day when there is sunlight, such as during the daytime.

[0046] FIG. 9 is a schematic diagram for explaining a process of identifying an identifier 6 to be used for positioning from the region extracted in FIG. 8. As shown in FIG. 9, the image processing unit 101 executes a filtering process based on the actual shape and area of ​​the extracted region in order to eliminate a region that does not contribute to positioning due to a window 8, and identifies only those that are likely to be the marker 5 and the identifier 6. Then, the region center position for each of the multiple identifiers 6 is obtained by image processing, and the region center position is stored as the image identifier position 61. Then, the identifier identification unit 106 creates a list of the image identifier positions 61 of the identifiers 6. Here, the center position is the center of gravity of the region recognized as the identifier 6 in the world coordinate system 70, the average of each of the x, y, and z coordinates, the intermediate value between the maximum coordinate and the minimum coordinate, etc. are adopted. It is not limited to these, and the center position represents the position of the identifier 6 as one position.

[0047] In the captured image 7, the identifier 6 forms, for example, a high brightness area. When the high brightness area is a polygon as shown in FIG. 9, a plurality of image outer shape positions 64 indicating the positions of the outer shape of the identifier 6 are acquired for each of the identifiers 6. The image outer shape positions 64 are positions indicating the characteristics of the outer shape arranged along the contour of the identifier 6. For example, the positions of each vertex are acquired as the image outer shape positions 64 through image processing. As the center position of the identifier 6, for example, the position of the center of gravity of the identifier 6 or the position of the intersection connecting the image outer shape positions 64 is acquired. The center position of the identifier 6 is registered as the image identifier position 61.

[0048] The image identifier position 61 on the captured image 7 is specified by setting two-dimensional coordinates on the captured image 7. In this specification, the two-dimensional coordinate system on the captured image 7 is referred to as an image coordinate system 71.

[0049] FIG. 11 shows an example of a list of image identifier positions 61. Identifiers 6 that are given image identifier positions 61 as the center points of the areas of the identifiers 6 in the captured image 7 are assigned primary identification numbers such as L1 and L2. Here, an example of setting image identifier positions 61 is shown. At this stage, the positions of multiple identifiers 6 are merely identified from the captured image 7, and it is not possible to determine whether the multiple identifiers 6 correspond to any of the identifiers 6a to 6p shown in FIG. 6, and they cannot be uniquely identified. For this reason, as shown in FIG. 11, at this point, the identifiers 6 are not linked to the world known identifier positions 62.

[0050] Next, the marker identification unit 102 performs a marker identification process (step S103). Fig. 10 is a schematic diagram illustrating a process for identifying the marker 5 from the captured image 7. In the marker identification process, the marker identification unit 102 identifies the position of the marker 5 from the captured image 7 by image processing.

[0051] 11 shows an example of a list of image marker positions 51 and world known marker positions 52. A primary identification number such as M1 or M2 is assigned to a marker 5 given an image marker position 51. Since the markers 5 are identified, for example, the image marker positions 51 of M1 and M2 are linked to the markers 5a and 5b shown in FIG.

[0052] An example of the marker identification process will now be described. The marker identification unit 102 acquires a light emission pattern based on the successive captured images 7, and identifies the position of the marker 5 in the captured images 7 by comparing the light emission pattern with a preset light emission pattern.

[0053] Next, the primary positioning unit 104 performs a positioning possibility determination process (step S104). As the positioning possibility determination process, for example, the primary positioning unit 104 determines whether two or more markers 5 are present in the captured image 7 and whether positioning is possible.

[0054] If positioning is possible using the marker 5 (step S104: Yes), the primary positioning unit 104 performs a primary positioning process (step S105).

[0055] An example of the primary positioning process will be described below. As described above, since the markers 5a and 5b can be individually identified from the captured image 7, the primary positioning unit 104 acquires the world known marker positions 52 of the markers 5a and 5b based on the correspondence relationship in FIG.

[0056] As the primary positioning process, for example, the primary positioning unit 104 performs primary positioning of the imaging unit 2 through PnP positioning process based on the image marker position 51 and the world known marker position 52.

[0057] As is conventionally known, when the positions of a certain point group and a position in the captured image 7 are given, the position and attitude, etc. of the imaging unit 2 can be obtained by PnP positioning processing. When positioning is performed based on six points, the position and imaging direction (in-plane orientation, elevation angle) of the imaging unit 2 can be obtained. Even when the number of points is less than six, for example when the imaging unit 2 is located on a plane parallel to the ceiling 4 on which the marker 5 and the identifier 6 are provided, the position and elevation angle of the imaging unit 2 can be obtained by positioning using only two points, although not with high accuracy.

[0058] Since the moving object 3 moves on the floor surface parallel to the ceiling 4, the primary positioning unit 104 determines the primary world imaging position 23, which is the primary position and elevation angle of the imaging unit 2, through the P2P method. The primary positioning unit 104 locates the primary world imaging position 23, which is the primary position of the imaging unit 2, through the P2P method using the image marker positions 51 represented by identification numbers M1 and M2 and the world known marker positions 52 represented by identification numbers 5a and 5b shown in Fig. 11.

[0059] If positioning is not possible using the marker 5 (step S104: No), the primary positioning unit 104 performs a position estimation process (step S106). As the position estimation process, for example, the primary positioning unit 104 estimates the position of the imaging unit 2 based on the previously confirmed world imaging unit position 24 and the movement vector history, and sets the result as a primary positioning result.

[0060] Next, the calculation unit 105 performs an identifier recognition process (step S107). As the identifier recognition process, for example, the calculation unit 105 determines whether the identifier 6 is present in the captured image .

[0061] If the identifier 6 does not exist in the captured image 7 (step S107: No), the primary positioning unit 104 performs a primary positioning result application process (step S111). In the primary positioning result application process, for example, the primary positioning unit 104 applies the primary positioning result as a confirmed positioning result.

[0062] If the identifier 6 is present in the captured image 7 (step S107: Yes), the calculation unit 105 performs a calculated identifier position calculation process (step S108). As the calculated identifier position calculation process, for example, the calculation unit 105 calculates a position of the world known identifier position 62 captured in the captured image 7, in other words, a calculated identifier position 63 indicating a projection position, when the world known identifier position 62 is captured from the primary world imaging position 23 obtained based on the primary positioning result.

[0063] This calculation is a known process that reversely calculates how a specified arbitrary three-dimensional point is drawn on an image from the position and orientation of the imaging unit 2. The calculation itself is a simple matrix calculation. Therefore, the projection is in an infinite field of view, and the back side of the world imaging unit position is also projected onto a plane. If an attempt is made to execute the PnP positioning process as it is, the calculation will not be valid or a large error will occur. Therefore, the calculation is performed after selecting only the identifier 6 at the world known identifier position 62 that is approximately within the imaging angle of view when imaged from the primary world imaging position 23 obtained by the primary positioning.

[0064] 12 shows a world known identifier position 62, the position of which is known, and a calculated identifier position 63, which is the drawing position of the world known identifier position 62 on the captured image 7. The identification number of the calculated identifier position 63 calculated for the identification number 6a of the world known identifier position 62 is written as 6a-C.

[0065] The final positioning unit 107 performs a linking process (step S109). As the linking process, for example, the final positioning unit 107 associates the image identifier position 61 with the calculated identifier position 63 based on the positional relationship on the same captured image, and links the identifier 6 to the world known identifier position 62 from which the calculated identifier position 63 is calculated.

[0066] FIG. 13 shows the relationship between the calculated identifier position 63 and the image identifier position 61. As shown in FIG. 13a, the calculated identifier position 63 on the captured image 7 of the world known identifier position 62 including the world known marker position 52 is plotted in the image coordinate system 71. On the other hand, as shown in FIG. 13b, the image identifier position 61 of the identifier 6 including the image marker position 51 is plotted in the image coordinate system 71. FIG. 13c is an enlarged view of the upper right of the image coordinate system 71 when these are superimposed. The calculated identifier position 63 is similar to the image identifier position 61, but there is a slight deviation due to the positioning error. Points that are the nearest or surrounding points and have a positional relationship that does not destroy the mutual positional relationship are linked together. In FIG. 13C, the calculated identifier position 63 of the identification number 6p-C and the image identifier position 61 of the identification number L13 are plotted nearby, and the calculated identifier position 63 of the identification number 6o-C and the image identifier position 61 of the identification number L14 are plotted nearby. These are treated as corresponding coordinate points and are then linked or associated with each other.

[0067] 14 shows a list of calculated identifier positions 63 and image identifier positions 61 linked in a similar manner by the final positioning unit 107. The world known identifier position 62 on which the calculated identifier position 63 was calculated is also displayed. The image identifier position 61 is linked to the world known identifier position 62. At this point, the identifier 6 that was imaged by the imaging unit 2 but not identified is identified as an identifier 6 with a known position. This identified identifier 6 is similar to the marker 5 because it has been identified and its position is known.

[0068] The primary positioning unit 104 determines the primary world imaging position 23 of the imaging unit 2 based on the two markers 5. In contrast, by linking the identifiers 6 to the world known identifier positions 62 as shown in Fig. 14, the number of markers 5 increases from two to 18, including the 16 identified identifiers 6 from 6a to 6p.

[0069] Furthermore, the final positioning unit 107 performs a PnP positioning process (step S110). In the PnP positioning process, for example, the final positioning unit 107 performs the PnP positioning process based on the world known marker position 52, the world known identifier position 62, the image marker position 51, and the image identifier position 61.

[0070] What was a P2P positioning calculation in the primary positioning can be made into a P18P problem in which positioning is performed from the imaging results of 18 3D known points, resulting in high accuracy.

[0071] Then, the output unit 108 outputs the result of the PnP positioning process as a confirmed positioning result.

[0072] The processing unit 100 performs a positioning end determination process (step S112).

[0073] As the positioning end determination process, if the positioning is to be ended (step S112: Yes), the processing unit 100 ends the positioning process. If the positioning is not to be ended (step S112: No), the processing unit 100 returns the positioning process to A in the flowchart.

[0074] In this embodiment, P2P positioning processing is adopted for primary positioning, and two markers 5 are set as the minimum number for primary positioning. The minimum number of markers 5 may be six, i.e., P6P. This is because P6P makes it possible to derive the position and imaging direction of the imaging unit 2. Also, this embodiment is not limited to the above two points or six points, and is effective for all PnP positioning processing.

[0075] <Known external locations in the world> The entire flow of positioning has been described above using as an example the world known identifier position 62, which is the center position in the identifier 6. Below, details of the case where the world known outer shape position 65 is used will be described with reference to Figs. 15 to 19.

[0076] FIG. 15 is a schematic diagram showing an example in which the identifier 6 is a rectangle and the world known outer shape position 65 is used. Each identifier 6 is assigned an identification number 100 to 130. For example, these identification numbers are assigned to the identifiers 6 with reference numbers 6a and the like in FIG. 5. In the figure, arrows are used to indicate the identification numbers. The same applies below. FIG. 16 is a schematic diagram showing an example in which identifier positions are assigned to vertices of the identifier 6. As shown in FIG. 15, a rectangular light in a planar view is also used as the identifier 6. As shown in FIG. 16(a), a process of adding vertex partial codes 111 to 114 to the world known identifier position 62 is executed. In FIG. 16(a), arrows are used for the codes. The same applies below. In the case of a rectangle identifier 6, three-dimensional positions are thus obtained for a total of five positions, including the world known identifier position 62, which is the center position of the identifier 6, and the world known outer shape positions 65 at the four corners.

[0077] World known identifier positions 62 can also be assigned to the vertices of an asymmetric polygonal identifier 6, such as a trapezoid that is not a rectangle as shown in FIG. 16(b). Identification numbers 200 to 204 are assigned to each vertex. Unlike a square identifier 6, an identifier 6 that forms such an asymmetric polygon can be identified by referencing the database 19 due to the uniqueness of its shape. In this case, the identifier 6 can be used for primary positioning in the same way as the marker 5. And positioning is possible even when the marker 5 is not present.

[0078] In the example of Fig. 16, each of the shapes is a quadrangle, but the shape is not limited to this. For example, even if the shape is a triangle or a pentagon or more, each vertex can be added as a known global contour position 65, and the image contour positions 64 of the vertices can be obtained to perform secondary positioning.

[0079] <Worldwide known identifier location extraction process> 17 is a flowchart for explaining the flow of processing when the identifier 6 is a polygon. By the processing executed by the processing unit 100, the world known identifier position 62 and the world known outline position 65 are extracted.

[0080] When the creation of the table of image identifier positions 61 starts, first, the image processing unit 101 performs an identifier recognition process (step S201). In the identifier recognition process, for example, the image processing unit 101 extracts an area where the brightness is saturated in the captured image 7, and assigns a primary identification number to the extracted area.

[0081] Furthermore, the image processing unit 101 performs an identifier selection process (step S202). As the identifier selection process, for example, the image processing unit 101 performs selection based on the shape of the obtained region. For example, the image processing unit 101 extracts ellipses and rectangles, and excludes those that overlap the edge of the image.

[0082] Next, the image processing unit 101 performs table creation processing (step S203). In the table creation processing, for example, the image processing unit 101 registers the extracted candidate regions in a table.

[0083] The identifier identifying unit 106 performs a selection process (step S204). In the extraction process, for example, the identifier identifying unit 106 selects one of the regions.

[0084] The identifier identification unit 106 performs a size determination process. As the size determination process, for example, the identifier identification unit 106 determines whether the size of the identifier 6 in the captured image 7 is equal to or larger than a predetermined size. The predetermined size is, for example, a size of 7×7 pixels in the captured image 7.

[0085] The size determination process will be described with reference to Fig. 18. In cases where the image is far away and small, using the points on the image at the four corners may actually result in errors. Therefore, in the size determination process, to avoid a determination that is unlikely to result in errors, it is determined whether the area that is a candidate for identifier 6 is a predetermined size.

[0086] If the extracted area is equal to or larger than the specified size (step S205: Yes), the identifier identification unit 106 performs a vertex registration process (step S206). As the vertex registration process, for example, the identifier identification unit 106 registers four vertices of the rectangular identifier 6 as the image outer shape position 64. As shown in FIG. 18(a), when there is a candidate for obtaining the image identifier position 61 as one of the identifiers 6, only a partial area 612 located at the center of the area 611 may be used. Also, as shown in FIG. 18(b), although it appears to be a polygon, the boundary may be blurred and the vertices may not be clear. In such a case, the partial area 612 located at the center of the area 611 may be used.

[0087] Next, the identifier identifying unit 106 performs a center registration process (step S207). In the center registration process, for example, the identifier identifying unit 106 registers the center of the extracted area as the image identifier position 61 regardless of the size of the extracted area.

[0088] The processing unit 100 performs an end determination process (step S208).

[0089] If the process does not end (step S208: No), the processing unit 100 returns the positioning process to step S204. If the process does end (step S208: Yes), the processing unit 100 ends the creation of the table of image identifier positions 61.

[0090] 19 shows an example in which a world known outline position 65 is identified and registered as a world known identifier position 62 at the position of a vertex of such a polygon identifier 6. A tens digit or higher is used for the center position, and a single digit is used for the world known outline position 65, which is the vertex.

[0091] In this embodiment, a polygonal identifier 6 installed in the environment can be used for positioning. When an indoor light is normally captured as the identifier 6, the brightness is captured as a saturated area, so that the image signal is very stable as the first identifier area candidate to be extracted by simply binarizing it at a saturation value. In addition, lights are generally installed in places with good visibility in the environment, and many of them have a simple rectangular shape, which is suitable as the identifier 6 in this embodiment. The identifier 6 is not limited to lights, and other methods may be used as long as they are easy to detect stably and simply in the usage scene. For example, a high-saturation object with good visibility in a low-saturation environment can be binarized at a specific high saturation threshold, so other methods are possible.

[0092] An identifier 6 having a polygonal shape is used, and a world known outer shape position 65 is used for PnP positioning as one form of a world known identifier position 62. It is possible to increase n in the PnP positioning process without increasing the illumination, and it is possible to improve the accuracy of the PnP positioning process.

[0093] If the size of the identifier 6 in the captured image 7 is smaller than a predetermined size, the processing unit 100 included in the information processing device 10 acquires the center position of the identifier 6, and acquires the image identifier position 61 based on the center position.

[0094] The globally known identifier position 62 used in the PnP positioning process is optimized to obtain higher positioning accuracy.

[0095] (Modification of marker) In the above-mentioned positioning process, the marker 5 having an identification function is used as one of the methods for primary positioning. Other methods may be used for primary positioning. Examples are listed below.

[0096] This is not possible in the initial state, but if PnP positioning processing has been performed by the positioning confirmation unit 107 up until just before, the current position can be estimated from the past position and the last vector using the difference, and this can be used as the primary world imaging position 23.

[0097] The two-dimensional code is captured by the imaging unit 2, and the position of the imaging unit 2 can be primarily set based on the code and the imaging angle of the imaging unit 2. When determining position by recognizing a two-dimensional code or the like, it is preferable to display the two-dimensional code in a relatively large area on the imaging screen.

[0098] For example, an infrared sensor is prepared as the specific point, and the position of the imaging unit 2 can be primarily set based on the detection of the passage of the specific point. When the imaging unit 2 is installed on the moving body 3, if it is guaranteed that the starting point of the moving body 3 is a specific point, the position of the imaging unit 2 can be primarily set based on the position of the specific point. Information on this specific point may be stored in the database 19 as the imaging unit installation parameters 26 shown in FIG. 1. For example, if the starting point is a charging station, the position of the imaging unit 2 can be primarily set if the identification number of the charging station is known. In addition, the elevation angle, focal length, presence or absence of an autofocus function, etc. of the imaging unit 2 may be stored in the database 19 as the imaging unit internal parameters 25 shown in FIG. 1. In addition, a combination of these various methods may be used. However, a method using a marker 5 having an identification function, particularly a method using a marker 5 identified by a color change signal of a light-emitting body, is suitable for performing primary positioning over a wide area with relatively high accuracy.

[0099] As explained above, the marker 5 is not limited to the marker 5 of the above embodiment that performs optical communication by changing the light emission pattern, and other types of markers 5 can also be used, as shown in the modified example.

[0100] Next, an example of processing after the world known identifier position 62 is recognized on the captured image 7 will be described.

[0101] 20 shows a more specific positioning process using a ceiling 4 on which a marker 5 and an identifier 6 are installed. The recognized identifier 6 is identified in the same way as the marker 5. In the following description, the identified world known identifier position 62 will be used by the primary positioning unit 104 for primary positioning in the same way as the marker 5.

[0102] For ease of explanation, it is assumed in Fig. 20 that the markers 5 and identifiers 6 are installed in an orderly fashion, and the imaging unit 2 is installed perpendicular to these. In the figure, the rectangular area indicates the imaging range of the captured image 7. Fig. 20a shows the initial state. The marker 5 is imaged and the image marker position 51 is recognized. Then, the image identifier position 61 is obtained. The marker 5 is colored black in Fig. 20a to indicate that it has been identified.

[0103] Based on the image marker position 51 and the world known marker position 52, the primary world imaging position 23 of the imaging unit 2 is calculated. Then, a calculated identifier position 63 is calculated when an identifier 6 located at a world known identifier position 62 is imaged from the primary world imaging position 23. That is, a projection calculation process is performed. Then, for example, by matching nearby points, the identifier 6 that produces the image identifier position 61 is linked to the world known identifier position 62 from which it was calculated. As a result, the identifier 6 shown in the captured image 7 is recognized as an identified identifier 6 whose position is known. Here, the identifier 6 becomes an identified index similar to the marker 5. To show this situation, the identifier 6 that was represented by a white circle in FIG. 20a is painted black in FIG. 20b.

[0104] From this, when the moving object 3 moves continuously, the captured image 7 captured by the imaging unit 2 changes as shown in FIG. 20c. Here, the association between the image identifier position 61 in the captured image 7 and the world known identifier position 62 is maintained by image processing. In FIG. 20C, the image marker position 51 and the image identifier position 61 of the marker 5 and the identifier 6 are maintained by blackening, which means identification. The four blackened points in the captured image 7 in FIG. 20c are identified, so they can be used for the primary positioning of the imaging unit 2, similar to the image marker position 51 in FIG. 20a. Then, through comparison with the calculated identifier position 63, the image identifier position 61 in the captured image 7 in FIG. 20C is newly associated with the world known identifier position 62, and becomes an identified index similar to the marker 5. To show this situation, the identifier 6 indicated by a white circle in FIG. 20C is blackened in FIG. 20d.

[0105] This process is repeated. Fig. 20e shows the captured image 7 when the moving object 3 has moved further following Fig. 20c. At this point in time, the image marker position 51 caused by the marker 5 is not present in the captured image 7. The primary positioning unit 104 performs primary positioning of the position of the imaging unit 2 based on the identified identifier 6, and further, the final positioning unit 107 performs final positioning of the position of the imaging unit 2.

[0106] This allows the markers 5 to be placed only near the initial operation position, and thereafter, by continuously locating and identifying the identifiers 6, it becomes possible to place the markers 5 only in the initial positions even in a wide area. That is, the process transition using the markers 5 makes it possible to derive the confirmed world imaging unit position 24 of the imaging unit 2 even if the markers 5 are not included in the captured image 7, as long as the identifier 6 is included.

[0107] <Effects of the embodiment>

[0108] The information processing device 10 includes a processing unit 100 that acquires multiple image outline positions 64 in an image coordinate system 71 from the outline of the identifier 6 by image processing based on an captured image 7 including an identifier 6 provided in a space captured by an imaging unit 2, and determines the position of the imaging unit 2 based on the image outline positions 64 and a world known outline position 65 indicating a known three-dimensional position of the outline of the identifier in a world coordinate system 70.

[0109] This makes it possible to realize an information processing device 10 that derives highly accurate positioning results using only existing identifiers 6 or lighting. There is no difficulty in pursuing high accuracy by using waves. In addition, there is no problem with the difficulty in guaranteeing definite positioning as in Visual-SLAM and LiDAR-SLAM, and no problem of being trapped in a loop. A practical positioning system can be constructed.

[0110] Thereby, the processing unit 100 of the information processing device 10 obtains, as the image contour position 64, at least one of the vertex positions of the contour and a center position derived from the vertex positions of the contour.

[0111] This allows polygonal lighting to be used for positioning, expanding the degree of freedom in positioning and improving the accuracy of positioning. The number of vertices is the minimum of three points of a triangle, and is two or more, which is the minimum number required for PnP positioning processing. In addition, obtaining the center position provides at least one position that can be used as a reference for positioning.

[0112] The processing unit 100 of the information processing device 10 determines the position of the imaging unit 2 based on the image contour position 64 of the identifier 6 having an asymmetric contour.

[0113] As a result, the asymmetric outline is matched with the database 19, the image outline position 64 is associated with the world known outline position 65, and the orientation can be identified, so that PnP positioning processing can be performed even without the marker 5.

[0114] The processing unit 100 of the information processing device 10 determines the position of the imaging unit based on the image outer shape position 64 of the identifier 6 whose size on the captured image 7 is equal to or larger than a predetermined size. .

[0115] This allows the resolution of the positioning coordinates on the captured image 7 to be at least a certain level, improving the reliability of the positioning accuracy.

[0116] When the size of the identifier 6 in the captured image 7 is smaller than a predetermined size, the processing unit 100 of the information processing device acquires the center position of the identifier 6, and determines the position of the imaging unit based on the center position.

[0117] This allows the resolution of the coordinates for positioning on the captured image 7 to be at least a certain level, improving the reliability of the positioning accuracy. By using the center point, at least one position for positioning is provided, improving the positioning accuracy.

[0118] The processing unit 100 of the information processing device 10 acquires the primary world imaging position 23 of the imaging unit 2 based on the image outer shape position 64 and the world known outer shape position 65 of the identifier 6, and corresponds the image identifier position 61 indicating the position of the identifier 6 on the captured image 7 to the world known identifier position 62 based on the primary world imaging position 23 and the world known identifier position 62 indicating the known three-dimensional position of the identifier 6 in the world coordinate system 70, and determines the position of the imaging unit 2 based on the image identifier position 61 and the world known identifier position 62.

[0119] As a result, in the secondary positioning, the number n in the PnP positioning process, that is, the number of reference positions for positioning, increases, thereby improving the positioning accuracy.

[0120] In addition, the program of this embodiment executes a function of acquiring multiple image outline positions 64 in an image coordinate system 71 from the outline of the identifier 6 by image processing based on an captured image 7 including an identifier 6 provided in a space captured by the imaging unit 2, and a function of determining the position of the imaging unit 2 based on the image outline positions 64 and a world known outline position 65 indicating a known three-dimensional position of the outline of the identifier 6 in the world coordinate system 70.

[0121] This provides a program that derives highly accurate positioning results using only the existing identifier 6 or lighting.

[0122] In addition, the positioning method of this embodiment includes a step of acquiring multiple image outline positions 64 in an image coordinate system 71 from the outline of the identifier 6 by image processing based on an captured image 7 including an identifier 6 provided in a space captured by an imaging unit 2, and a step of determining the position of the imaging unit 2 based on the image outline positions 64 and a world known outline position 65 indicating a known three-dimensional position of the outline of the identifier 6 in a world coordinate system 70.

[0123] A method is provided for deriving highly accurate positioning results using only existing identifiers 6 or lighting.

[0124] The above describes the positioning process using the identifier 6. Note that the present invention is not limited to the above-described embodiment and modified examples, and the present invention includes modifications, improvements, etc. within the scope of achieving the object of the present invention.

[0125] In the above embodiment, the information processing device 10 to which the present invention is applied has been described using a forklift as an example of the moving object 3, but is not particularly limited thereto. For example, the present invention can be applied to electronic devices in general that have an image processing function. Specifically, the present invention can be applied to, for example, notebook personal computers, portable navigation devices, mobile phones, smartphones, portable game consoles, and the like.

[0126] The above-mentioned series of processes can be executed by hardware or software. In other words, the functional configuration of Fig. 4 is merely an example and is not particularly limited. That is, it is sufficient that the information processing device 10 has a function capable of executing the above-mentioned series of processes as a whole, and the type of functional block used to realize this function is not particularly limited to the example of Fig. 4.

[0127] Moreover, one functional block may be configured as a single piece of hardware, a single piece of software, or a combination of both. The functional configuration in this embodiment is realized by the processor 11 that executes arithmetic processing, and the processor 11 that can be used in this embodiment includes those configured as single pieces of various processing devices such as a single processor, a multiprocessor, and a multicore processor, as well as those in which these various processing devices are combined with processing circuits such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field-Programmable Gate Array).

[0128] When a series of processes is executed by software, the programs constituting the software are installed into a computer or the like from a network or a recording medium. The computer may be a computer incorporated in dedicated hardware. The computer may also be a computer capable of executing various functions by installing various programs, such as a general-purpose personal computer.

[0129] A recording medium including such a program is not only distributed separately from the device main body in order to provide the program to the user, but is also configured as a recording medium provided to the user in a state in which it is pre-installed in the device main body. The removable medium is, for example, a magnetic disk (including a floppy disk), an optical disk, or a magneto-optical disk. The optical disk is, for example, a CD-ROM (Compact Disk-Read Only Memory), a DVD (Digital Versatile Disk), a Blu-ray (registered trademark) Disc, etc. The magneto-optical disk is, for example, an MD (Mini-Disk), etc. Also, the recording medium provided to the user in a state in which it is pre-installed in the device main body is, for example, a ROM 12 in which the program is recorded, a hard disk (not shown), etc.

[0130] In this specification, the steps describing the program recorded on the recording medium include not only processes performed in chronological order, but also processes that are not necessarily performed in chronological order but are performed in parallel or individually. In addition, in this specification, the term "system" refers to an overall device composed of multiple devices, multiple means, etc.

[0131] Although several embodiments of the present invention have been described above, these embodiments are merely illustrative and do not limit the technical scope of the present invention. The present invention can take various other embodiments, and further, various modifications such as omissions and substitutions can be made without departing from the gist of the present invention. These embodiments and their modifications are included in the scope and gist of the invention described in this specification, etc., and are included in the scope of the invention described in the claims and their equivalents.

[0132] The invention as described in the claims of the present application as originally filed is set forth below.

[0133] [Appendix 1] Based on a captured image 7 including an identifier 6 provided in a space captured by the imaging unit 2, a plurality of image outer shape positions 64 are acquired in an image coordinate system 71 from the outer shape of the identifier 6 by image processing; determining a position of the imaging unit 2 based on the image contour position 64 and a world known contour position 65 indicating a known three-dimensional position of the contour of the identifier in a world coordinate system 70; An information processing device 10 including a processing unit 100.

[0134] [Appendix 2] The processing unit 100 determines the position of the imaging unit 2 based on the image contour position 64 of the identifier 6 having an asymmetric contour. 13. An information processing device 10 as described in claim 1.

[0135] [Appendix 3] The processing unit 100 includes: At least one of the vertex positions of the contour and the center position derived from the vertex positions of the contour is obtained as the image contour position 64. An information processing device 10 according to claim 1 or 2.

[0136] [Appendix 4] When the size of the identifier 6 in the captured image 7 is larger than a predetermined value, the processing unit 100 determines the position of the imaging unit based on the image outer shape position 64 of the identifier 6. An information processing device 10 according to any one of appendices 1 to 3.

[0137] [Appendix 5] The processing unit 100 includes: If the size of the identifier 6 in the captured image 7 is smaller than a predetermined size, the position of the identifier 6 is acquired, and the position of the imaging unit 2 is determined based on the center position. 4. An information processing device 10 as described in appendix 3.

[0138] [Appendix 6] The processing unit 100 includes: Acquire a primary world imaging position 23 of the imaging unit 2 based on the image outer shape position 64 of the identifier 6 and the world known outer shape position 65; based on the primary world imaging position 23 and a world known identifier position 62 indicating a known three-dimensional position of the identifier 6 in a world coordinate system 70, the image identifier position 61 indicating the position of the identifier 6 on the captured image 7 is associated with the world known identifier position 62; determining a position of the imaging unit 2 based on the image identifier position 61 and the world known identifier position 62; 6. An information processing device 10 according to any one of appendices 1 to 5.

[0139] [Appendix 7] A function of acquiring a plurality of image outer shape positions 64 in an image coordinate system 71 from the outer shape of the identifier 6 by image processing based on a captured image 7 including the identifier 6 provided in the space captured by the imaging unit 2; A function of determining the position of the imaging unit 2 based on the image contour position 64 and a world known contour position 65 indicating a known three-dimensional position of the contour of the identifier 6 in a world coordinate system 70; A program that executes the following.

[0140] [Appendix 8] A step of acquiring a plurality of image outer shape positions 64 in an image coordinate system 71 from the outer shape of the identifier 6 by image processing based on a captured image 7 including the identifier 6 provided in the space captured by the imaging unit 2; determining a position of the imaging unit 2 based on the image contour position 64 and a world known contour position 65 indicating a known three-dimensional position of the contour of the identifier 6 in a world coordinate system 70; A positioning method including: [Explanation of symbols]

[0141] 1. Enthronement System 2. Imaging unit 3. Mobile 4. Ceiling 5 Markers 6 Identifiers 8. Windows 9. Building 10. Information processing device 18 Communication Equipment 28 Imaging unit communication device

Claims

1. acquiring, by image processing, image outer shape positions of each of a plurality of vertices of the outer shape of the identifier in an image coordinate system based on a captured image including an identifier having an asymmetric outer shape that is provided in a predetermined space and captured by an imaging unit present in the predetermined space; determining a position of the imaging unit based on the image contour position and a world known contour position indicating a known three-dimensional position corresponding to a plurality of vertices of the contour of the identifier in a world coordinate system; An information processing device comprising a processing unit.

2. acquiring a plurality of image outer shape positions in an image coordinate system from the outer shape of the identifier by image processing based on a captured image including an identifier provided in a predetermined space captured by an imaging unit present in the predetermined space; determining a position of the imaging unit based on the image outer shape position of the identifier whose size in the captured image is larger than a predetermined value and a world known outer shape position indicating a known three-dimensional position of the outer shape of the identifier in a world coordinate system; An information processing device comprising a processing unit.

3. acquiring a plurality of image outer shape positions in an image coordinate system from the outer shape of the identifier by image processing based on a captured image including an identifier provided in a predetermined space captured by an imaging unit present in the predetermined space; obtaining a primary world imaging position of the imaging unit based on the image outer shape position and a world known outer shape position indicating a known three-dimensional position of the outer shape of the identifier in a world coordinate system; Correlating an image identifier position indicating a position of the identifier on an image with a world known identifier position based on the primary world imaging position and a world known identifier position indicating a known three-dimensional position of the identifier in a world coordinate system; determining a position of the imaging unit based on the image identifier position and the world known identifier position; An information processing device comprising a processing unit.

4. The processing unit determines a position of the imaging unit based on the image outline position of the identifier having an asymmetric outline.

4. The information processing device according to claim 2.

5. The processing unit includes: At least one of the vertex positions of the contour and the center position derived from the vertex positions of the contour is obtained as the image contour position. The information processing device according to any one of claims 1 to 3.

6. The processing unit includes: When the size of the identifier in the captured image is larger than a predetermined size, the position of the imaging unit is determined based on the image outer shape position of the identifier.

4. The information processing device according to claim 1 or 3.

7. The processing unit includes: When the size of the identifier in the captured image is smaller than a predetermined size, the position of the identifier is acquired, and the position of the imaging unit is determined based on the center position. The information processing device according to claim 5 .

8. The processing unit includes: obtaining a primary world imaging position of the imaging unit based on the image outer shape position of the identifier and the world known outer shape position; Correlating an image identifier position indicating a position of the identifier on an image with a world known identifier position based on the primary world imaging position and a world known identifier position indicating a known three-dimensional position of the identifier in a world coordinate system; determining a position of the imaging unit based on the image identifier position and the world known identifier position; 3. The information processing device according to claim 1 or 2.

9. a function of acquiring, based on a captured image including an identifier having an asymmetrical contour located in a predetermined space captured by an imaging unit present in the predetermined space, image contour positions of each of a plurality of vertices of the contour of the identifier in an image coordinate system by image processing; a function of determining a position of the imaging unit based on the image contour position and a world known contour position indicating a known three-dimensional position corresponding to a plurality of vertices of the contour of the identifier in a world coordinate system; A program that executes the following.

Images