Image processing system, dimensional measurement system, image processing method, and program

Abstract

The present disclosure addresses the problem of providing an image processing system, a dimensional measurement system, an image processing method, and a program that can accurately measure the dimensions of an appliance installed on an installation surface. In an image processing system (1), a data acquisition unit (11) acquires data regarding a two-dimensional image generated by imaging an appliance (8) installed on an installation surface (9), and measurement data regarding the three-dimensional shape of the appliance (8). An outer shape setting unit (12) sets a region corresponding to the outer shape of the appliance (8) in the two-dimensional image as an appliance region. A plane estimation unit (13) estimates a detection plane corresponding to the installation surface (9) on the basis of the measurement data. A projection unit (14) forms a projection region on the detection plane by projecting the appliance region to the detection plane. A measurement unit (15) measures the dimensions of the appliance (8) on the basis of the projection region.

Description

【Technical Field】 【0001】 The present disclosure relates to an image processing system, a dimensional measurement system, an image processing method, and a program. 【Background Art】 【0002】 Patent Document 1 discloses a three-dimensional information measurement and display device that measures the dimensions of a rectangular plane such as a rectangle or a square on an object without optically requiring colors, patterns, or markers that serve as landmarks on the object. 【0003】 The three-dimensional information measurement and display device includes a depth image capturing unit, a plane detection unit, a plane contour extraction unit, and a dimension / position and orientation calculation unit. The depth image capturing unit captures an object and acquires a depth image having depth information for each pixel of the two-dimensional image. The plane detection unit extracts a plane region from the depth image acquired by the depth image capturing unit and calculates an equation of a plane including the extracted plane region. The plane contour extraction unit extracts a contour obtained by approximating the region obtained by projecting the plane region extracted by the plane detection unit onto the two-dimensional image to a quadrilateral. The dimension / position and orientation calculation unit converts the contour on the two-dimensional image extracted by the plane contour extraction unit into a contour on the plane specified by the equation by perspective projection transformation, and detects the lengths of the vertical and horizontal sides of the region surrounded by the converted contour. 【0004】 The above-mentioned Patent Document 1 discloses a technique for obtaining the dimensions of a rectangular plane on an object. 【0005】 However, with the technique of Patent Document 1, when attempting to measure the dimensions of an instrument installed on an installation surface, it is difficult to distinguish between the plane of the instrument and the installation surface, and it was difficult to measure the dimensions of the instrument installed on the installation surface. For example, with the technique of Patent Document 1, when the ratio of the plane occupying the surface of the instrument is low, it was difficult to measure the dimensions of the instrument installed on the installation surface. 【Prior Art Documents】 【Patent Documents】 【0006】 [Patent Document 1] International Publication No. 2014 / 147863 [Overview of the Initiative] 【0007】 The purpose of this disclosure is to provide an image processing system, a dimension measuring system, an image processing method, and a program that can accurately measure the dimensions of an appliance installed on a mounting surface. 【0008】 An image processing system relating to one aspect of this disclosure is: An image processing system that operates on both an instrument having a rectangular end face and an instrument having a circular end face, The system comprises a data acquisition unit, an external shape setting unit, a plane estimation unit, a projection unit, and a measurement unit. The data acquisition unit acquires a depth image including data of a two-dimensional image generated by imaging the fixture installed on the installation surface, and measurement data of the three-dimensional shape of the fixture. The external shape setting unit sets the area corresponding to the external shape of the fixture in the two-dimensional image as the fixture area. The plane estimation unit estimates the detection plane corresponding to the installation surface based on the measurement data. The projection unit forms a projected area on the detection plane by projecting the fixture area onto the detection plane. The measurement unit measures the dimensions of the fixture based on the projected area. . beforeThe external shape of the device in the two-dimensional image is the shape of the end face in the two-dimensional image. The external shape setting unit sets a rectangular area, which is the shape of the end face in the two-dimensional image, as the device area when the end face is rectangular, and sets an elliptical area, which is the shape of the end face in the two-dimensional image, as the device area when the end face is circular. The plane estimation unit, when the external shape setting unit has set a rectangular area as the device area, superimposes the rectangular device area onto the depth image and creates the detection plane data based on the measurement data around the rectangular device area. When the external shape setting unit has set an elliptical area as the device area, the plane estimation unit superimposes the elliptical device area onto the depth image and creates the detection plane data based on the measurement data around the elliptical device area. The measuring unit measures at least one of the dimensions of the long side and short side of the end face of the rectangle when the outer shape setting unit sets the rectangular area as the device area, and measures the diameter dimension of the end face of the circle when the outer shape setting unit sets the elliptical area as the device area. 【0009】 A dimension measuring system according to one aspect of this disclosure comprises the image processing system described above, the data of the two-dimensional image, and the measurement data. Depth image including It comprises an imaging unit that generates an image, and a device that generates an image. 【0010】 The image processing method relating to one aspect of this disclosure is: An image processing method performed on an instrument having a rectangular end face and an instrument having a circular end face,The process includes a data acquisition step, an outline setting step, a plane estimation step, a projection step, and a measurement step. The data acquisition step acquires a depth image including data from a two-dimensional image generated by imaging the fixture installed on the installation surface, and measurement data of the three-dimensional shape of the fixture. The outline setting step sets the area corresponding to the outline of the fixture in the two-dimensional image as the fixture area. The plane estimation step estimates the detection plane corresponding to the installation surface based on the measurement data. The projection step forms a projected area on the detection plane by projecting the fixture area onto the detection plane. The measurement step measures the dimensions of the fixture based on the projected area. . before The external shape of the device in the two-dimensional image is the shape of the end face in the two-dimensional image. The external shape setting step sets a rectangular area, which is the shape of the end face in the two-dimensional image, as the device area if the end face is rectangular, and sets an elliptical area, which is the shape of the end face in the two-dimensional image, as the device area if the end face is circular. The plane estimation step, if the external shape setting step sets a rectangular area as the device area, superimposes the rectangular device area onto the depth image and creates the detection plane data based on the measurement data around the rectangular device area, and if the external shape setting step sets an elliptical area as the device area, superimposes the elliptical device area onto the depth image and creates the detection plane data based on the measurement data around the elliptical device area. The measurement step measures at least one of the long side dimension and short side dimension of the end face of the rectangle if the outer shape setting step has set the rectangular area as the fixture area, and measures the diameter dimension of the end face of the circle if the outer shape setting step has set the elliptical area as the fixture area. 【0011】 A program relating to one aspect of this disclosure causes a computer system to execute the image processing method described above. [Brief explanation of the drawing] 【0012】 [Figure 1] Figure 1 is a block diagram showing a dimension measurement system equipped with an image processing system according to an embodiment. [Figure 2] Figure 2 is a perspective view showing an example of an instrument to be measured by the same dimensional measurement system. [Figure 3] Figure 3 shows a first example of a two-dimensional image in the same image processing system. [Figure 4] Figure 4 shows the instrument area set in the same two-dimensional image. [Figure 5] Figure 5 shows a second example of a two-dimensional image in the same image processing system. [Figure 6] Figure 6 shows the instrument area set in the same two-dimensional image. [Figure 7] Figure 7 shows the projection area on the detection plane in the same image processing system. [Figure 8] Figure 8 shows the projection area on another detection plane in the same image processing system. [Figure 9] Figure 9A shows a two-dimensional image in the image processing system of the first modified example. Figure 9B shows the contour of the instrument extracted from the two-dimensional image. [Figure 10] Figure 10 shows the instrument area set in the same two-dimensional image. [Figure 11] Figure 11 is a block diagram showing the outline setting section of the image processing system in the second modified example. [Figure 12] Figure 12 shows the auxiliary device area set in a 2D image in the same image processing system. [Figure 13] Figure 13 shows a contour image on the detection plane in the same image processing system. [Figure 14] Figure 14 is a block diagram showing the specific configuration of the third modified dimensional measurement system. [Figure 15] Figure 15 shows the first operation screen of the fourth modified example. [Figure 16] FIG. 16 is a diagram showing the second operation screen described above. [Figure 17] FIG. 17 is a flowchart showing an image processing method executed by the image processing system. 【Embodiments for Carrying Out the Invention】 【0013】 The following embodiments generally relate to an image processing system, a dimension measurement system, an image processing method, and a program. More specifically, the following embodiments relate to an image processing system, a dimension measurement system, an image processing method, and a program for measuring the dimensions of an instrument installed on an installation surface. 【0014】 Hereinafter, the image processing system, dimension measurement system, image processing method, and program according to the embodiments will be described in detail with reference to the drawings. However, each of the drawings described in the following embodiments is a schematic diagram, and the ratio of the size and thickness of each component does not necessarily reflect the actual dimensional ratio. 【0015】 Also, the embodiments described below are merely examples of the embodiments of the present disclosure. The present disclosure is not limited to the following embodiments, and various modifications can be made according to the design and the like as long as the effects of the present disclosure can be achieved. 【0016】 (1) Overview In a building, various instruments are installed on installation surfaces such as a ceiling surface, a wall surface, and a floor surface. The instrument 8 is a lighting fixture, an air conditioner, an electric appliance, a gas appliance, or the like. Also, the instrument is either embedded in the installation surface or installed on the surface of the installation surface. 【0017】 And, in building renovation work (such as reform, renovation, reconstruction, and extension), replacement of instruments (including exchange) may be considered. When considering replacement of instruments in such renovation work, it is preferable that information regarding the dimensions of the currently installed instruments can be measured easily and accurately. 【0018】 Therefore, the image processing system 1, with the configuration shown in Figure 1, measures the dimensions of the device 8 installed on the mounting surface 9. 【0019】 The image processing system 1 comprises a data acquisition unit 11, an outline setting unit 12, a plane estimation unit 13, a projection unit 14, and a measurement unit 15. The data acquisition unit 11 acquires data of a two-dimensional image generated by imaging the fixture 8 installed on the installation surface 9, and measurement data of the three-dimensional shape of the fixture 8. The outline setting unit 12 sets the area corresponding to the outline of the fixture 8 in the two-dimensional image as the fixture area. The plane estimation unit 13 estimates the detection plane corresponding to the installation surface 9 based on the measurement data. The projection unit 14 forms a projected area on the detection plane by projecting the fixture area onto the detection plane. The measurement unit 15 measures the dimensions of the fixture 8 based on the projected area. 【0020】 The image processing system 1 having the above configuration can accurately measure the dimensions of the fixture 8 installed on the mounting surface 9. In particular, the image processing system 1 can easily and accurately measure the dimensions of the fixture 8 installed on the mounting surface 9, even if, for example, the proportion of the surface of the fixture 8 is flat. 【0021】 The dimensional measurement system 100 also includes an image processing system 1 and an imaging unit 2. The imaging unit 2 images the instrument 8 and generates two-dimensional image data and measurement data of the three-dimensional shape of the instrument 8. 【0022】 The dimensional measuring system 100 having the above configuration can also accurately measure the dimensions of the device 8 installed on the mounting surface 9. 【0023】 (2)Details The following describes the details of the dimension measurement system 100 equipped with the image processing system 1 of this embodiment. 【0024】 The dimensional measurement system 100 further comprises an image processing system 1, an imaging unit 2, a display unit 3, and an operation unit 4. The image processing system 1 comprises a data acquisition unit 11, an outline setting unit 12, a plane estimation unit 13, a projection unit 14, and a measurement unit 15. 【0025】 The dimensional measurement system 100 is preferably implemented using a portable information terminal such as a tablet or smartphone. In this embodiment, the functions of the image processing system 1, the imaging unit 2, the display unit 3, and the operation unit 4 are all implemented in a single information terminal. The information terminal has an application for dimensional measurement processing to measure the dimensions of the equipment 8 pre-installed, and the dimensional measurement processing starts when the application is launched. The user, such as a worker, then moves around the building with the information terminal and takes an image of the equipment 8 to be measured. The image processing system 1 measures the dimensions of the equipment 8 based on the captured image of the equipment 8. 【0026】 (2.1) Apparatus The device 8 is installed on the mounting surface 9. 【0027】 The mounting surface 9 is a ceiling surface, wall surface, or floor surface, and is a flat surface or a surface with few irregularities. For example, the surface of a ceiling panel, wall panel, or floor panel corresponds to the mounting surface 9. 【0028】 The fixture 8 is a lighting fixture, air conditioning unit, electrical appliance, or gas appliance, etc. The fixture 8 may be installed either embedded in the installation surface 9 or installed on the surface of the installation surface 9. Furthermore, the shape of the fixture 8 is not limited to a specific shape. 【0029】 Figure 2 illustrates an example of fixture 8, which is a long, rectangular lighting fixture embedded in the ceiling. In this case, fixture 8 is the lighting fixture, and the installation surface 9 is the ceiling. 【0030】 (2.2) Imaging Unit The imaging unit 2 includes a two-dimensional camera 21 and a depth camera 22. The imaging unit 2 drives the two-dimensional camera 21 and the depth camera 22 simultaneously (or nearly simultaneously). The two-dimensional camera 21 and the depth camera 22 are positioned in close proximity, and the imaging range and direction of the two-dimensional camera 21 and the imaging range and direction of the depth camera 22 can be considered to be the same (or nearly the same). Furthermore, the imaging unit 2 has a function to correct for any discrepancies between the imaging range and direction of the two-dimensional camera 21 and the imaging range and direction of the depth camera 22. In other words, the imaging range, imaging direction, and imaging timing of the two-dimensional camera 21 and the depth camera 22 are the same (or nearly the same). 【0031】 The 2D camera 21 is a camera that generates 2D image data. The pixel value of each pixel that makes up the 2D image represents the grayscale value. If the 2D image is a monochrome image, the pixel value represents the grayscale grayscale value. If the 2D image is a color image, the pixel value represents the grayscale value of each color: red (R), green (G), and blue (B). 【0032】 The depth camera 22 is a camera that generates depth images. The pixel value of each pixel that makes up the depth image indicates the distance from the depth camera 22. In other words, the depth image contains measurement data of the three-dimensional shape of the object being imaged. The depth camera 22 can generate depth images using methods such as TOF (Time of Flight), stereo camera, structured illumination, and LiDAR (Light Detection and Ranging). 【0033】 While it is preferable that the imaging unit 2 is integrated with the information terminal T1, it may also be provided separately from the information terminal T1. 【0034】 (2.3) Display unit and operation unit The display unit 3 is a liquid crystal display or an organic EL display, and receives image data from the image processing system 1 and displays the image data. The image data is, for example, data for operation screens such as executing and stopping the dimension measurement process for measuring the dimensions of the device 8, and data for notification screens such as the execution process and execution results of the dimension measurement process. 【0035】 The operation unit 4 has a user interface function that accepts user input. The system has at least one user interface, such as the operation unit 4, a touch panel display, a keyboard, and a mouse. The user performs operations on the operation unit 4 to cause the image processing system 1 to perform dimensional measurement processing. 【0036】 The user then operates the control unit 4 while viewing the screen displayed on the display unit 3 to perform operations such as starting and stopping the dimension measurement process, as well as to check the process and results of the dimension measurement process. 【0037】 While it is preferable that the display unit 3 and the operation unit 4 are integrated with the information terminal T1, they may also be provided separately from the information terminal T1. 【0038】 (2.4) Image processing system The image processing system 1 preferably includes a computer system. The computer system implements part or all of the image processing system 1 by executing a program. The computer system includes a processor as its main hardware component, which operates according to the program. The processor can be of any type as long as it can implement functions by executing a program. The processor consists of one or more electronic circuits, including a semiconductor integrated circuit (IC) or a Large Scale Integration (LSI). Here, we refer to them as ICs and LSIs, but the name changes depending on the degree of integration, and they may also be called system LSIs, VLSIs (Very Large Scale Integrations), or ULSIs (Ultra Large Scale Integrations). Field-programmable gate arrays (FPGAs) that are programmed after the manufacture of the LSI, or reconfigurable logic devices that allow for the reconfiguration of junction relationships within the LSI or the setup of circuit compartments within the LSI, can also be used for the same purpose. Multiple electronic circuits may be integrated on a single chip or provided on multiple chips. Multiple chips may be aggregated in a single device or provided in multiple devices. The program is recorded on a non-temporary recording medium such as ROM, optical disc, or hard disk drive, which is readable by the computer system. The program may be pre-stored on the non-temporary recording medium, or it may be supplied to the non-temporary recording medium via a wide-area communication network, including the Internet. 【0039】 In this embodiment, a portable information terminal such as a tablet or smartphone constitutes the computer system. 【0040】 Furthermore, the computer system is not limited to a single computer device; it may be implemented using multiple computer devices working in conjunction with each other. Additionally, the computer system may be built as a cloud computing system. 【0041】 (2.4.1) Data Acquisition Section The data acquisition unit 11 acquires 2D image data and depth image data from the imaging unit 2. The 2D image data is image data generated by the 2D camera 21 of the imaging unit 2. The depth image data is image data generated by the depth camera 22 of the imaging unit 2 and includes measurement data of the 3D shape of the object being imaged. 【0042】 (2.4.2) External shape setting section The external shape setting unit 12 sets the area corresponding to the external shape of the device 8 in the two-dimensional image as the device area. In this embodiment, the external shape setting unit 12 sets the device area based on at least two coordinates set in the two-dimensional image by user operation. 【0043】 The following explanation will use the two-dimensional image Ga1 shown in Figure 3 and the two-dimensional image Ga2 shown in Figure 5. 【0044】 Figure 3 shows a two-dimensional image Ga1 as a first example of a two-dimensional image. Two-dimensional image Ga1 is, for example, an image of a long rectangular lighting fixture shown in Figure 2, taken from diagonally below as fixture 8. In this case, fixture 8 is a lighting fixture with a rectangular bottom surface, and the installation surface 9 is the ceiling surface. Two-dimensional image Ga1 includes the imaging area Ra1 of fixture 8 and the imaging area Ra2 of the installation surface 9. Two-dimensional image Ga1 is either a monochrome or color image, and the pixel value of each pixel represents the grayscale value. 【0045】 The external shape setting unit 12 displays the two-dimensional image Ga1 on the display unit 3. The user places pins Pn (where n is a natural number) on the two-dimensional image Ga1 by operating the operation unit 4 while viewing the two-dimensional image Ga1 displayed on the display unit 3. Pin Pn points to a point in the two-dimensional image Ga1 and specifies one coordinate in the two-dimensional image Ga1. In the two-dimensional image Ga1, the imaging area Ra1 is a long rectangle corresponding to the lower end surface of the device 8, and has two long sides and two short sides. Pins P1 and P2 each point to the ends of the longer of the two long sides L1 of the imaging area Ra1, and specify two coordinates corresponding to the ends of the long side L1 in the two-dimensional image Ga1. Pins P3 and P4 each point to the ends of the longer of the two short sides L2 of the imaging area Ra1, and specify two coordinates corresponding to the ends of the short side L2 in the two-dimensional image Ga1. 【0046】 Next, the outline setting unit 12 sets the parallelogram-shaped fixture region Ra3 in the 2D image Ga1 by fitting (parallelogram approximation) a parallelogram to include each coordinate specified by pins P1-P4 (see Figure 3), as shown in Figure 4. For example, the outline setting unit 12 generates a parallelogram approximated to include the long side L1 and the short side L2 by fitting. Then, for the parallelogram-shaped fixture region Ra3, the outline setting unit 12 determines the coordinates of the start Q1 and end Q2 of the long side L3 and the start Q3 and end Q4 of the short side L4 in the 2D image Ga1. 【0047】 Figure 5 shows a second example of a two-dimensional image, a two-dimensional image Ga2. Two-dimensional image Ga2 is, for example, an image of a downlight embedded in a ceiling surface, taken from an oblique angle below. In this case, fixture 8 is a lighting fixture with a circular lower end surface, and the installation surface 9 is the ceiling surface. Two-dimensional image Ga2 includes the imaging area Ra11 of fixture 8 and the imaging area Ra12 of the installation surface 9. Two-dimensional image Ga2 is either a monochrome or color image, and the pixel value of each pixel represents the grayscale value. 【0048】 The external shape setting unit 12 displays the two-dimensional image Ga2 on the display unit 3. The user places pins Pn (where n is a natural number) on the two-dimensional image Ga2 by operating the operation unit 4 while viewing the two-dimensional image Ga2 displayed on the display unit 3. Pin Pn points to a point in the two-dimensional image Ga2 and specifies one coordinate in the two-dimensional image Ga2. In the two-dimensional image Ga2, the imaging area Ra11 is elliptical. Pins P11 and P12 each point to the ends of the major axis L11 of the imaging area Ra11 (corresponding to the diameter of the lower end surface of the circular shape of the device 8), and specify two coordinates corresponding to the ends of the major axis L11 in the two-dimensional image Ga2. 【0049】 Next, the outline setting unit 12 sets the elliptical fixture region Ra13 in the 2D image Ga2 by fitting (elliptical approximation) an ellipse to include the coordinates indicated by pins P11-P12 (see Figure 5), as shown in Figure 6. For example, the outline setting unit 12 generates an ellipse that approximates to include the major axis L11 by fitting. Then, for the elliptical fixture region Ra13, the outline setting unit 12 determines the coordinates of the start Q11 and end Q12 of the major axis L12 of the fixture region Ra13 in the 2D image Ga2. 【0050】 (2.4.3) Plane estimation part The plane estimation unit 13 estimates a detected plane, which is the plane corresponding to the installation surface 9, based on the measurement data of the three-dimensional shape of the device 8. The measurement data is included in the depth image captured by the depth camera 22 of the imaging unit 2. The pixel value of each pixel constituting the depth image indicates the distance from the depth camera 22. In other words, the depth image contains the measurement data of the three-dimensional shape of the device 8. 【0051】 The plane estimation unit 13 overlays the equipment area onto the depth image and creates detection plane data corresponding to the installation surface 9 based on measurement data around the equipment area (outside the equipment area). The pixel value of each pixel in the detection plane indicates the distance from the depth camera 22 and includes measurement data of the three-dimensional shape of the equipment 8. Therefore, the plane estimation unit 13 can create detection plane data with high accuracy. 【0052】 For example, the plane estimation unit 13 overlays the equipment region Ra3 (see Figure 4) onto the depth image corresponding to the 2D image Ga1, and creates data for the detected plane M1 (see Figure 7) corresponding to the installation surface 9 based on the measurement data around the equipment region Ra3. 【0053】 Furthermore, the plane estimation unit 13 overlays the equipment region Ra13 (see Figure 6) onto the depth image corresponding to the 2D image Ga2, and creates data for the detected plane M11 (see Figure 8) corresponding to the installation surface 9 based on the measurement data around the equipment region Ra13. 【0054】 (2.4.4) Projection section The projection unit 14 projects the device area onto the detection plane estimated by the plane estimation unit 13, thereby forming a projection area on the detection plane. 【0055】 The following explanation will be given using Figures 7 and 8. 【0056】 In Figure 7, the projection unit 14 projects the instrument region Ra3 onto the detection plane M1 estimated by the plane estimation unit 13. For example, the projection unit 14 projects the coordinates of the start Q1 and end Q2 of the long side L3 and the start Q3 and end Q4 of the short side L4 of the instrument region Ra3 in the 2D image Ga1 onto the detection plane M1 as measurement coordinates Q1b, Q2b, Q3b, and Q4b. As a result, a parallelogram projection region Rb3 is formed on the detection plane M1 with the measurement coordinates Q1b, Q2b, Q3b, and Q4b as its vertices. The measurement coordinates Q1b, Q2b, Q3b, and Q4b are coordinates on the contour of the projection region Rb3. 【0057】 In Figure 8, the projection unit 14 projects the instrument region Ra13 onto the detection plane M11 estimated by the plane estimation unit 13. For example, the projection unit 14 projects the coordinates of the start Q11 and end Q12 of the major axis L12 of the instrument region Ra13 in the 2D image Ga2 onto the detection plane M11 as measured coordinates Q11b and Q12b. As a result, an elliptical projection region Rb13 with the measured coordinates Q11b and Q12b as its major axis is formed on the detection plane M11. The measured coordinates Q11b and Q12b are coordinates on the contour of the projection region Rb13. 【0058】 (2.4.5) Measuring part The measuring unit 15 measures the dimensions of the instrument 8 based on the projection area. The measuring unit 15 also uses the pixel values ​​of each pixel in the depth image corresponding to each of the pair of measurement coordinates on the detection plane, along with the pair of measurement coordinates, to determine the length dimension between the pair of measurement coordinates. In other words, the measuring unit 15 also uses the measurement data of the three-dimensional shape corresponding to each of the pair of measurement coordinates, along with the pair of measurement coordinates, to determine the length dimension between the pair of measurement coordinates. The combination of measurement coordinates and measurement data of the three-dimensional shape corresponds to three-dimensional coordinates that specify a position in three-dimensional space. 【0059】 In Figure 7, the measurement unit 15 measures the length dimension W1 between measurement coordinates Q1b and Q2b in the projected region Rb3 projected onto the detection plane M1 as the length dimension of the device 8. Specifically, the measurement unit 15 uses measurement coordinates Q1b, Q2b, the 3D measurement data at measurement coordinate Q1b, and the 3D measurement data at measurement coordinate Q2b to determine the distance in 3D space between measurement coordinates Q1b and Q2b as the length dimension W1. The measurement unit 15 also detects the length dimension W2 between measurement coordinates Q3b and Q4b in the projected region Rb3 projected onto the detection plane M1 as the length dimension of the device 8. Specifically, the measurement unit 15 uses measurement coordinates Q3b, Q4b, the 3D measurement data at measurement coordinate Q3b, and the 3D measurement data at measurement coordinate Q4b to determine the distance in 3D space between measurement coordinates Q3b and Q4b as the length dimension W2. In other words, the measuring unit 15 measures the length dimensions W1 and W2 of the instrument 8 based on multiple measurement coordinates Q1b, Q2b, Q3b, and Q4b on the contour of the projection area Rb3 in the detection plane M1, and the measurement data corresponding to each of the multiple measurement coordinates Q1b, Q2b, Q3b, and Q4b. The measuring unit 15 can measure the length dimensions W1 and W2 of the instrument 8 with high accuracy. 【0060】 In Figure 8, the measuring unit 15 measures the length dimension W11 between the measurement coordinates Q11b and Q12b of the projected region Rb13 projected onto the detection plane M11, as the diameter dimension of the device 8. Specifically, the measuring unit 15 uses the measurement coordinates Q11b, Q12b, the three-dimensional measurement data at measurement coordinate Q11b, and the three-dimensional measurement data at measurement coordinate Q12b to determine the distance in three-dimensional space between measurement coordinates Q11b and Q12b as the length dimension W11. In other words, the measuring unit 15 measures the length dimension W11 of the device 8 based on multiple measurement coordinates Q11b, Q12b on the contour of the projected region Rb13 on the detection plane M11, and the measurement data corresponding to each of the multiple measurement coordinates Q11b, Q12b. The measuring unit 15 can measure the length dimension W11 of the device 8 with high accuracy. 【0061】 The measuring unit 15 then outputs the measurement result of the dimensions of the instrument 8 to the display unit 3. The display unit 3 displays the measurement result of the dimensions of the instrument 8. 【0062】 The measurement unit 15 may also display on the display unit 3 a screen including the projection area Rb3 and measurement coordinates Q1b-Q4b shown in Figure 7, or a screen including the projection area Rb13 and measurement coordinates Q11b-Q12b shown in Figure 8. In this case, the user can correct the measurement results by operating the operation unit 4 to change the positions of the measurement coordinates Q1b-Q4b and Q11b-Q12b on the screen. 【0063】 Furthermore, the measurement unit 15 may display on the display unit 3 a screen including the detection plane M1 shown in Figure 7, or a screen including the detection plane M11 shown in Figure 8. In this case, the user can switch whether or not to display the detection planes M1 and M11 on the screen by operating the operation unit 4. 【0064】 (2.4.6) Summary As described above, the image processing system 1 estimates detection planes M1 and M11 corresponding to the installation surface 9, and projects the fixture areas Ra3 and Ra13 onto the detection planes M1 and M11 to form projected areas Rb3 and Rb13 on the detection planes M1 and M11. Then, the image processing system 1 measures the dimensions of the fixture 8 based on the projected areas Rb3 and Rb13. Therefore, the image processing system 1 can accurately measure the dimensions of the fixture 8 installed on the installation surface 9. The image processing system 1 can easily and accurately measure the dimensions of the fixture 8 installed on the installation surface 9, even if, for example, the proportion of the surface occupied by the fixture 8 is small. 【0065】 (3) First modified example The first modified example describes a modified version of the external shape setting unit 12. 【0066】 In this modified example, the external shape setting unit 12 sets the device area using a learning model. That is, the external shape setting unit 12 includes a learning model that receives 2D image data as input and outputs 2D image data with the device area set. 【0067】 The learning model is a model that takes a 2D image as input and outputs data for the instrument area set in the 2D image. The learning model is constructed using machine learning with a large number of 2D images as training data. For example, it is preferable that the learning model be constructed using machine learning such as deep learning using a neural network. 【0068】 For example, the learning model is constructed using deep learning with FCN (Fully Convolutional Networks). The machine learning uses a 2D image and a segmented image of the 2D image as training data. The segmented image of the 2D image is an image obtained by semantic segmentation, where labels are associated with all pixels that make up the 2D image, and the 2D image is divided into multiple classes. In this embodiment, the multiple classes divided by semantic segmentation include "appliances". As a result, when the learning model receives a 2D image as input, it can perform semantic segmentation on the 2D image and divide the 2D image into multiple classes. 【0069】 The learning model then sets the region corresponding to the "equipment" class as the equipment region if the "equipment" class exists in the 2D image. If the "equipment" class does not exist in the 2D image, the learning model determines that no equipment was detected and outputs that result. 【0070】 Specifically, the two-dimensional image Ga3 shown in Figure 9A is an image of the long, rectangular lighting fixture shown in Figure 2, taken from a diagonal downward angle, with fixture 8 as the fixture. In this case, fixture 8 is a lighting fixture with a rectangular bottom surface, and the installation surface 9 is the ceiling surface. The two-dimensional image Ga3 includes the imaging area Ra21 of fixture 8 and the imaging area Ra22 of the installation surface 9. The two-dimensional image Ga3 is either a monochrome or color image, and the pixel value of each pixel indicates the grayscale value. 【0071】 The outline setting unit 12 inputs the 2D image Ga3 to the learning model. The learning model performs semantic segmentation on the 2D image Ga3 and extracts the contour L20 of the "appliance" class from the 2D image Ga3 as shown in Figure 9B. Next, the learning model fits a rectangle to the contour L20 (quadrilateral approximation) to set a rectangular appliance region Ra23 in the 2D image Ga3 as shown in Figure 10. 【0072】 The fixture area Ra23 is a long rectangle with two long sides and two short sides. The learning model selects the longer of the two long sides of the rectangular fixture area Ra23, L21, and determines the coordinates of its start point Q21 and end point Q22. The learning model also selects the longer of the two short sides of the rectangular fixture area Ra23, L22, and determines the coordinates of its start point Q23 and end point Q24. 【0073】 The projection unit 14 then projects the coordinates of the start Q21 and end Q22 of the long side L21 and the start Q23 and end Q24 of the short side L22 of the instrument region Ra23 in the 2D image Ga3 onto the detection plane as measurement coordinates. 【0074】 Furthermore, the learning model may also be a model that uses other algorithms such as multiple regression analysis or support vector machines. 【0075】 As described above, the image processing system 1 can easily and accurately set the instrument region Ra23 by using a learning model. 【0076】 (4) Second variation In the second modified example, we will describe the external shape setting unit 12A, which is a modified version of the external shape setting unit 12. 【0077】 As shown in Figure 11, the outline setting unit 12A includes an auxiliary outline setting unit 121, an auxiliary projection unit 122, a contour line generation unit 123, and an outline determination unit 124. The auxiliary outline setting unit 121 sets an area corresponding to the outline of the instrument 8 as an auxiliary instrument area in a two-dimensional image. The auxiliary projection unit 122 forms an auxiliary projection area on the detection plane by projecting the auxiliary instrument area onto the detection plane. The contour line generation unit 123 generates a contour image of the auxiliary projection area based on the detection plane, using the measurement data as a reference. The outline determination unit 124 sets the instrument area based on the contour image. 【0078】 This will be explained in detail using Figures 12 and 13. 【0079】 The plane estimation unit 13 estimates a detected plane M1 (see Figure 13), which is the plane corresponding to the installation surface 9, based on the measurement data of the three-dimensional shape of the device 8. 【0080】 As shown in Figure 12, the auxiliary outline setting unit 121 sets the area corresponding to the outline of the device 8 in the two-dimensional image Ga1 as the auxiliary device area Rc1. The auxiliary outline setting unit 121 sets the area corresponding to the outline of the device 8 in the two-dimensional image Ga1 as the auxiliary device area Rc1, similar to the setting of the device area Ra3 (see Figure 4) by user operation as described in the above embodiment, or the setting of the device area Ra23 (see Figure 10) by the learning model as described in the first modified example. 【0081】 The auxiliary projection unit 122 projects the auxiliary device region Rc1 onto the detection plane M1, thereby forming an auxiliary projection region Rd1 on the detection plane M1 as shown in Figure 13. The auxiliary projection region Rd1 is a region formed in the same manner as the projection region Rb3 (see Figure 7) described above. 【0082】 The contour line generation unit 123 generates a contour line image Gb1 (see Figure 13) of the auxiliary projection region Rd1 based on the detection plane M1, using measurement data of the three-dimensional shape of the instrument 8. The measurement data is included in the depth image. The contour line image Gb1 is an image in which contour lines L31 to L33 are formed inside the auxiliary projection region Rd1. Contour lines L31 to L33 can be set in both the positive direction (convex) and the negative direction (concave) with respect to the detection plane M1. In other words, the contour line image Gb1 visually represents the three-dimensional shape of the instrument 8, such as its irregularities, by the contour lines L31 to L33 formed inside the auxiliary projection region Rd1. Specifically, the contour line image Gb1 is configured so that the irregularities of the instrument 8 can be easily visually grasped by making at least one of the colors or shades between each of the contour lines L31 to L33 different. 【0083】 The outline determination unit 124 then sets the instrument area based on the contour image Gb1. For example, the outline determination unit 124 superimposes the contour image Gb1 onto the two-dimensional image Ga1. The outline determination unit 124 sets the instrument area on the two-dimensional image Ga1, which includes the contour image Gb1, similar to the setting of the instrument area Ra3 by user operation described in the above embodiment, or the setting of the instrument area Ra3 by the learning model described in the first modified example. 【0084】 In machine learning, the learning model used by the outline determination unit 124 uses contour images and segmented contour images as training data. As a result, when the learning model used by the outline determination unit 124 receives contour images as input, it applies semantic segmentation processing to the contour images to divide them into multiple classes. Then, if the learning model used by the outline determination unit 124 has a "device" class in the contour images, it determines the region corresponding to the "device" class as the device region. If the learning model does not have a "device" class in the contour images, it determines that no device was detected and outputs the determination result. 【0085】 Then, similar to the embodiment described above, the projection unit 14 projects the instrument area onto the detection plane M1, thereby forming a projected area on the detection plane M1. The measuring unit 15 measures the dimensions of the instrument 8 based on the projected area. 【0086】 In this modified example, the external shape setting unit 12A sets the instrument area using a contour image Gb1 that visually represents the three-dimensional shape (concave and convex shape) of the instrument 8. Therefore, even if the contour of the imaging area Ra1 of the instrument 8 is unclear in the two-dimensional image Ga1, the contour of the imaging area Ra1 of the instrument 8 can be accurately determined by using the contour image Gb1. As a result, the external shape determination unit 124 can accurately set the instrument area. 【0087】 For example, when fixture 8 is a lighting fixture, if the imaging unit 2 images the lighting fixture while it is lit, the contour of the imaging area Ra1 of fixture 8 may become unclear in the 2D image Ga1 due to light expansion (halation). However, the external shape setting unit 12A in this modified example uses a contour image Gb1 to accurately determine the contour of the imaging area Ra1 of fixture 8 and accurately set the fixture area. 【0088】 (5) Third variation Figure 14 shows a specific configuration example of the dimensional measurement system 100. The dimensional measurement system 100 is preferably implemented by a user-portable information terminal T1 and a server device T2. The information terminal T1 is, for example, a tablet or a smartphone. The information terminal T1 includes the functions of a data acquisition unit 11, a plane estimation unit 13, a projection unit 14, a measurement unit 15, an imaging unit 2, a display unit 3, and an operation unit 4. The user carries the information terminal T1 and moves around inside the building. The server device T2 is installed in a remote location away from the building and includes the function of an external shape setting unit 12. 【0089】 Information terminal T1 has an application pre-installed for measuring the dimensions of the instrument 8, and is configured to communicate with server device T2 via a wide-area communication network NT1, including the Internet, by running this application. Information terminal T1 running the application can send and receive various types of data with server device T2. 【0090】 Then, the information terminal T1 transmits the 2D image data acquired by the data acquisition unit 11 to the server device T2. The outline setting unit 12 of the server device T2 sets the area corresponding to the outline of the device 8 as the device area in the 2D image. The server device T2 transmits the data of the set device area to the information terminal T1. In the information terminal T1, the plane estimation unit 13, projection unit 14, and measurement unit 15 perform the same processing as described above. 【0091】 In this modified example, the external shape setting unit 12 preferably sets the device area using a learning model, similar to the first modified example described above. In this case, by providing the external shape setting unit 12 on the server device T2, resources for the external shape setting unit 12 to perform processing using the learning model can be easily secured, and the load on the information terminal T1 can be reduced. 【0092】 (6) Fourth variation In the fourth modified example, the screen displayed on the display unit 3 (display screen) will be described. 【0093】 Figure 15 shows the first operation screen F1 displayed on the display unit 3 when the imaging unit 2 images the instrument 8. The display unit 3 has a rectangular display, and the rectangular first operation screen F1 displays the image Gc1 captured by the two-dimensional camera 21 of the imaging unit 2 in real time. 【0094】 Furthermore, the display unit 3 is a touch panel display, and the first operation screen F1 also has an operation unit 4. Specifically, operation units 41-43, which constitute the operation unit 4, are arranged in a line along the right edge of the first operation screen F1. Operation unit 41 is a shooting setting button, and by operating operation unit 41, it is possible to determine whether or not to use the shooting light and to select the data to be saved. Operation unit 42 is a shooting button, and when operation unit 42 is operated, the 2D camera 21 and the depth camera 22 are driven, and the imaging unit 2 generates data for the 2D image and the depth image. Operation unit 43 is a measurement mode setting button, and by operating operation unit 43, the mode for measuring the dimensions of the instrument 8 (measurement mode) is selected. Measurement modes include linear measurement mode, rectangular measurement mode, and diameter measurement mode. Linear measurement mode is a mode for measuring the dimension of one side of a straight line included in the contour of the instrument 8. Rectangular measurement mode is a mode for measuring the dimensions of the long side and short side of the rectangular contour of the instrument 8. The diameter measurement mode is a mode for measuring the diameter dimension of the circular contour of the instrument 8. 【0095】 Figure 16 shows the second operation screen F2 displayed on the display unit 3 after the imaging unit 2 has captured an image of the instrument 8. The display unit 3 has a rectangular display, and the rectangular second operation screen F2 displays the two-dimensional image Ga4 captured by the two-dimensional camera 21 of the imaging unit 2. 【0096】 Furthermore, the display unit 3 is a touch panel display, and the second operation screen F2 also has an operation unit 4. Specifically, the operation unit 44, which is the operation unit 4, is located outside the right edge of the two-dimensional image Ga4. The operation unit 44 is a measurement setting button, and is used for selecting display content and saving data as the dimensional measurement process progresses. 【0097】 Furthermore, a result display area Rx is also formed on the second operation screen F2. The result display area Rx is located outside the right-hand side of the two-dimensional image Ga4. The result display area Rx displays the measurement results of the dimensions of the instrument 8 by the measuring unit 15. 【0098】 Furthermore, it is preferable that the second operation screen F2 has a different screen configuration for each measurement mode selected by operating the operation unit 43 (measurement mode setting button) of the first operation screen F1. In other words, the second operation screen F2, which is the destination screen from the first operation screen F1, is different for each measurement mode selected by operating the operation unit 43 of the first operation screen F1. 【0099】 Furthermore, when 2D image and depth image data are generated by operating the operation unit 42 on the first operation screen F1, it is also possible to save the 2D image and depth image data to the image processing system 1 without transitioning to the second operation screen F2. 【0100】 (7) Fifth variation The data acquisition unit 11 is not limited to acquiring 2D image data of the instrument 8 and 3D shape measurement data of the instrument 8 from the imaging unit 2. For example, the 2D image data and 3D shape measurement data may be stored in a data server in advance, and the data acquisition unit 11 may acquire (read) the 2D image data and 3D shape measurement data from the data server. 【0101】 Furthermore, in this modified example, it is preferable that the second operation screen F2 displayed on the display unit 3 after reading the 2D image data and the 3D shape measurement data is configured differently for each measurement mode by selecting a measurement mode. 【0102】 (8) Sixth variation It is preferable that an object ID (identification information) is assigned to the combination of the 2D image data and the 3D shape measurement data of the device 8. In this case, the 2D image data and the 3D shape measurement data are managed using the object ID. 【0103】 The shape of the device 8 is not limited to the shapes described above, and may be other shapes such as hemispheres, prisms, cylinders, and irregular shapes. 【0104】 The fixture 8 is not limited to lighting fixtures, but may also be other fixtures such as air conditioners, electrical appliances, and gas appliances. 【0105】 The imaging unit 2 may be configured to generate point cloud data using a laser, such as a 3D laser scanner or LiDAR (Light Detection and Ranging). In this case, the measurement data of the 3D shape of the instrument 8 is included in the point cloud data. 【0106】 Furthermore, the imaging unit 2 may be a stereo camera having two or more lenses. 【0107】 Furthermore, the configurations of the above-described embodiments and each modified example can be combined as appropriate. 【0108】 (9) Image processing method The image processing method performed by the image processing system 1 described above can be summarized as shown in the flowchart in Figure 17. 【0109】 The image processing method includes a data acquisition step S1, an outline setting step S2, a plane estimation step S3, a projection step S4, and a measurement step S5. In the data acquisition step S1, the data acquisition unit 11 acquires data of two-dimensional images Ga1 and Ga2 generated by imaging the fixture 8 installed on the installation surface 9, and measurement data of the three-dimensional shape of the fixture 8. In the outline setting step S2, the outline setting unit 12 (or 12A) sets the areas corresponding to the outline of the fixture 8 in the two-dimensional images Ga1 and Ga2 as fixture areas Ra3 and Ra13. In the plane estimation step S3, the plane estimation unit 13 estimates the detection planes M1 and M11 corresponding to the installation surface 9 based on the measurement data. In the projection step S4, the projection unit 14 projects the fixture areas Ra3 and Ra13 onto the detection planes M1 and M11 to form projected areas Rb3 and Rb13 on the detection planes M1 and M11. In measurement step S5, the measuring unit 15 measures the dimensions of the instrument 8 based on the projection areas Rb3 and Rb13. 【0110】 The image processing method, including the steps described above, can accurately measure the dimensions of the fixture 8 installed on the mounting surface 9. The image processing method can easily and accurately measure the dimensions of the fixture 8 installed on the mounting surface 9, even if, for example, the proportion of the surface of the fixture 8 is flat. 【0111】 (10) Summary The first embodiment of the image processing system (1) comprises a data acquisition unit (11), an outline setting unit (12, 12A), a plane estimation unit (13), a projection unit (14), and a measurement unit (15). The data acquisition unit (11) acquires data of a two-dimensional image (Ga1, Ga2, Ga3) generated by imaging an instrument (8) installed on a mounting surface (9), and measurement data of the three-dimensional shape of the instrument (8). The outline setting units (12, 12A) set areas corresponding to the outline of the instrument (8) as instrument areas (Ra3, Ra13, Ra23) in the two-dimensional image (Ga1, Ga2, Ga3). The plane estimation unit (13) estimates detection planes (M1, M11) corresponding to the mounting surface (9) based on the measurement data. The projection unit (14) projects the instrument regions (Ra3, Ra13, Ra23) onto the detection planes (M1, M11), thereby forming projected regions (Rb3, Rb13) on the detection planes (M1, M11). The measuring unit (15) measures the dimensions of the instrument (8) based on the projected regions (Rb3, Rb13). 【0112】 The image processing system (1) described above can accurately measure the dimensions of the fixture (8) installed on the mounting surface (9). In particular, the image processing system (1) can easily and accurately measure the dimensions of the fixture (8) installed on the mounting surface (9) even if the proportion of the surface of the fixture (8) is flat is small. 【0113】 In the image processing system (1) of the second embodiment, in the first embodiment, it is preferable that the external shape setting unit (12) sets the instrument area (Ra3, Ra13) based on at least two coordinates set in a two-dimensional image (Ga1, Ga2) by user operation. 【0114】 The image processing system (1) described above can set the instrument area (Ra3, Ra13) based on user operation. 【0115】 In the third embodiment of the image processing system (1) according to the embodiment, in the first embodiment, the external shape setting unit (12) includes a learning model that receives data of a two-dimensional image (Ga3) as input and outputs data of an instrument region (Ra23) set in the two-dimensional image (Ga3). 【0116】 The image processing system (1) described above can easily and accurately define the instrument region (Ra23) by using a learning model. 【0117】 In the image processing system (1) of the fourth embodiment according to the embodiment, in any one of the first to third embodiments, the outline setting unit (12A) includes an auxiliary outline setting unit (121), an auxiliary projection unit (122), a contour line generation unit (123), and an outline determination unit (124). The auxiliary outline setting unit (121) sets an area corresponding to the outline of the instrument (8) as an auxiliary instrument area (Rc1) in a two-dimensional image (Ga1). The auxiliary projection unit (122) projects the auxiliary instrument area (Rc1) onto the detection plane (M1) to form an auxiliary projection area (Rd1) on the detection plane (M1). The contour line generation unit (123) generates a contour line image (Gb1) of the auxiliary projection area (Rd1) based on the detection plane (M1) using measurement data. The outline determination unit (124) sets the instrument area based on the contour line image (Gb1). 【0118】 The image processing system (1) described above can accurately determine the contour of the imaging area of ​​the instrument (8) and accurately set the instrument area by using contour images (Gb1). 【0119】 In the image processing system (1) of the fifth embodiment, in any one of the first to fourth embodiments, the measurement data is included in the depth image. The plane estimation unit (13) preferably superimposes the instrument regions (Ra3, Ra13, Ra23) onto the depth image and creates detection plane (M1, M11) data based on the measurement data around the instrument regions (Ra3, Ra13, Ra23). 【0120】 The image processing system (1) described above can create accurate detection plane data (M1, M11). 【0121】 In the sixth embodiment of the image processing system (1) according to the embodiment, in any one of the first to fifth embodiments, the measuring unit (15) preferably measures the dimensions (W1, W2, W11) of the instrument (8) based on a plurality of coordinates (Q1b, Q2b, Q3b, Q4b, Q11b, Q12b) on the contour of the projection area (Rb3, Rb13) in the detection plane (M1, M11), and measurement data corresponding to each of the plurality of coordinates (Q1b, Q2b, Q3b, Q4b, Q11b, Q12b). 【0122】 The image processing system (1) described above can measure the dimensions (W1, W2, W11) of the instrument (8) with high precision. 【0123】 A seventh embodiment of the dimensional measuring system (100) comprises an image processing system (1) according to any one of the first to sixth embodiments, and an imaging unit (2) that generates two-dimensional image (Ga1, Ga2, Ga3) data and measurement data. 【0124】 The above-described dimensional measuring system (100) can accurately measure the dimensions of the device (8) installed on the mounting surface (9). 【0125】 The eighth aspect of the image processing method according to the above-described embodiment includes a data acquisition step (S1), an outline setting step (S2), a plane estimation step (S3), a projection step (S4), and a measurement step (S5). The data acquisition step (S1) acquires data of a two-dimensional image (Ga1, Ga2, Ga3) generated by imaging the fixture (8) installed on the installation surface (9), and measurement data of the three-dimensional shape of the fixture (8). The outline setting step (S2) sets the area corresponding to the outline of the fixture (8) as the fixture area (Ra3, Ra13, Ra23) in the two-dimensional image (Ga1, Ga2, Ga3). The plane estimation step (S3) estimates the detection plane (M1, M11) corresponding to the installation surface (9) based on the measurement data. The projection step (S4) projects the instrument area (Ra3, Ra13, Ra23) onto the detection plane (M1, M11), thereby forming a projected area (Rb3, Rb13) on the detection plane (M1, M11). The measurement step (S5) measures the dimensions of the instrument (8) based on the projected area (Rb3, Rb13). 【0126】 The image processing method described above can accurately measure the dimensions of the device (8) installed on the mounting surface (9). 【0127】 The program of the ninth embodiment causes a computer system to execute the image processing method of the eighth embodiment. 【0128】 The program described above can accurately measure the dimensions of the device (8) installed on the mounting surface (9). [Explanation of Symbols] 【0129】 100 Dimensional Measurement System 1. Image Processing System 11 Data Acquisition Unit 12, 12A external shape setting section 121 Auxiliary external shape setting unit 122 Auxiliary projection section 123 Contour line generator 124 External shape determination section 13 Plane estimation part 14 Projection section 15 Measuring part 2 Imaging Unit 8. Equipment 9 Installation surface Ga1, Ga2, Ga3 2D images Ra3, Ra13, Ra23 Equipment area M1, M11 detection plane Rb3, Rb13 projection area Rc1 Auxiliary equipment area Rd1 Auxiliary projection area Gb1 contour image Q1b, Q2b, Q3b, Q4b, Q11b, Q12b Measurement coordinates (coordinates) W1, W2, W11 Length dimensions (dimensions) S1 Data Acquisition Step S2 Outline setting step S3 Plane estimation step S4 Projection Step S5 Measurement Step

Claims

[Claim 1] An image processing system that operates on an instrument having a rectangular end face and an instrument having a circular end face, A data acquisition unit that acquires two-dimensional image data generated by imaging the device installed on the mounting surface, and depth images including measurement data of the three-dimensional shape of the device, An external shape setting unit sets the area corresponding to the external shape of the device in the two-dimensional image as the device area, A plane estimation unit estimates a detection plane corresponding to the installation surface based on the measurement data, A projection unit that projects the device area onto the detection plane to form a projection area on the detection plane, The device comprises a measuring unit for measuring the dimensions of the device based on the projection area, The external shape of the device in the two-dimensional image is the shape of the end face in the two-dimensional image. The external shape setting unit is, If the end face is rectangular, the rectangular area in the two-dimensional image that corresponds to the shape of the end face is set as the device area. If the end face is circular, the elliptical region in the two-dimensional image that is the shape of the end face is set as the device area. The plane estimation unit, When the external shape setting unit sets the rectangular area as the device area, The rectangular instrument region is superimposed on the depth image, and the detection plane data is created based on the measurement data around the rectangular instrument region. When the external shape setting unit sets the elliptical region as the instrument region, the elliptical instrument region is superimposed on the depth image, and the detection plane data is created based on the measurement data around the elliptical instrument region. The aforementioned measuring unit is When the external shape setting unit sets the rectangular area as the device area, at least one of the dimensions of the long side and short side of the end face of the rectangle is measured. When the external shape setting unit sets the elliptical area as the device area, the diameter dimension of the circular end face is measured. Image processing system. [Claim 2] The external shape setting unit sets the device area based on at least two coordinates set in the two-dimensional image by user operation. The image processing system according to claim 1. [Claim 3] The external shape setting unit includes a learning model that receives the data of the two-dimensional image as input and outputs the data of the device area set in the two-dimensional image. The image processing system according to claim 1. [Claim 4] The external shape setting unit is, The two-dimensional image includes an auxiliary external shape setting unit that sets an area corresponding to the external shape of the device as an auxiliary device area, An auxiliary projection unit that projects the auxiliary device area onto the detection plane to form an auxiliary projection area on the detection plane, A contour generation unit generates contour images of the auxiliary projection region based on the measurement data, with the detection plane as the reference. The system includes an outline determination unit that sets the instrument area based on the contour image. The image processing system according to claim 1. [Claim 5] The measuring unit measures the dimensions of the instrument based on a plurality of coordinates on the contour of the projection area in the detection plane, and measurement data corresponding to each of the plurality of coordinates. The image processing system according to claim 1. [Claim 6] An image processing system comprising any one of claims 1 to 5, The system includes an imaging unit that generates the depth image, which includes the data of the two-dimensional image and the measurement data. Dimensional measurement system. [Claim 7] An image processing method performed on an instrument having a rectangular end face and an instrument having a circular end face, A data acquisition step includes acquiring a two-dimensional image data generated by imaging the device installed on the mounting surface, and a depth image including measurement data of the three-dimensional shape of the device, An outline setting step of setting the region corresponding to the outline of the device in the two-dimensional image as the device region, A plane estimation step in which a detection plane corresponding to the installation surface is estimated based on the measurement data, A projection step in which the device area is projected onto the detection plane to form a projection area on the detection plane, The measurement step includes measuring the dimensions of the instrument based on the projection area, The external shape of the device in the two-dimensional image is the shape of the end face in the two-dimensional image. The aforementioned external shape setting step is, If the end face is rectangular, the rectangular area in the two-dimensional image that corresponds to the shape of the end face is set as the device area. If the end face is circular, the elliptical region in the two-dimensional image that is the shape of the end face is set as the device area. The aforementioned plane estimation step is, If the outline setting step sets the rectangular area as the instrument area, the rectangular instrument area is superimposed on the depth image, and the detection plane data is created based on the measurement data around the rectangular instrument area. If the outline setting step sets the elliptical region as the instrument region, the elliptical instrument region is superimposed on the depth image, and the detection plane data is created based on the measurement data around the elliptical instrument region. The measurement step is, If the external shape setting step sets the rectangular area as the appliance area, then at least one of the dimensions of the long side and short side of the end face of the rectangle is measured. If the external shape setting step sets the elliptical region as the device region, the diameter dimension of the circular end face is measured. Image processing methods. [Claim 8] Cause a computer system to execute the image processing method of claim 7. program.

Images