Detection method and detection system

Abstract

The present application provides a detection method and a detection system to obtain three-dimensional coordinate values of a first region with sufficient accuracy. The detection method detects a three-dimensional shape of a first region (AR1) including a second region (AR2) smaller than the first region (AR1) by an infrared projector (13) and a depth camera (11), and includes the following processes: obtaining three-dimensional coordinate values of at least two points of the second region (AR2) based on a depth image (DP) of the depth camera (11); calculating a position of a plane (PL) including the second region (AR2) based on the three-dimensional coordinate values of the two points; and determining the position of the plane (PL) as the three-dimensional shape of the first region (AR1).

Description

Technical Field

[0001] The present invention relates to a detection method and a detection system. Background Art

[0002] Conventionally, there is known a technique for obtaining the three-dimensional shape of a detection area using an infrared distance measuring sensor.

[0003] For example, Patent Document 1 describes a method in which three-dimensional coordinate point cloud data of the ground is acquired using a depth sensor, and a plane representing the ground is estimated based on the acquired plurality of point cloud data using, for example, the least squares method.

[0004] Patent Document 1: International Publication No. 2020-013021

[0005] In existing document 1, it is envisioned that the three-dimensional coordinates of the ground can be accurately obtained through a depth sensor. However, for example, when using an infrared-based ranging sensor, it may not be possible to obtain the three-dimensional coordinate values ​​of the ground with sufficient accuracy due to the reflection characteristics of the ground. Summary of the Invention

[0006] One method disclosed herein is a method for detecting the three-dimensional shape of a first area using an infrared ranging sensor, wherein the first area includes a second area that is smaller than the first area. The detection method includes the following processing: obtaining three-dimensional coordinate values ​​of at least two points in the second area based on measurement values ​​of the ranging sensor; calculating the position of a plane including the second area based on the three-dimensional coordinate values ​​of the at least two points; and determining the position of the plane as the three-dimensional shape of the first area.

[0007] Another embodiment of the present disclosure is a detection system comprising an infrared ranging sensor and an information processing device for detecting the three-dimensional shape of a first area, wherein the first area includes a second area that is smaller than the first area, and the information processing device performs the following processing: obtaining the three-dimensional coordinate values ​​of at least two points in the second area based on the measurement values ​​of the ranging sensor; calculating the position of a plane including the second area based on the three-dimensional coordinate values ​​of the at least two points; and determining the position of the plane as the three-dimensional shape of the first area. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1 This is a diagram showing an example of the configuration of the detection system according to the first embodiment.

[0009] Figure 2 This is a diagram showing the correspondence between the camera coordinates of the depth camera and the RGB camera.

[0010] Figure 3This is a diagram showing an example of the structure of a personal computer.

[0011] Figure 4 This is a diagram showing an example of a method for detecting the three-dimensional shape of the first region.

[0012] Figure 5 This is a diagram showing an example of a method for detecting a three-dimensional shape including a first object.

[0013] Figure 6 This is a diagram showing an example of a method for detecting a three-dimensional shape including a first object and a second object.

[0014] Figure 7 This is a diagram showing an example of a method for detecting a first object and a second object.

[0015] Figure 8 This is a flowchart showing an example of processing of a personal computer.

[0016] Figure 9 This is a flowchart showing an example of processing of a personal computer.

[0017] Figure 10 This is a diagram showing an example of the configuration of a personal computer according to the second embodiment.

[0018] Figure 11 This is a flowchart showing an example of processing of a personal computer.

[0019] Label Description

[0020] 1: Detection system; 10: Detection device; 11: Depth camera; 12: RGB camera; 13: Infrared projector; 20: Personal computer (information processing device); 21: Control unit; 21A: Processor; 211: Coordinate acquisition unit; 212: Plane calculation unit; 213: Shape determination unit; 214: Ratio determination unit; 215: Notification unit; 216: Position determination unit; 221: First acquisition unit; 222: Second acquisition unit; 223: First detection unit; 224: Third acquisition unit; 225, 226: Second detection unit; 21B: Memory; 23: Display mechanism; 231: Display panel; 24: Sound output mechanism; 241: Speaker Sounder; 25: Interface mechanism; 26: Operating mechanism; 30: Projector; AR1: First area; AR2: Second area; B: Brightness; BJ1: First object; BJ11: Rectangular block; BJ12: Wall; BJ2: Second object; CP: RGB image; DP: Depth image; DP1: First depth image; DP2: Second depth image; DP3: Third depth image; FL: Ground; JL1: First distance information; JL2: Second distance information; JL3: Third distance information; L1: First length; L2: Second length; MK: Marker; PGM: Control program; PL: Plane; RF: Reflector; RT: Ratio; TH: Threshold. DETAILED DESCRIPTION

[0021] Hereinafter, this embodiment will be described with reference to the accompanying drawings. Figures 1 to 9 The first embodiment and reference Figure 10 and Figure 11 The second embodiment will be described.

[0022] Figure 1 It is a diagram showing an example of the configuration of the detection system 1 according to the first embodiment.

[0023] The detection system 1 includes a detection device 10 , a personal computer 20 , and a projector 30 .

[0024] The personal computer 20 is communicably connected to the detection device 10 and the projector 30 .

[0025] The personal computer 20 is communicatively connected to the detection device 10 via a USB (Universal Serial Bus) cable, for example. The personal computer 20 is also communicatively connected to the projector 30 via an HDMI (registered trademark) cable.

[0026] The detection device 10 is provided with a distance measuring sensor of an infrared system. In the present embodiment, the detection device 10 is constituted by an RGB-D (Depth) camera. As shown in FIG. 1, the detection device 10 is provided with a depth camera 11, an RGB camera 12, and an infrared projector 13. Figure 2

[0027] The infrared projector 13 irradiates infrared light to an object.

[0028] The depth camera 11 acquires a depth image DP in which a depth value (Z coordinate) is associated with a coordinate (Ud, Vd) of the depth camera 11, by measuring reflected light of the infrared light irradiated by the infrared projector 13, in a so-called ToF (Time Of Flight) system. The X axis is parallel to the Ud axis, and the Y axis is parallel to the Vd axis. The X axis, the Y axis, and the Z axis are orthogonal to each other.

[0029] The RGB camera 12 acquires an RGB image CP of the object.

[0030] Reference will be made to Figure 2 The depth image DP, the Ud axis, the Vd axis, and the RGB image CP will be described.

[0031] The infrared projector 13 and the depth camera 11 correspond to an example of the "distance measuring sensor of an infrared system".

[0032] The depth image DP corresponds to an example of the "measured value of the distance measuring sensor".

[0033] In the present embodiment, a case where the "distance measuring sensor of an infrared system" is constituted by the infrared projector 13 and the depth camera 11 of the RGB-D camera will be described, but is not limited thereto.

[0034] The "distance measuring sensor of an infrared system" can also be constituted by stereo vision including two IR (Infrared) cameras. In addition, in the stereo vision, it is necessary to associate the coordinates of the corresponding positions on two captured images. However, it is difficult to accurately determine the corresponding point positions on the captured images, and the measurement accuracy can become unstable. Therefore, a specific marker can be arranged on the floor FL or the like, and the corresponding positions between the two captured images can be determined. As the specific marker, a retro-reflective material can be preferably used.

[0035] In addition, the "distance measuring sensor of an infrared system" can also be constituted by structured light provided with an IR camera and an IR projector.

[0036] The personal computer 20 acquires the depth image DP from the detection device 10. In addition, the personal computer 20 detects the three-dimensional shape of the first region AR1 based on the depth image DP. ​

[0037] The angle of view θ1 represents the field of view of the depth camera 11 corresponding to the first area AR1. The center line LC1 represents the center of the field of view of the depth camera 11. The center line LC1 is parallel to the Z axis.

[0038] The personal computer 20 corresponds to an example of an “information processing device”.

[0039] The projector 30 projects image light onto the floor FL according to instructions from the personal computer 20, forming a projection image in a projection area ARP of the floor FL. The projection angle θP represents the diffusion angle of the image light projected by the projector 30 toward the projection area ARP of the floor FL.

[0040] The projection area ARP is included in the first area AR1 . The center line LC2 indicates the center of the projection light projected by the projector 30 .

[0041] like Figure 1 As shown, the detection device 10 and the projector 30 are arranged so that the position of the intersection of the center line LC2 and the floor FL coincides with the position of the intersection of the center line LC1 and the floor FL.

[0042] Figure 2 1 is a diagram showing the correspondence relationship between the camera coordinates of the depth camera 11 and the RGB camera 12 .

[0043] The detection device 10, that is, the RGB-D camera, can obtain the RGB image CP obtained by the RGB camera 12, the depth image DP obtained by the depth camera 11, and the three-dimensional XYZ coordinates, and generate point group data, that is, point cloud data, having six-dimensional data (R, G, B, X, Y, Z) with color information.

[0044] The RGB camera 12 and the depth camera 11 are different devices, so their coordinate systems are usually inconsistent. Therefore, in order to generate point cloud data with color information, a correspondence between the pixel coordinate systems of the RGB image CP from the RGB camera 12 and the depth image DP from the depth camera 11 is required.

[0045] The correspondence relationship of the pixel coordinate system is prepared as a default value in, for example, a commercially available SDK (Software Development Kit) or an external library, and can be acquired using an API (Application Programming Interface).

[0046] For example, Figure 2As shown, by inputting a point (Ud, Vd) in the depth image DP and the corresponding depth value (Z coordinate), the corresponding point (Uc, Vc) in the RGB image CP is obtained. The Ud axis is, for example, parallel to the long side of the depth image DP. The Vd axis is, for example, parallel to the short side of the depth image DP. The Uc axis is, for example, parallel to the long side of the RGB image CP. The Vc axis is, for example, parallel to the short side of the RGB image CP.

[0047] When using a commercially available RGB-D camera unit in which the RGB camera 12 and the depth camera 11 are integrally assembled, there is no need to perform additional operations such as calibration of external parameters. By using the default values ​​prepared by the manufacturer, the pixel coordinate systems of the RGB image CP and the depth image DP can be matched.

[0048] Figure 3 It is a diagram showing an example of the configuration of the personal computer 20 .

[0049] The personal computer 20 includes a control unit 21, a display unit 23, a sound output unit 24, and an interface unit 25. The control unit 21 is connected to an operating unit 26 in a communicable manner.

[0050] The display mechanism 23 includes a display panel 231 that displays various images according to the control of the control unit 21. The display panel 231 includes, for example, an LCD (Liquid Crystal Display) and is configured in a rectangular shape.

[0051] The sound output mechanism 24 includes a speaker 241 that outputs various sounds according to the control of the control unit 21 .

[0052] The interface mechanism 25 is a communication interface for performing data communication with external devices including the detection device 10 and the projector 30. For example, it is a wired communication interface compliant with the HDMI (registered trademark) standard and the USB standard. The interface mechanism 25 is, for example, an interface substrate having a connector and an interface circuit, and is connected to a main substrate on which the processor 21A of the control unit 21 is mounted. Alternatively, the connector and interface circuit constituting the interface mechanism 25 are mounted on the main substrate of the control unit 21.

[0053] The operating mechanism 26 receives an operation from a user, generates an operation signal, and outputs the operation signal to the control unit 21. The operating mechanism 26 includes, for example, a keyboard and a mouse.

[0054] The control unit 21 includes a processor 21A and a memory 21B, and controls each unit of the personal computer 20 .

[0055] The memory 21B is a storage device that nonvolatilely stores programs and data executed by the processor 21A. The memory 21B is composed of a magnetic storage device, a semiconductor storage element such as a flash ROM (Read Only Memory), or another type of nonvolatile storage device.

[0056] Furthermore, the memory 21B may include a RAM (Random Access Memory) constituting a work area of ​​the processor 21 A. The memory 21B stores data processed by the control unit 21 and a control program PGM executed by the processor 21A.

[0057] The processor 21A may be composed of a single processor or a plurality of processors may function as the processor 21 A. The processor 21A executes a control program to control each unit of the personal computer 20 .

[0058] The control unit 21 includes a coordinate acquisition unit 211, a plane calculation unit 212, a shape determination unit 213, a ratio determination unit 214, a notification unit 215, a position identification unit 216, a first acquisition unit 221, a second acquisition unit 222, a first detection unit 223, a third acquisition unit 224, and a second detection unit 225. Specifically, the processor 21A of the control unit 21 executes a control program PGM stored in the memory 21B to function as the coordinate acquisition unit 211, the plane calculation unit 212, the shape determination unit 213, the ratio determination unit 214, the notification unit 215, the position identification unit 216, the first acquisition unit 221, the second acquisition unit 222, the first detection unit 223, the third acquisition unit 224, and the second detection unit 225.

[0059] Below, refer to Figure 3 and Figures 4 to 7 , the structure of the control unit 21 is explained.

[0060] Figure 4 It is a diagram showing an example of a method for detecting the three-dimensional shape of the first area AR1.

[0061] like Figure 4 As shown, the first area AR1 includes a second area AR2 which is smaller than the first area AR1. The second area AR2 is an area where a reflector RF that reflects infrared rays is provided by the user.

[0062] The reflector RF is an object that reflects infrared rays, such as white cloth.

[0063] In this embodiment, the reflector RF is described as a white cloth, but the invention is not limited thereto. The reflector RF can be any object that reflects infrared light. For example, the reflector RF can be a thin plate-shaped object coated with a paint that reflects infrared light.

[0064] In addition, a mark MK for identifying the second area AR2 is arranged in the second area AR2. The mark MK is formed of, for example, a specific color, for example, red.

[0065] The position identifying unit 216 acquires the RGB image CP from the RGB camera 12 and identifies the position corresponding to the second area AR2 in the RGB image CP. The position identifying unit 216 identifies the position corresponding to the second area AR2 by detecting the red marker MK in the RGB image CP.

[0066] The RGB image CP corresponds to an example of a “captured image”.

[0067] Alternatively, the position identifying unit 216 may identify the position corresponding to the second area AR2 by receiving an operation for specifying a measurement value corresponding to the second area AR2 in the depth image DP via the operating mechanism 26. For example, the position identifying unit 216 may identify the position corresponding to the second area AR2 in the depth image DP based on an operation by the user for specifying the position of the depth image DP corresponding to the four corners of the second area AR2.

[0068] Furthermore, in the depth image DP, when the position identifying unit 216 receives an operation by the user to designate a measurement value corresponding to the second area AR2 via the operating mechanism 26 , the marker MK identifying the second area AR2 may not be arranged in the second area AR2 .

[0069] The depth image DP corresponds to an example of “the measurement value of the first area by the distance measuring sensor”.

[0070] In this embodiment, the second area AR2 is formed in a rectangular shape, and the marks MK include a first mark MK1, a second mark MK2, a third mark MK3, and a fourth mark MK4. The first mark MK1 to the fourth mark MK4 are respectively arranged at the four corners of the second area AR2.

[0071] like Figure 4 As shown, the first length L1 indicating the length of the first area AR1 in the X-axis direction is longer than the second length L2 indicating the length of the second area AR2 in the X-axis direction. Furthermore, the first width W1 indicating the width of the first area AR1 in the Y-axis direction is longer than the second width W2 indicating the width of the second area AR2 in the Y-axis direction.

[0072] The coordinate acquisition unit 211 acquires three-dimensional coordinate values ​​of two points in the second area AR2 based on the depth image DP.

[0073] In this embodiment, for example, the three-dimensional coordinate values ​​of the first corner CN1 of the second area AR2 where the first marker MK1 is arranged and the second corner CN2 of the second area AR2 where the second marker MK2 is arranged are obtained.

[0074] The depth image DP corresponds to an example of “measurement value of the distance measuring sensor”.

[0075] In this embodiment, the markers MK are arranged at the four corners of the second area AR2, but the present invention is not limited thereto. The markers MK may be arranged in such a manner that the second area AR2 can be identified in the RGB image CP. For example, the markers MK may be arranged in the center of the second area AR2.

[0076] The plane calculation unit 212 calculates the position of the plane PL including the second area AR2 based on the three-dimensional coordinate values ​​of two points in the second area AR2.

[0077] The plane PL is defined by the following equation (1).

[0078] Z=((Z2-Z1) / (Y2-Y1))×Y (1)

[0079] Here, (X1, Y1, Z1) represents the (X, Y, Z) coordinate values ​​of the first corner CN1, and (X2, Y2, Z2) represents the (X, Y, Z) coordinate values ​​of the second corner CN2.

[0080] When the Xd axis of the depth camera 11 is parallel to the X axis and the Yd axis of the depth camera 11 is parallel to the Y axis, the plane PL defined by equation (1) coincides with the plane including the second area AR2. In other words, when the depth camera 11 is positioned at an appropriate rotation angle relative to the center line LC1, the plane PL is defined by equation (1).

[0081] The shape determination unit 213 determines the position of the plane PL as the three-dimensional shape of the first area AR1 .

[0082] In addition, in this embodiment, the position of the plane PL including the second area AR2 is calculated based on the three-dimensional coordinate values ​​of two points in the second area AR2, but the present invention is not limited to this. The plane calculation unit 212 can simply calculate the position of the plane PL including the second area AR2 based on the three-dimensional coordinate values ​​of at least two points in the second area AR2.

[0083] For example, the plane calculation unit 212 may calculate the position of the plane PL including the second area AR2 based on the three-dimensional coordinate values ​​of three points in the second area AR2.

[0084] In this case, the plane PL is defined by the following equation (2).

[0085]

[0086] Here, (X3, Y3, Z3) represents the (X, Y, Z) coordinate values ​​of the third corner CN3.

[0087] In this embodiment, the case where (X1, Y1, Z1), (X2, Y2, Z2) and (X3, Y3, Z3) represent the (X, Y, Z) coordinate values ​​of the first corner CN1, the second corner CN2 and the third corner CN3, respectively, is described, but is not limited to this.

[0088] When the plane PL is defined by equation (1), (X1, Y1, Z1) and (X2, Y2, Z2) exist in the second area AR2, and the (X, Y, Z) coordinate values ​​of any two points having different (X, Y, Z) coordinate values ​​can be used.

[0089] When the plane PL is defined by formula (2), (X1, Y1, Z1), (X2, Y2, Z2) and (X3, Y3, Z3) exist in the second area AR2, and the (X, Y, Z) coordinate values ​​of any three points with different (X, Y, Z) coordinate values ​​can be used.

[0090] Alternatively, for example, the plane calculation unit 212 may calculate the position of the plane PL including the second area AR2 based on the three-dimensional coordinate values ​​of four or more points in the second area AR2. In this case, the position of the plane PL can be calculated using a least squares method or the like.

[0091] In this way, the shape determination unit 213 determines the position of the plane PL as the three-dimensional shape of the first area AR1 , and thus can determine the accurate three-dimensional shape of the first area AR1 through simple processing.

[0092] Furthermore, the plane calculation unit 212 calculates the position of the plane PL including the second area AR2 based on the three-dimensional coordinate values ​​of two points in the second area AR2 . Therefore, the position of the plane PL can be calculated through simple processing.

[0093] The ratio determination unit 214 determines whether or not the ratio RT of the size of the second area AR2 to the size of the first area AR1 is equal to or smaller than a threshold value TH.

[0094] For example, when the plane calculation unit 212 calculates the position of the plane PL including the second area AR2 based on the three-dimensional coordinate values ​​of the first corner CN1 and the second corner CN2 of the second area AR2, the size of the first area AR1 is, for example, a first length L1, and the size of the second area AR2 is, for example, a second length L2. The first length L1 is the length of the first area AR1 in the X-axis direction. The second length L2 is the length of the second area AR2 in the X-axis direction.

[0095] In this case, the ratio RT is found by the following formula (3).

[0096] RT = L2 / L1 (3)

[0097] In this case, the threshold value TH is, for example, 0.2.

[0098] In addition, for example, in a case where the planar calculating section 212 calculates the position of the plane PL including the second region AR2 based on the three-dimensional coordinate values of the first corner portion CN1, the second corner portion CN2, and the third corner portion CN3 of the second region AR2, the size of the first region AR1 is, for example, the first area S1, and the size of the second region AR2 is, for example, the second area S2. The first area S1 is the area of the first region AR1. The second area S2 is the area of the second region AR2.

[0099] In this case, the ratio RT is found by the following formula (4).

[0100] RT = S2 / S1 = (L2 x W2) / (L1 x W1) (4)

[0101] In this case, the threshold value TH is, for example, 0.14.

[0102] The informing section 215 performs informing that urges the user to perform the setting of the reflector RF in a case where the ratio determining section 214 determines that the ratio RT is equal to or lower than the threshold value TH. In other words, the informing section 215 performs informing that urges the user to increase the size of the second region AR2 in a case where the ratio determining section 214 determines that the ratio RT is equal to or lower than the threshold value TH.

[0103] In addition, the informing section 215 informs the user of the possibility that the detection accuracy of the three-dimensional shape of the first region AR1 is low in a case where the ratio determining section 214 determines that the ratio RT is equal to or lower than the threshold value TH.

[0104] The informing performed by the informing section 215 to the user is performed, for example, by at least one of the display of the display panel 231, the sound output from the sound output mechanism 24, and the display of the projected image by the projector 30.

[0105] The first acquiring section 221 acquires first distance information JL1 that represents the three-dimensional shape of the first region AR1 based on the position of the plane PL. The first distance information JL1 corresponds to the first depth image DP1 that represents the position of the plane PL. For example, the first acquiring section 221 generates a depth image DP that represents the three-dimensional shape of the first region AR1 decided by the shape deciding section 213 as the first depth image DP1.

[0106] With regard to the first depth image DP1, reference will be made to Figure 7Further explanation.

[0107] Figure 5 1 is a diagram showing an example of a method for detecting a three-dimensional shape including a first object BJ1.

[0108] Figure 5 and Figure 4 The difference is that the first object BJ1 is disposed on the ground FL. In this embodiment, the first object BJ1 includes a rectangular parallelepiped BJ11 and a wall BJ12.

[0109] The rectangular parallelepiped BJ11 is placed on the floor FL. The wall BJ12 is fixed to the floor FL and is erected in a direction parallel to the vertical direction.

[0110] After the first acquisition unit 221 acquires the first distance information JL1 , the second acquisition unit 222 acquires second distance information JL2 indicating the three-dimensional shape within the first area AR1 based on the depth image DP generated by the depth camera 11 .

[0111] The depth image DP corresponds to an example of “measurement value of the distance measuring sensor”.

[0112] like Figure 5 As shown, the second distance information JL2 corresponds to the second depth image DP2 generated by the depth camera 11 when the first object BJ1 is arranged on the ground FL.

[0113] Regarding the second depth image DP2, reference will be made to Figure 7 Further explanation.

[0114] The first detection unit 223 detects the first object BJ1 within the first area AR1 based on the first distance information JL1 and the second distance information JL2 .

[0115] Specifically, the first detection unit 223 detects the first object BJ1 in the first area AR1 based on the first depth image DP1 and the second depth image DP2 .

[0116] Will refer to Figure 7 The processing of the first detection unit 223 will be further described.

[0117] Figure 6 1 is a diagram showing an example of a method for detecting a three-dimensional shape including a first object BJ1 and a second object BJ2 ​​.

[0118] Figure 6 and Figure 5 The difference is that the second object BJ2 ​​is disposed on the ground FL. In this embodiment, the second object BJ2 ​​corresponds to a person entering the first area AR1.

[0119] After acquiring the second distance information JL2 , the third acquisition unit 224 acquires third distance information JL3 indicating the three-dimensional shape within the first area based on the depth image DP generated by the depth camera 11 .

[0120] The depth image DP corresponds to an example of “measurement value of the distance measuring sensor”.

[0121] like Figure 6 As shown, the third distance information JL3 corresponds to the third depth image DP3 generated by the depth camera 11 when the first object BJ1 and the second object BJ2 ​​are arranged on the ground FL.

[0122] Regarding the third depth image DP3, reference will be made to Figure 7 Further explanation.

[0123] The second detection unit 225 detects the second object BJ2 ​​in the first area AR1 based on the second distance information JL2 and the third distance information JL3 .

[0124] Regarding the processing of the second detection unit 225, please refer to Figure 7 Further explanation.

[0125] The notification unit 215 notifies the user of the detection of the second object BJ2 ​​when the second detection unit 225 detects the second object BJ2. Otherwise, the notification unit 215 notifies the user of the detection of the second object BJ2 ​​when the second detection unit 225 does not detect the second object BJ2.

[0126] Figure 7 It is a diagram showing an example of a method for detecting the first object BJ1 and the second object BJ2.

[0127] exist Figure 7 , the first depth image DP1 , the second depth image DP2 , and the third depth image DP3 are shown from top to bottom.

[0128] In addition, Figure 7 , a scale image CD is recorded corresponding to each of the first depth image DP1, the second depth image DP2, and the third depth image DP3, indicating the relationship between the depth value (Z coordinate) and the brightness B of the pixel in the depth image DP. The depth value (Z coordinate) indicates the distance from the depth camera 11.

[0129] As shown in the scale image CD, the longer the distance from the depth camera 11 , the higher the brightness B of the pixels in the depth image DP. In other words, the closer the distance from the depth camera 11 , the denser the image in the depth image DP.

[0130] In the first depth image DP1 , a ground image DP11 having a first brightness value B1 corresponding to the distance from the depth camera 11 to the ground FL is displayed.

[0131] In the second depth image DP2 , in addition to the image of the first brightness value B1 , a first image DP21 and a second image DP22 corresponding to the first object BJ1 are displayed.

[0132] The first image DP21 is an image corresponding to the cuboid BJ11 and is displayed at a second brightness value B2 that is lower than the first brightness value B1.

[0133] The second image DP22 is an image corresponding to the wall BJ12 and is displayed at a third brightness value B3 that is lower than the first brightness value B1.

[0134] The cuboid BJ11 is closer to the depth camera 11 than the wall BJ12 , so the second brightness value B2 is lower than the third brightness value B3 .

[0135] The first detection unit 223 detects the first object BJ1 within the first area AR1 based on the first depth image DP1 and the second depth image DP2. In other words, the first detection unit 223 detects the first object BJ1 by extracting an image having a luminance value different from the first luminance value B1 of the ground image DP11 from the image included in the second depth image DP2. For example, the first detection unit 223 detects the first object BJ1 by extracting an image having a luminance value lower than the first luminance value B1 of the ground image DP11 from the image included in the second depth image DP2.

[0136] In this way, the first detection unit 223 can detect the first image DP21 corresponding to the cuboid BJ11 and the second image DP22 corresponding to the wall BJ12.

[0137] In the third depth image DP3 , in addition to the ground image DP11 , the first image DP21 , and the second image DP22 , a third image DP31 is also displayed.

[0138] The third image DP31 is an image corresponding to the second object BJ2 ​​representing a human body, and is displayed with a fourth brightness value B4. Since the second object BJ2 ​​is closer to the depth camera 11 than the first object BJ1, the fourth brightness value B4 is lower than the third brightness value B3.

[0139] The second detection unit 225 detects the second object BJ2 ​​within the first area AR1 based on the second depth image DP2 and the third depth image DP3. In other words, the second detection unit 225 detects the second object BJ2 ​​by ​​extracting an image having a luminance value different from the first luminance value B1 of the ground image DP11, the second luminance value B2 of the first image DP21 corresponding to the first object BJ1, and the third luminance value B3 of the second image DP22 from the image included in the third depth image DP3. For example, the second detection unit 225 detects the second object BJ2 ​​by ​​extracting an image having a luminance value lower than the third luminance value B3 of the second image DP22 from the image included in the third depth image DP3.

[0140] In this way, the second detection unit 225 can detect the third image DP31 corresponding to the second object BJ2.

[0141] When the second detection unit 225 detects the second object BJ2, the notification unit 215 notifies the user of the detection of the second object BJ2. The notification unit 215 notifies the user of the detection of the second object BJ2 ​​by, for example, causing the projector 30 to display a projection image indicating the detection of the second object BJ2 ​​on the floor FL.

[0142] When the second detection unit 225 does not detect the second object BJ2, the notification unit 215 notifies the user that the second object BJ2 ​​has not been detected. The notification unit 215 notifies the user that the second object BJ2 ​​has not been detected by, for example, causing the projector 30 to display a projection image on the floor FL indicating that the second object BJ2 ​​has not been detected.

[0143] Figure 8 and Figure 9 This is a flowchart showing an example of processing of the personal computer 20 .

[0144] First, if Figure 8 As shown, in step S101 , the user arranges a reflector RF and a marker MK in the second area AR2 .

[0145] Next, in step S103 , the position identifying unit 216 acquires the RGB image CP from the RGB camera 12 .

[0146] Next, in step S105 , the position identifying unit 216 identifies the position corresponding to the second area AR2 in the RGB image CP.

[0147] Next, in step S107, the ratio determination unit 214 calculates the first length L1. The first length L1 indicates the length of the first area AR1 in the X-axis direction.

[0148] Next, in step S109, the ratio determination unit 214 calculates the second length L2. The second length L2 indicates the length of the second area AR2 in the X-axis direction.

[0149] Next, in step S111 , the ratio determination unit 214 calculates the ratio RT.

[0150] Next, in step S113 , the ratio determination unit 214 determines whether the ratio RT is equal to or less than the threshold value TH.

[0151] When the ratio determination unit 214 determines that the ratio RT is equal to or smaller than the threshold value TH, that is, when the answer is “YES” in step S113 , the process proceeds to step S115 .

[0152] Then, in step S115, the notification unit 215 notifies the user to install the reflector RF. Then, the process returns to step S101.

[0153] If the ratio determination unit 214 determines that the ratio RT is not equal to or smaller than the threshold value TH, that is, if the answer is "No" in step S113, the process proceeds to step S114. Figure 9 Step S117 is shown.

[0154] Then, if Figure 9 As shown, in step S117 , the coordinate acquisition unit 211 acquires the three-dimensional coordinate values ​​of two points in the second area AR2 based on the depth image DP.

[0155] Next, in step S119 , the plane calculation unit 212 calculates the position of the plane PL including the second area AR2 based on the three-dimensional coordinate values ​​of two points in the second area AR2 .

[0156] Next, in step S121 , the shape determination unit 213 determines the position of the plane PL as the three-dimensional shape of the first area AR1 .

[0157] Next, in step S123 , the first acquisition unit 221 acquires first distance information JL1 indicating the three-dimensional shape of the first area AR1 based on the position of the plane PL.

[0158] Next, in step S125, the second acquisition unit 222 acquires second distance information JL2 representing the three-dimensional shape of the first area AR1 based on the depth image DP. The second distance information JL2 corresponds to the second depth image DP2 generated by the depth camera 11 when the first object BJ1 is placed on the ground FL.

[0159] Next, in step S127, the first detection unit 223 detects the first object BJ1 in the first area AR1 based on the first distance information JL1 and the second distance information JL2. Specifically, the first detection unit 223 detects the first object BJ1 in the first area AR1 based on the first depth image DP1 and the second depth image DP2.

[0160] Next, in step S129, the third acquisition unit 224 acquires third distance information JL3 representing the three-dimensional shape within the first area based on the depth image DP. The third distance information JL3 corresponds to the third depth image DP3 generated by the depth camera 11 when the first object BJ1 and the second object BJ2 ​​are positioned on the ground surface FL.

[0161] Next, in step S131 , the second detection unit 225 determines whether the second object BJ2 ​​in the first area AR1 is detected based on the second distance information JL2 and the third distance information JL3 .

[0162] If the second detection unit 225 determines that the second object BJ2 ​​has not been detected, that is, if NO in step S131 , the process proceeds to step S135 .

[0163] Then, in step S135, the notification unit 215 notifies the user that the second object BJ2 ​​has not been detected. The notification unit 215 notifies the user that the second object BJ2 ​​has not been detected by, for example, causing the projector 30 to display a projection image on the floor FL indicating that the second object BJ2 ​​has not been detected. The process then proceeds to step S139.

[0164] If the second detection unit 225 determines that the second object BJ2 ​​has been detected, that is, if YES in step S131 , the process proceeds to step S137 .

[0165] Then, in step S137, the notification unit 215 notifies the user that the second object BJ2 ​​has been detected. The notification unit 215 notifies the user of the detection of the second object BJ2 ​​by, for example, causing the projector 30 to display a projection image indicating the detection of the second object BJ2 ​​on the floor FL. The process then proceeds to step S139.

[0166] Next, in step S139, the control unit 21 determines whether to end the detection of the second object BJ2. The control unit 21 receives an operation from the user, for example, and determines whether to end the detection of the second object BJ2 ​​based on the received operation.

[0167] If the control unit 21 determines that the detection of the second object BJ2 ​​is not to be completed, that is, if the answer is "No" in step S139, the process returns to step S129. If the control unit 21 determines that the detection of the second object BJ2 ​​is to be completed, that is, if the answer is "Yes" in step S139, the process is then completed.

[0168] As reference Figure 8 as well as Figure 9 As described above, the coordinate acquisition unit 211 acquires the three-dimensional coordinate values ​​of two points in the second area AR2 based on the depth image DP. The plane calculation unit 212 calculates the position of the plane PL that includes the second area AR2 based on the three-dimensional coordinate values ​​of the two points in the second area AR2. The shape determination unit 213 then determines the position of the plane PL as the three-dimensional shape of the first area AR1. Therefore, if the three-dimensional coordinate values ​​of the two points in the second area AR2 can be acquired, the three-dimensional coordinate values ​​of the first area AR1 can be acquired with sufficient accuracy.

[0169] Furthermore, since the reflector RF is arranged in the second area AR2 , the three-dimensional coordinate values ​​of two points in the second area AR2 can be accurately acquired.

[0170] Furthermore, since the marker MK is arranged in the second area AR2 , the position of the second area AR2 in the RGB image CP can be easily identified.

[0171] Next, a detection system 1 according to a second embodiment will be described. The basic configuration of the detection system 1 according to the second embodiment is the same as that of the detection system 1 according to the first embodiment, and therefore detailed description thereof will be omitted.

[0172] While the control unit 21 of the personal computer 20 of the detection system 1 of the first embodiment includes the second detection unit 225 , the control unit 21 of the personal computer 20 of the detection system 1 of the second embodiment differs in that it includes a second detection unit 226 instead of the second detection unit 225 .

[0173] Figure 10 This is a diagram showing an example of the configuration of a personal computer 20 according to the second embodiment.

[0174] The second detection unit 226 detects the second object BJ2 ​​within the first area AR1 based on the first distance information JL1 , the second distance information JL2 , and the third distance information JL3 .

[0175] Reference Figure 7 , the more specific processing content of the second detection unit 226 is explained.

[0176] First, the second detection unit 226 detects the first object BJ1 and the second object BJ2 ​​within the first area AR1 based on the first depth image DP1 and the third depth image DP3. In other words, the second detection unit 226 detects the first object BJ1 and the second object BJ2 ​​by ​​extracting an image having a luminance value different from the first luminance value B1 of the ground image DP11 from the image included in the third depth image DP3. For example, the second detection unit 226 detects the first object BJ1 and the second object BJ2 ​​by ​​extracting an image having a luminance value lower than the first luminance value B1 of the ground image DP11 from the image included in the third depth image DP3.

[0177] Next, the second detection unit 226 detects a second object BJ2 ​​different from the first object BJ1 detected by the first detection unit 223 within the first area AR1 based on the second depth image DP2 and the third depth image DP3 .

[0178] In other words, the second detection unit 226 extracts, from the images included in the third depth image DP3, the brightness values ​​of the images corresponding to the first object BJ1 and the second object BJ2, respectively, detected by the second detection unit 226 based on the first depth image DP1 and the third depth image DP3. Specifically, the second detection unit 226 extracts the second brightness value B2 of the first image DP21 corresponding to the first object BJ1, the third brightness value B3 of the second image DP22 corresponding to the first object BJ1, and the fourth brightness value B4 of the third image DP31 corresponding to the second object BJ2.

[0179] Then, the second detection unit 226 extracts, from the image included in the second depth image DP2, the brightness value of the image corresponding to the first object BJ1 detected by the first detection unit 223 based on the first depth image DP1 and the second depth image DP2. Specifically, the second detection unit 226 extracts the second brightness value B2 of the first image DP21 corresponding to the first object BJ1 within the first area AR1, and the third brightness value B3 of the second image DP22 corresponding to the first object BJ1.

[0180] Afterwards, the second brightness value B2, the third brightness value B3 and the fourth brightness value B4 in the third depth image DP3 are compared with the second brightness value B2 and the third brightness value B3 in the second depth image DP2, and the different brightness values ​​between the second depth image DP2 and the third depth image DP3 are extracted, thereby detecting the second object BJ2 ​​corresponding to the fourth brightness value B4.

[0181] In this way, the second detection unit 226 can detect the second object BJ2 ​​within the first area AR1 based on the first depth image DP1 , the second depth image DP2 , and the third depth image DP3 .

[0182] Figure 11is a flowchart indicating an example of the process of the personal computer 20 included in the detection system 1 of the second embodiment. The processes of steps S117 to S129 are the same as the processes performed by the detection system 1 of the first embodiment, and thus the description thereof is omitted.

[0183] After the personal computer 20 performs step S129, it performs step S231. In step S231, the second detection section 226 determines whether the second object BJ2 within the first region AR1 is detected, based on the first distance information JL1, the second distance information JL2, and the third distance information JL3.

[0184] In a case where the second detection section 226 determines that the second object BJ2 is not detected, that is, in a case where the process of step S231 is "No", the process proceeds to step S135. In a case where the second detection section 226 determines that the second object BJ2 is detected, that is, in a case where the process of step S231 is "Yes", the process proceeds to step S137.

[0185] The processes after steps S135 and S137 are the same as the processes performed by the detection system 1 of the first embodiment, and thus the description thereof is omitted.

[0186] As described above with reference to Figures 1 to 11 As described above with reference to

[0187] That is, based on the depth image DP, the three-dimensional coordinate values of the two points of the second region AR2 are obtained, and based on the three-dimensional coordinate values of the two points of the second region AR2, the position of the plane PL including the second region AR2 is calculated. Then, the position of the plane PL is determined as the three-dimensional shape of the first region AR1.

[0188] Therefore, in a case where the three-dimensional coordinate values of the two points of the second region AR2 can be obtained, the three-dimensional coordinate values of the first region AR1 can be obtained with sufficient accuracy.

[0189] In addition, the second region AR2 is a region in which the reflector RF that reflects infrared rays is provided.

[0190] Therefore, the three-dimensional coordinate values of the 2 points of the second region AR2 can be accurately obtained based on the depth image DP. Therefore, the three-dimensional coordinate values of the first region AR1 can be obtained with sufficient accuracy.

[0191] Further, the detection method includes determining whether the ratio RT of the size of the second region AR2 to the size of the first region AR1 is equal to or less than a threshold value TH, and in the case where the ratio RT is equal to or less than the threshold value TH, giving a notification to urge the user to perform the setting of the reflector RF.

[0192] That is, in the case where the ratio RT of the size of the second region AR2 to the size of the first region AR1 is equal to or less than the threshold value TH, a notification is given to urge the user to perform the setting of the reflector RF.

[0193] Therefore, the second region AR2 can be set to an appropriate size. Therefore, the three-dimensional coordinate values of the first region AR1 can be obtained with sufficient accuracy.

[0194] Further, the detection method includes determining whether the ratio RT of the size of the second region AR2 to the size of the first region AR1 is equal to or less than a threshold value TH, and in the case where the ratio RT is equal to or less than the threshold value TH, giving a notification to the user that there is a possibility that the detection accuracy of the three-dimensional shape of the first region AR1 is low.

[0195] That is, in the case where the ratio RT of the size of the second region AR2 to the size of the first region AR1 is equal to or less than the threshold value TH, a notification is given to the user that there is a possibility that the detection accuracy of the three-dimensional shape of the first region AR1 is low.

[0196] Therefore, the user can recognize that there is a possibility that the detection accuracy of the three-dimensional shape of the first region AR1 is low. Therefore, the convenience of the user can be improved.

[0197] For example, in the case where it is not problematic even if the detection accuracy of the three-dimensional shape of the first region AR1 is low, the user can use the detected three-dimensional shape of the first region AR1. Further, for example, in the case where it is not preferable that the detection accuracy of the three-dimensional shape of the first region AR1 is low, the user can improve the detection accuracy of the three-dimensional shape of the first region AR1 by increasing the size of the second region AR2.

[0198] Further, the detection method includes accepting an operation of designating a region corresponding to the second region AR2 in the depth image DP of the first region AR1 obtained by the infrared projector 13 and the depth camera 11.

[0199] Therefore, in the depth image DP of the first region AR1, the operation of designating the region corresponding to the second region AR2 is accepted, and thus the second region AR2 can be easily determined without providing the marker MK. Therefore, the convenience of the user can be improved.

[0200] Further, the detection method includes: the RGB camera 12 capturing the first region AR1, and acquiring the RGB image CP from the RGB camera 12; and determining a position corresponding to the second region AR2 in the RGB image CP.

[0201] Therefore, in the RGB image CP, the position corresponding to the second region AR2 is determined, and thus the position corresponding to the second region AR2 can be accurately determined. Therefore, the detection accuracy of the three-dimensional shape of the first region AR1 can be improved.

[0202] Further, the detection method includes: acquiring first distance information JL1 representing the three-dimensional shape of the first region AR1 based on the position of the plane PL; after the first distance information JL1 is acquired, acquiring second distance information JL2 representing the three-dimensional shape within the first region AR1 based on a depth image DP of the first region AR1 obtained by the infrared projector 13 and the depth camera 11; and detecting a first object BJ1 within the first region AR1 based on the first distance information JL1 and the second distance information JL2.

[0203] That is, the first object BJ1 within the first region AR1 is detected based on the first distance information JL1 representing the three-dimensional shape of the first region AR1 and the second distance information JL2 representing the three-dimensional shape within the first region AR1 after the first distance information JL1 is acquired. For example, as described with reference to Figure 7 The first object BJ1 within the first region AR1 is detected based on the first depth image DP1 and the second depth image DP2. The first depth image DP1 corresponds to the first distance information JL1. The second depth image DP2 corresponds to the second distance information JL2.

[0204] Therefore, the first object BJ1 within the first region AR1 can be accurately detected.

[0205] Further, the detection method further includes: after the second distance information JL2 is acquired, acquiring third distance information JL3 representing the three-dimensional shape within the first region AR1 based on a depth image DP of the first region AR1 obtained by the infrared projector 13 and the depth camera 11; and detecting a second object BJ2 within the first region AR1 different from the first object BJ1 based on the second distance information JL2 and the third distance information JL3.

[0206] That is, the second object BJ2 within the first region AR1 different from the first object BJ1 is detected based on the second distance information JL2 and the third distance information JL3 representing the three-dimensional shape within the first region AR1 after the second distance information JL2 is acquired. For example, as described with reference to Figure 7As described above, the second object BJ2 ​​in the first area AR1 is detected based on the second depth image DP2 and the third depth image DP3. The second depth image DP2 corresponds to the second distance information JL2. The third depth image DP3 corresponds to the third distance information JL3.

[0207] Therefore, the second object BJ2 ​​within the first area AR1 can be accurately detected.

[0208] Furthermore, in addition to the second distance information JL2 and the third distance information JL3 indicating the three-dimensional shape in the first area AR1 after obtaining the second distance information JL2, a second object BJ2 ​​different from the first object BJ1 in the first area AR1 may be detected based on the first distance information JL1.

[0209] The detection system 1 of this embodiment includes an infrared projector 13, a depth camera 11 and a personal computer 20, which detects the three-dimensional shape of a first area AR1, wherein the first area AR1 includes a second area AR2 which is smaller than the first area AR1, and the personal computer 20 performs the following processing: based on the depth image DP of the first area AR1 obtained using the infrared projector 13 and the depth camera 11, obtains the three-dimensional coordinate values ​​of at least two points of the second area AR2; based on the three-dimensional coordinate values ​​of the two points, calculates the position of the plane PL including the second area AR2; and determines the position of the plane PL as the three-dimensional shape of the first area AR1.

[0210] Therefore, the detection system 1 of the present embodiment achieves the same effects as the detection method of the present embodiment.

[0211] The above-described embodiment is a preferred embodiment, but the present invention is not limited to the above-described embodiment, and various modifications can be implemented without departing from the spirit and scope of the present invention.

[0212] In this embodiment, the "distance measuring sensor" and the imaging device that generates the captured image are described as being composed of an RGB-D camera, but the present invention is not limited thereto. For example, the "distance measuring sensor" and the imaging device that generates the captured image may be configured as separate bodies.

[0213] In this embodiment, the case where the "information processing device" is the personal computer 20 is described, but the present invention is not limited thereto. The "information processing device" may also be a tablet terminal, a smartphone, or the like.

[0214] In this embodiment, the detection system 1 is described as including the detection device 10, the personal computer 20, and the projector 30. However, the present invention is not limited thereto. For example, the detection system 1 may be composed of the detection device 10 and the personal computer 20. In other words, the detection system 1 may not include the projector 30.

[0215] In this embodiment, the first area AR1 is described as being included in the floor surface FL, but the present invention is not limited thereto. The first area AR1 may be included in a planar component, structure, or the like. For example, the first area AR1 may be included in a wall surface. Alternatively, the first area AR1 may be included in a ceiling surface.

[0216] In this embodiment, the personal computer 20 is connected to the detection device 10 and the projector 30 in a manner that allows wired communication, but the present invention is not limited thereto. The personal computer 20 may also be connected to the detection device 10 and the projector 30 in a manner that allows wireless communication, such as Bluetooth (registered trademark) or Wi-Fi (registered trademark).

[0217] In this embodiment, the control unit 21 of the personal computer 20 is described as having a coordinate acquisition unit 211, a plane calculation unit 212, a shape determination unit 213, a ratio determination unit 214, a notification unit 215, a position determination unit 216, a first acquisition unit 221, a second acquisition unit 222, a first detection unit 223, a third acquisition unit 224 and a second detection unit 225, but is not limited to this.

[0218] For example, the control unit (not shown) of the projector 30 may include a coordinate acquisition unit 211, a plane calculation unit 212, a shape determination unit 213, a ratio determination unit 214, a notification unit 215, a position determination unit 216, a first acquisition unit 221, a second acquisition unit 222, a first detection unit 223, a third acquisition unit 224, and a second detection unit 225. In this case, the detection system 1 does not need to include the personal computer 20.

[0219] in addition, Figure 3 The functional units shown represent functional structures, and the specific implementation method is not particularly limited. That is, it is not necessary to install hardware corresponding to each functional unit. Alternatively, a single processor executing a program can implement the functions of multiple functional units. Furthermore, in the above-described embodiment, a portion of the functions implemented by software can be implemented by hardware, and vice versa. Furthermore, the specific details of the other components of personal computer 20 can be arbitrarily modified without departing from the main purpose.

[0220] in addition, Figure 8 as well as Figure 9 The processing units of the flowchart shown are divided according to the main processing contents in order to make the processing of the personal computer 20 easier to understand. Figure 8 and Figure 9The division method and name of the processing units shown in each flowchart are not limited. The processing unit can be divided into more processing units according to the processing content, or one processing unit can be divided to include more processing. In addition, the processing order of the above flowchart is not limited to the example shown in the figure.

[0221] The detection method of the detection system 1 can be implemented by causing the processor 21A of the personal computer 20 to execute a control program PGM corresponding to the detection method of the detection system 1. Alternatively, the control program PGM may be pre-recorded on a computer-readable recording medium. The recording medium may be a magnetic or optical recording medium or a semiconductor memory device.

[0222] Specifically, examples include removable or fixed recording media such as floppy disks, HDDs, CD-ROMs (Compact Disk Read Only Memory), DVDs, Blu-ray discs (registered trademark), magneto-optical discs, flash memories, and card-type recording media. Alternatively, the recording medium may be a non-volatile storage device such as a RAM, ROM, or HDD, which is an internal storage device included in the image processing apparatus.

[0223] Alternatively, the control program PGM corresponding to the detection method of the detection system 1 may be stored in advance in a server device or the like, and the control program PGM may be downloaded from the server device to the personal computer 20 to implement the detection method of the detection system 1 .

[0224] In addition, in the present embodiment, the case where the user sets the reflector RF in the second area AR2 is described, but it is not limited to this. The second area AR2 may also be a plane that reflects infrared rays and an area that can obtain three-dimensional coordinate values ​​based on the depth image DP obtained by the depth camera 11. For example, the first area AR1 may be the ground, and the area in the first area AR1 formed by a material that fully reflects infrared rays may be used as the second area AR2. In addition, the area in the first area AR1 that cannot fully reflect infrared rays to the depth camera 11 due to the influence of the relative angle between the ground as the first area AR1 and the depth camera 11 may not be used as the second area AR2, but only the area that can fully reflect infrared rays to the depth camera 11 may be used as the second area AR2. In addition, for example, as Figure 5 As shown in FIG. 1 , when the first object BJ1 is included in the first area AR1 , an area of ​​a plane reflecting infrared rays without the first object BJ1 may be defined as the second area AR2 .

Claims

1. A detection method comprising the following steps: Based on a measurement value obtained by measuring a first area using an infrared distance measuring sensor, obtaining three-dimensional coordinate values ​​of at least two points in a second area that is included in the first area and is smaller than the first area; Calculating a position of a plane including the second area based on the three-dimensional coordinate values ​​of the at least two points; as well as determining the position of the plane as the three-dimensional shape of the first region, The second area is an area where a reflector that reflects infrared rays is provided. The detection method further includes the following processing: determining whether a ratio of a size of the second region to a size of the first region is equal to or smaller than a threshold; and When the ratio is equal to or less than the threshold value, notification is performed to urge the user to install the reflector.

2. The detection method according to claim 1, wherein The detection method includes a process of accepting an operation of specifying a measurement value corresponding to the second area among the measurement values ​​of the first area by the distance measuring sensor.

3. A detection method comprising the following steps: Based on a measurement value obtained by measuring a first area using an infrared distance measuring sensor, obtaining three-dimensional coordinate values ​​of at least two points in a second area that is included in the first area and is smaller than the first area; Calculating the position of a plane including the second area based on the three-dimensional coordinate values ​​of the at least two points; and determining the position of the plane as the three-dimensional shape of the first region, The detection method further includes the following processing: determining whether a ratio of a size of the second region to a size of the first region is equal to or smaller than a threshold; and When the ratio is equal to or smaller than the threshold, the user is informed that there is a possibility that the detection accuracy of the three-dimensional shape of the first area is low.

4. The detection method according to claim 3, wherein The detection method includes a process of accepting an operation of specifying a measurement value corresponding to the second area among the measurement values ​​of the first area by the distance measuring sensor.

5. A detection method comprising the following steps: Based on a measurement value obtained by measuring a first area using an infrared distance measuring sensor, obtaining three-dimensional coordinate values ​​of at least two points in a second area that is included in the first area and is smaller than the first area; Calculating the position of a plane including the second area based on the three-dimensional coordinate values ​​of the at least two points; and determining the position of the plane as the three-dimensional shape of the first region, The detection method further includes the following processing: Obtaining an image obtained by photographing the first area; and A position corresponding to the second area is determined in the captured image.

6. The detection method according to any one of claims 1 to 5, wherein The detection method includes the following processing: obtaining first distance information representing a three-dimensional shape of the first area based on the position of the plane; After obtaining the first distance information, obtaining second distance information representing a three-dimensional shape within the first area based on a measurement value of the distance measuring sensor; as well as A first object within the first area is detected based on the first distance information and the second distance information.

7. The detection method according to claim 6, wherein The detection method includes the following processing: After obtaining the second distance information, obtaining third distance information representing a three-dimensional shape within the first area based on a measurement value of the distance measuring sensor; and A second object different from the first object is detected within the first area based on the second distance information and the third distance information.

8. A detection system comprising: Infrared distance measuring sensor; and information processing device, The information processing device performs the following processing: Obtaining three-dimensional coordinate values ​​of at least two points in a second area that is included in the first area and is smaller than the first area based on measurement values ​​obtained by the distance measuring sensor on the first area; Calculating a position of a plane including the second area based on the three-dimensional coordinate values ​​of the at least two points; as well as determining the position of the plane as the three-dimensional shape of the first region, The second area is an area where a reflector that reflects infrared rays is provided. The information processing device further performs the following processing: determining whether a ratio of a size of the second region to a size of the first region is equal to or smaller than a threshold; and When the ratio is equal to or less than the threshold value, notification is performed to urge the user to install the reflector.

9. A detection system comprising: Infrared distance measuring sensor; and information processing device, The information processing device performs the following processing: Obtaining three-dimensional coordinate values ​​of at least two points in a second area that is included in the first area and is smaller than the first area based on measurement values ​​obtained by the distance measuring sensor on the first area; Calculating the position of a plane including the second area based on the three-dimensional coordinate values ​​of the at least two points; and determining the position of the plane as the three-dimensional shape of the first region, The information processing device further performs the following processing: determining whether a ratio of a size of the second region to a size of the first region is equal to or smaller than a threshold; and When the ratio is equal to or smaller than the threshold, the user is informed that there is a possibility that the detection accuracy of the three-dimensional shape of the first area is low.

10. A detection system comprising: Infrared distance measuring sensor; and information processing device, The information processing device performs the following processing: Obtaining three-dimensional coordinate values ​​of at least two points in a second area that is included in the first area and is smaller than the first area based on measurement values ​​obtained by the distance measuring sensor on the first area; Calculating the position of a plane including the second area based on the three-dimensional coordinate values ​​of the at least two points; and determining the position of the plane as the three-dimensional shape of the first region, The information processing device further performs the following processing: Obtaining an image obtained by photographing the first area; and A position corresponding to the second area is determined in the captured image.

Images