Imaging device

Abstract

The present invention makes it possible to execute monitoring of the surroundings of an object with higher accuracy by using an imaging device that generates distance image data using a sub-range scheme. With respect to an object (OB) present across adjacent first and second distance zones, there are derived a first position at a near-side position in the first distance zone, a second position at a boundary position between the first and second distance zones, and a third position indicating a position at a deep-side position in the second distance zone. Position data pertaining to the object (OB) is generated on the basis of position information for the second position on a straight line connecting the first position and the third position.

Description

Technical Field

[0001] The present disclosure relates to an imaging device that generates distance image data using a sub-range method.

Background Art

[0002] Imaging devices and image processing devices are used for obstacle monitoring.

[0003] In Patent Document 1, a technique is disclosed in which a laser scanner or the like is installed on a moving body such as a railway vehicle or a maintenance vehicle, three-dimensional point cloud data representing the shape of a ground object along a railway is acquired while moving on a railway line, and the position of a reference structure is calculated by processing this three-dimensional point cloud data. In Patent Document 1, with respect to the absolute position of the three-dimensional point cloud, the detection accuracy is improved based on summary statistics.

[0004] In Patent Document 2, a technique for performing weighted interpolation between a plurality of zones and calculating an average distance with respect to distance interpolation is disclosed.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0006] However, in the method of performing object detection using an aggregate of point cloud data as in Patent Document 1, there is a problem that it is difficult to discriminate the shape of a distant object. Further, in the method of simply weighting pixel data to calculate the distance as in Patent Document 2, the measurement accuracy of the distance is insufficient.

[0007] The present disclosure has been made in view of such a point, and an object thereof is to enable more accurate peripheral monitoring of an object by using an imaging device that generates distance image data using a sub-range method.

Means for Solving the Problems

[0008] An imaging device according to an aspect of the present disclosure includes a detection unit having a distance measuring camera, and divides a target space into a plurality of distance zones based on the distance from a reference point in the depth direction, and generates distance image data representing each of the distance zones from the output of the detection unit. The control unit includes an input unit that inputs setting information of the distance zone, including the number of the distance zones and the front side position and the back side position of the distance zone, and a distance image generation unit that generates the distance image data from the output of the detection unit using the setting information of the distance zone input to the input unit. The control unit further includes an extraction unit that extracts an object existing across adjacent first and second distance zones, and a calculation unit that calculates object position data for estimating the position and shape of the object extracted by the extraction unit based on the distance image data. The calculation unit obtains a first position indicating the position of the object at the front side position of the first distance zone, a second position indicating the position of the object at the boundary position between the first and second distance zones, and a third position indicating the position of the object at the back side position of the second distance zone in a plane including the depth direction and a horizontal direction perpendicular to the depth direction, and generates the object position data based on the position information of the second position with respect to a straight line connecting the first position and the third position.

Advantages of the Invention

[0009] According to the present disclosure, it is possible to accurately perform peripheral monitoring of an object by using an imaging device that generates distance image data using a sub-range method.

Brief Description of the Drawings

[0010]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Mode for Carrying Out the Invention

[0011] (Overview) The imaging device according to the first aspect of the present disclosure includes a detection unit having a distance measurement camera, and a control unit that divides a target space into a plurality of distance zones based on the distance from a reference point in the depth direction, and generates distance image data representing each of the distance zones from the output of the detection unit. The control unit includes an input unit that inputs setting information of the distance zone, including the number of the distance zones and the front side position and the back side position of the distance zone, and a distance image generation unit that generates the distance image data from the output of the detection unit using the setting information of the distance zone input to the input unit. The control unit further includes an extraction unit that extracts an object existing across adjacent first and second distance zones, and a calculation unit that calculates object position data for estimating the position and shape of the object extracted by the extraction unit based on the distance image data. The calculation unit obtains a first position indicating the position of the object at the front side position of the first distance zone, a second position indicating the position of the object at the boundary position between the first and second distance zones, and a third position indicating the position of the object at the back side position of the second distance zone in a plane including the depth direction and a horizontal direction perpendicular to the depth direction, and generates the object position data based on the position information of the second position with respect to a straight line connecting the first position and the third position.

[0012] Accordingly, for an object existing across adjacent first and second distance zones, a first position at the front side position of the first distance zone, a second position at the boundary position between the first and second distance zones, and a third position indicating the position at the back side position of the second distance zone are obtained. Then, object position data is generated based on the position information of the second position with respect to the straight line connecting the first position and the third position. Thereby, even when the object is curved, its position and shape can be represented more accurately. Therefore, for an object existing across adjacent first and second distance zones, its position data can be generated with higher accuracy.

[0013] The imaging device according to the second aspect of the present disclosure includes a detection unit having a distance measurement camera, and a control unit that divides a target space into a plurality of distance zones based on the distance from a reference point in the depth direction, and generates distance image data representing each of the distance zones from the output of the detection unit. The control unit includes an input unit that inputs setting information of the distance zone, including the number of the distance zones and the front side position and the rear side position of the distance zone, and a distance image generation unit that generates the distance image data from the output of the detection unit using the setting information of the distance zone input to the input unit. The control unit further includes an extraction unit that extracts an object existing across adjacent first and second distance zones, and a calculation unit that calculates object position data for estimating the position and shape of the object extracted by the extraction unit based on the distance image data. The calculation unit sets a third distance zone including a boundary position between the first and second distance zones, and obtains a first position indicating the position of the object at the front side position of the first distance zone, a second position indicating the position of the object at the boundary position between the first and second distance zones, a third position indicating the position of the object at the rear side position of the second distance zone, a fourth position indicating the position of the object at the front side position of the third distance zone, and a fifth position indicating the position of the object at the rear side position of the third distance zone in a plane including the depth direction and a horizontal direction perpendicular to the depth direction. The calculation unit generates object position data based on the position information of the fourth position with respect to the straight line connecting the first position and the second position, and the position information of the fifth position with respect to the straight line connecting the second position and the third position.

[0014] As a result, a third distance zone including a boundary position between the first and second distance zones is set. Then, for an object existing across adjacent first and second distance zones, a first position at a front side position of the first distance zone, a second position at a boundary position between the first and second distance zones, and a third position indicating a position at a rear side position of the second distance zone are obtained. In addition, a fourth position at a front side position of the third distance zone and a fifth position at a rear side position of the third distance zone are obtained. Then, object position data is generated based on the position information of the fourth position with respect to the straight line connecting the first position and the second position, and the position information of the fifth position with respect to the straight line connecting the second position and the third position. Thereby, even when the object is curved, its position and shape can be represented more accurately. Therefore, for an object existing across adjacent first and second distance zones, its position data can be generated with higher accuracy.

[0015] In the imaging device according to the first or second aspect, the imaging device may include a monitoring unit that sets a monitoring area based on a monitoring area function input to the input unit with reference to the object position data.

[0016] As a result, for an object existing across adjacent first and second distance zones, a desired monitoring area can be set around it.

[0017] Furthermore, the input unit can input a threshold value for alarm reporting due to the presence of an obstacle in the monitoring area, and when the imaging device recognizes an object that can be an obstacle in the target space, for a representative point of the object in the vertical plane, a probability value is calculated by integrating the monitoring area function in the depth direction, and a specifying unit that compares the probability value with the threshold value input to the input unit to determine the presence or absence of alarm reporting may be provided.

[0018] As a result, when an object that can be an obstacle is recognized, the presence or absence of alarm reporting can be accurately determined.

[0019] Alternatively, the input unit may be configured to be able to input a threshold value for alarm reporting due to the presence of an obstacle in the monitoring area, and when the imaging device recognizes an object that can be an obstacle in the target space, for the area of the object in the vertical plane, a probability value is calculated by integrating the monitoring area function in the depth direction, and a specifying unit is provided that compares the probability value with the threshold value input to the input unit to determine the presence or absence of alarm reporting.

[0020] Thereby, when an object that can be an obstacle is recognized, it is possible to accurately determine the presence or absence of alarm reporting.

[0021] Hereinafter, embodiments will be specifically described with reference to the drawings.

[0022] Note that each of the embodiments described below shows comprehensive or specific examples. The numerical values, shapes, materials, components, arrangement positions and connection forms of the components, steps, the order of steps, etc. shown in the following embodiments are merely examples and are not intended to limit the present disclosure. In addition, among the components in the following embodiments, components not described in the independent claims indicating the most general concept are described as optional components.

[0023] (Embodiment) FIG. 1 is a block diagram showing the configuration of an imaging device according to an embodiment. The imaging device 1 shown in FIG. 1 uses a so-called sub-range method. That is, the space to be imaged is divided into a plurality of distance zones based on the distance from a reference point in the depth direction, and distance image data representing each distance zone is output.

[0024] The detection unit 10 acquires distance image data based on a so-called TOF (Time Of Flight) method, and includes a distance measurement camera 11 using the TOF method and a light source 12 synchronized therewith.

[0025] The control unit 20 divides the target space into a plurality of distance zones based on the distance from a reference point in the depth direction, and generates distance image data representing each distance zone from the output of the detection unit 10. In the present embodiment, the control unit 20 includes an input unit 21, a distance image generation unit 22, an extraction unit 23, a calculation unit 24, a monitoring unit 25, a specification unit 26, an output unit 27, and a storage unit 28.

[0026] The input unit 21 is configured to be able to input setting information of the distance zones, including the number of distance zones and the front-side position and back-side position of each distance zone. Also, it is configured to be able to input setting information of a monitoring area described later and a threshold value for alarm reporting due to the presence of an obstacle in the monitoring area. The distance image generation unit 22 generates distance image data representing each distance zone from the output of the detection unit 10 using the setting information of the distance zones input to the input unit 21.

[0027] The extraction unit 23 extracts an object existing across a plurality of adjacent distance zones from the distance image data. The objects extracted here are, for example, rails, station platforms, curbstones, walls, etc. In the present embodiment, a monitoring area such as a building limit can be set around the object extracted here. The calculation unit 24 calculates object position data, which is data obtained by estimating the position and shape of the object extracted by the extraction unit 23 using the distance image data. Note that the monitoring unit 25 and the specification unit 26 will be described later.

[0028] The output unit 27 outputs the distance image data, and information such as object position data, monitoring area, and alarm information to an external output device 3 such as a display or a printer. The storage unit 28 stores the distance image data, and information such as object position data, monitoring area, and alarm information. Also, the information such as object position data, monitoring area, and alarm information is also output to an external control device 2.

[0029] Here, the problems in the case of using the sub-range method will be described.

[0030] FIG. 2 shows a case where an object OB, such as a line or a guardrail, exists across a plurality of distance zones in the imaging space. The lower diagram of FIG. 2 illustrates the position and shape of the object OB shown in the upper diagram of FIG. 2 in a plane coordinate system consisting of the depth direction and the horizontal direction perpendicular to this depth direction. The depth positions z1 to z2 are the first distance zone, and the depth positions z2 to z3 are the second distance zone. The object OB exists across the first and second distance zones and is gently curved.

[0031] One distance zone is treated as a single distance. In the lower diagram of FIG. 2, the first distance zone is treated as a representative distance / z12, and the second distance zone is treated as a representative distance / z23. Here, / z12 may be selected, for example, as the intermediate distance between z1 and z2. The same applies to / z23. For this reason, regardless of its shape, the object OB existing across the first and second distance zones becomes the distance / z12 at the horizontal positions x1 to x2 and the distance / z23 at the horizontal positions x2 to x3. Thus, in the sub-range method, there is a problem that the accuracy of the position and shape of the object OB decreases.

[0032] In order to solve this problem, for example, a method of narrowing the width of each distance zone to increase the distance resolution can be considered. However, when the width of each distance zone is narrowed, if detection is to be performed from the vicinity to the far distance, the amount of data will become extremely large, and it is actually difficult to achieve.

[0033] Therefore, in this embodiment, the following method is used.

[0034] <The first method> As shown in FIG. 3, for adjacent first and second distance zones, the position of the object OB at the front-side position z1 of the first distance zone (the first position), the position of the object OB at the boundary position z2 between the first and second distance zones (the second position), and the position of the object OB at the back-side position z3 of the second distance zone (the third position) are obtained. Then, it is determined on which side in the horizontal direction the second position is with respect to the straight line (straight line 3) connecting the first position and the third position. That is, the position information of the second position with respect to the straight line 3 is obtained.

[0035] In the example of FIG. 3(a), the second position is on the left side with respect to the straight line 3. In this case, the straight line (straight line 1) connecting the first position and the second position, and the straight line (straight line 2) connecting the second position and the third position are also on the left side of the straight line 3. At this time, the positions between the first position - the second position - the third position are interpolated with a left-convex curve function to generate the position data of the object OB.

[0036] Also, in the example of FIG. 3(b), the second position is on the right side with respect to the straight line 3. In this case, the straight line (straight line 1) connecting the first position and the second position, and the straight line (straight line 2) connecting the second position and the third position are also on the right side of the straight line 3. At this time, the positions between the first position - the second position - the third position are interpolated with a right-convex curve function to generate the position data of the object OB.

[0037] <The second method> As shown in FIG. 4, a third distance zone including the boundary position z2 between the first and second distance zones is set. In the setting of FIG. 4, the third distance zone is set between the intermediate position / z12 of the first distance zone and the intermediate position / z23 of the second distance zone.

[0038] Then, similar to the first method, for adjacent first and second distance zones, the position of the object OB at the front-side position z1 of the first distance zone (the first position), the position of the object OB at the boundary position z2 between the first and second distance zones (the second position), and the position of the object OB at the back-side position z3 of the second distance zone (the third position) are obtained. In addition, the position of the object OB at the front-side position / z12 of the third distance zone (the fourth position) and the position of the object OB at the back-side position / z23 of the third distance zone (the fifth position) are obtained.

[0039] Then, it is determined on which side in the horizontal direction the fourth position is with respect to the straight line (straight line 1) connecting the first position and the second position. Also, it is determined on which side in the horizontal direction the fifth position is with respect to the straight line (straight line 2) connecting the second position and the third position. That is, the position information of the fourth position with respect to the straight line 1 and the position information of the fifth position with respect to the straight line 2 are obtained.

[0040] In the example of Fig. 4(a), the fourth position is on the left side with respect to the straight line 1. At this time, the position data of the object OB is generated by interpolating between the first position - the fourth position - the second position with a left-convex curve function. Also, the fifth position is on the left side with respect to the straight line 2. At this time, the position data of the object OB is generated by interpolating between the second position - the fifth position - the third position with a left-convex curve function.

[0041] Also, in the example of Fig. 4(b), the fourth position is on the left side with respect to the straight line 1. At this time, the position data of the object OB is generated by interpolating between the first position - the fourth position - the second position with a left-convex curve function. Also, the fifth position is on the right side with respect to the straight line 2. At this time, the position data of the object OB is generated by interpolating between the second position - the fifth position - the third position with a right-convex curve function.

[0042] If, as a comparative example, straight lines 1 and 2 are used as the position data of the object OB, the shape of the curved object OB cannot be appropriately represented. In contrast, according to the above-described first method, it is determined on which side in the horizontal direction the second position is with respect to straight line 3. Also, according to the above-described second method, it is determined on which side in the horizontal direction the fourth position is with respect to straight line 1 and on which side in the horizontal direction the fifth position is with respect to straight line 2. Then, since the position data of the object OB is generated using these determination results, even if the object OB is curved, its shape and position can be represented more accurately.

[0043] Further, the imaging device 1 in FIG. 1 includes a monitoring unit 25 and an identifying unit 26.

[0044] The monitoring unit 25 sets a predetermined monitoring area according to the purpose based on the position data generated by the above-described method for the object OB. The setting information for setting the monitoring area can be input from the input unit 21. For example, when the object OB is a railway line, the monitoring area setting information is input based on the construction limit of the railway line. Also, for example, when the object OB is a road, the monitoring area setting information is input based on the construction limit of the road.

[0045] FIG. 5 is an example of setting the monitoring area. In FIG. 5, in a three-dimensional space with the horizontal direction as the x-axis, the vertical direction as the y-axis, and the depth direction as the z-axis, a monitoring area function f(x, y, z) is defined. When the point (x, y, z) is within the monitoring area, f(x, y, z) = 1, and when the point (x, y, z) is outside the monitoring area, f(x, y, z) = 0. The monitoring unit 25 determines the monitoring area function f(x, y, z) based on the position data of the object OB in accordance with the monitoring area setting information input to the input unit 21.

[0046] When the specific unit 26 recognizes an object that can be an obstacle in the space of the imaging target, it calculates the probability that the object exists within the monitoring area set based on the position data of the target object OB. Then, it compares this probability with the alarm threshold value input to the input unit 21 to determine whether to issue an alarm.

[0047] FIG. 6 shows a state where a balloon is caught on an electric wire stretched above the target object OB. Assume that the balloon, which is an example of an object that can be an obstacle, is found to exist within the second distance zone but not within the third distance zone from the distance image data. That is, it is specified that the balloon exists between distances / z23~z3. And when the balloon is at position (1), it is outside the monitoring area, and when it is at position (2), it is inside the monitoring area. Therefore, the specific unit 26 calculates the probability of the balloon existing within the monitoring area by the following method.

[0048] As shown in FIG. 7, for example, for the representative point (x0, y0) of the balloon in the vertical plane, the probability P(x0, y0) that the balloon is within the monitoring area is calculated. This calculation is performed using the following formula.

[0049]

Equation

[0050] Then, the probability P(x0, y0) is compared with the alarm threshold value Palarm to determine whether to issue an alarm.

[0051] Alternatively, as shown in FIG. 8, for example, for the area σ of the balloon in the vertical plane, the probability P(σ) that the balloon is within the monitoring area is calculated. This calculation is performed using the following formula.

[0052]

Equation

[0053] Then, the probability P(σ) is compared with the alarm threshold value Palarm to determine whether to issue an alarm.

[0054] As described above, according to this embodiment, for the object OB existing across the adjacent first and second distance zones, in the first method, the first position at the front side position of the first distance zone, the second position at the boundary position between the first and second distance zones, and the third position indicating the position at the back side position of the second distance zone are obtained, and the position information of the second position with respect to the straight line connecting the first position and the third position is obtained. Further, in the second method, in addition to the first position, the second position, and the third position, the fourth position at the front side position of the third distance zone including the boundary position between the first and second distance zones, and the fifth position at the back side position of the third distance zone are obtained, and the position information of the fourth position with respect to the straight line connecting the first position and the second position, and the position information of the fifth position with respect to the straight line connecting the second position and the third position are obtained. Then, using this position information, the position data of the object OB is generated. Thereby, even when the object OB is curved, its position and shape can be represented more accurately, and thus, for the object OB existing across the adjacent first and second distance zones, its position data can be generated with higher accuracy.

[0055] Also, for this object OB, a desired monitoring area can be set around it. Furthermore, when an object that can be an obstacle is recognized in the monitoring area of the object OB, the presence or absence of an alarm can be accurately determined.

[0056] Note that in the above-described embodiment, in the generation of the position data of the object OB, a method of interpolation using a curve function convex to the right or left is used, but the present disclosure is not limited to this.

[0057] FIG. 9 shows another method of generating the position data of the object OB. Now, as shown in FIG. 3(a), it is assumed that the second position is determined to be on the left side of the straight line 3. In this case, as shown in FIG. 9, the first distance zone is virtually divided into z0 to z2n in the depth direction. Then, the horizontal position xi for the distance zi (i = 0 to 2n) is defined by adding a correction value Δxi to the x coordinate of the straight line 1. And the correction Δxi is determined as follows.

[0058] [Number]

[0059] Here, α is a negative value when the second position is on the left side of the straight line 3, and a positive value when the second position is on the right side of the straight line 3. Also by this method, in the present disclosure, the position data of the object OB can be generated. [Industrial Applicability]

[0060] The imaging device according to the present invention can perform peripheral monitoring of an object with higher accuracy using the sub-range method, and thus is useful for, for example, obstacle detection on railways and roads, and position monitoring of people and objects in factories, logistics warehouses, hospitals, schools, commercial facilities, etc. [Description of Reference Numerals]

[0061] 1 Imaging device 10 Detection unit 11 Distance measurement camera 12 Light source 20 Control unit 21 Input unit 22 Distance image generation unit 23 Extraction unit 24 Calculation unit 25 Monitoring unit 26 Identification unit

Claims

1. A detection unit having a distance measurement camera, a control unit that divides a target space into a plurality of distance zones based on the distance from a reference point in the depth direction, and generates distance image data representing each of the distance zones from the output of the detection unit, wherein the control unit has an input unit for inputting setting information of the distance zone including the number of the distance zones and the front-side position and the back-side position of the distance zone, a distance image generation unit that generates the distance image data from the output of the detection unit using the setting information of the distance zone input to the input unit, an extraction unit that extracts an object existing across adjacent first and second distance zones, and a calculation unit that calculates object position data for estimating the position and shape of the object extracted by the extraction unit based on the distance image data, wherein the calculation unit in a plane composed of the depth direction and a horizontal direction perpendicular to the depth direction, obtains a first position indicating the position of the object at the front-side position of the first distance zone, a second position indicating the position of the object at the boundary position between the first and second distance zones, and a third position indicating the position of the object at the back-side position of the second distance zone, and generates the object position data based on the position information of the second position with respect to a straight line connecting the first position and the third position The imaging device is characterized by the above.

2. A detection unit having a distance measurement camera, a control unit that divides a target space into a plurality of distance zones based on the distance from a reference point in the depth direction, and generates distance image data representing each of the distance zones from the output of the detection unit, wherein the control unit has an input unit for inputting setting information of the distance zone including the number of the distance zones and the front-side position and the back-side position of the distance zone, a distance image generation unit that generates the distance image data from the output of the detection unit using the setting information of the distance zone input to the input unit, an extraction unit that extracts an object existing across adjacent first and second distance zones, and a calculation unit that calculates object position data for estimating the position and shape of the object extracted by the extraction unit based on the distance image data, wherein the calculation unit sets a third distance zone including the boundary position between the first and second distance zones, in a plane composed of the depth direction and a horizontal direction perpendicular to the depth direction, A first position indicating the position of the object at a position on the front side of the first distance zone, a second position indicating the position of the object at a boundary position between the first and second distance zones, a third position indicating the position of the object at a position on the back side of the second distance zone, a fourth position indicating the position of the object at a position on the front side of the third distance zone, and a fifth position indicating the position of the object at a position on the back side of the third distance zone are obtained. Based on the position information of the fourth position with respect to the straight line connecting the first position and the second position, and the position information of the fifth position with respect to the straight line connecting the second position and the third position, object position data is generated. An imaging device characterized by the above.

3. In the imaging device according to claim 1 or 2, the imaging device includes a monitoring unit that sets a monitoring area based on a monitoring area function input to the input unit with reference to the object position data. An imaging device characterized by the above.

4. In the imaging device according to claim 3, the input unit can input a threshold value for alarm reporting due to the presence of an obstacle in the monitoring area. When the imaging device recognizes an object that can be an obstacle in the target space, for a representative point of the object in the vertical plane, a probability value is calculated by integrating the monitoring area function in the depth direction, and the probability value is compared with the threshold value input to the input unit to determine whether to issue an alarm. The imaging device includes a specifying unit for this purpose. An imaging device characterized by the above.

5. In the imaging device according to claim 3, the input unit is configured to be able to input a threshold value for alarm reporting due to the presence of an obstacle in the monitoring area. When the imaging device recognizes an object that can be an obstacle in the target space, for the area of the object in the vertical plane, a probability value is calculated by integrating the monitoring area function in the depth direction, and the probability value is compared with the threshold value input to the input unit to determine whether to issue an alarm. The imaging device includes a specifying unit for this purpose. An imaging device characterized by the above.

Images