Image processing method and image processing device

Abstract

An image processing method according to the present disclosure includes: a first process for acquiring a range gate image (RG) using a range gate imaging device (2) that captures an image in a set distance range for a predetermined capturing area (CA); a second process for searching for an object to be detected in the range gate image using a search window having a size according to a capturing distance (S) from the range gate imaging device (2) to a set distance extent; and a third process for synthesizing, when a window region satisfying a predetermined condition is detected in a range gate image (RG) in the second process, object information included in the window region and a grayscale image acquired using a grayscale imaging device (3) that captures a grayscale image of a predetermined capturing area and outputting the synthesized image.

Description

[Technical Field]

[0001] The present disclosure relates to an image processing method and an image processing device. [Background technology]

[0002] Patent document 1 shows a configuration in which an image processing device is provided with an image processing unit that divides the image area of ​​a specified image into at least two areas based on distance information obtained by a distance measuring sensor, and performs image processing on at least one of the two areas of the image so that the two areas have different image quality, with the aim of preventing frame dropping when transmitting high-resolution or high-frame-rate moving images.

[0003] The image processing device in Patent Document 1 is designed to identify a region of interest designated by a user based on the position information of the image designated by the user and the distance information measured by a distance measuring sensor. Specifically, object regions having the same distance information are detected around the position designated by the user, and the detected region is determined as the region of interest. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2017-224970 Summary of the Invention [Problem to be solved by the invention]

[0005] However, the technology of Patent Document 1 requires input from the user, and has a problem in that it is not possible to automatically perform image processing to give the two regions different image quality.

[0006] Furthermore, when searching distance information (distance image) from a distance sensor, for example, the search window size must be adjusted because the image contains a mixture of distance values ​​with gradations. Furthermore, searching a distance image requires complex processing to match it with a 3D model, which slows down the processing speed.

[0007] The present disclosure has been made in consideration of the above points, and has an object to automatically and quickly extract an image region to be detected. [Means for solving the problem]

[0008] In order to solve the above problem, an image processing method using an image processing device according to one embodiment of the present disclosure includes a first process of acquiring a range gate image using a range gate imaging device that captures an image within a set distance range for a predetermined imaging area, a second process of searching for a detection target in the range gate image using a search window of a size according to the imaging distance from the range gate imaging device to the set distance range, and a third process of, if a window area that satisfies predetermined conditions is detected in the range gate image in the second process, combining a gradation image acquired using another imaging device that captures the predetermined imaging area with information about the window area and outputting the combined result. [Effects of the Invention]

[0009] According to the present disclosure, a search window of a size according to the imaging distance is used, thereby eliminating unnecessary searches that would otherwise be performed using a search window size that does not match the size of the detection target. Furthermore, since a range gate image is essentially a binary image, the amount of calculation required for the search process is smaller than that required for a grayscale image, and therefore the image region of the detection target can be extracted automatically and quickly. [Brief explanation of the drawings]

[0010] [Figure 1] Schematic diagram of the configuration of the image processing device, the imaging area, and the search window [Figure 2] Block diagram showing an example of the configuration of an image processing device [Figure 3A] FIG. 10 is a diagram showing an example of the relationship between the number of ranges and the imaging distance. [Figure 3B] FIG. 10 is a diagram showing another example of the relationship between the number of ranges and the imaging distance. [Figure 3C] FIG. 10 is a diagram showing another example of the relationship between the number of ranges and the imaging distance. [Figure 4] A block diagram showing an example of the configuration of a range gate imaging device. [Figure 5] Flowchart showing an example of operation of an image processing device [Figure 6] FIG. 1 is an explanatory diagram illustrating an example of the operation of an image processing apparatus according to a first embodiment; [Figure 7] FIG. 10 is an explanatory diagram showing another example of the operation of the image processing device; [Figure 8] FIG. 3C is an explanatory diagram showing an example of the operation of the image processing device corresponding to FIG. 3B; [Figure 9] FIG. 10 is an explanatory diagram showing another example of the operation of the image processing device; [Figure 10] A conceptual diagram showing an example of the search window settings [Figure 11] FIG. 10 is an explanatory diagram showing another example of the operation of the image processing apparatus according to the second embodiment; [Figure 12] An explanation of binary images (binary images) DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. The following description of the embodiments is merely exemplary in nature and is not intended to limit the present invention, its applications, or its uses. In other words, the numerical values, shapes, components, component placement positions, and connection forms shown in the following embodiments are merely examples and are not intended to limit the present disclosure. Therefore, among the components in the following embodiments, components that are not described in the independent claims that represent the superordinate concept of the present disclosure will be described as optional components.

[0012] First Embodiment Fig. 1 is a schematic diagram showing an outline of the configuration, imaging area, and search window of an image processing device according to this embodiment, while Fig. 2 is a block diagram showing an example of the configuration of the image processing device.

[0013] 1, image processing device 1 includes range gate imaging device 2, grayscale imaging device 3, and calculation unit 4. When a robot is operating in a factory, for example, image processing device 1 of the present disclosure is used to determine the position of detection target M from range gate image RG captured by range gate imaging device 2 when detection target M moves along a conveyer belt, and then extract and output an area including the search target from a grayscale image captured by grayscale imaging device 3 based on the position information. Information output from image processing device 1 is used in subsequent processing (e.g., image recognition processing).

[0014] -Range gate imaging device- The range gate imaging device 2 captures a range gate image RG within a set distance range (hereinafter referred to as imaging range b) for a predetermined imaging area CA. The range gate imaging device 2 outputs the range gate image RG captured for each imaging range b and information on the imaging distance S to the calculation unit 4.

[0015] Here, multiple imaging ranges b can be set, and the number of imaging ranges b is referred to as the number of ranges. In the following explanation, the number of ranges is n, where n is an arbitrary integer equal to or greater than 1. For convenience of explanation, the nth range will be referred to as the nth range, and the imaging range b of the nth range will be referred to as the imaging range b. n The imaging distance S, the range gate image RG, and the search window VA described later may also be described by assigning symbols in the same manner.

[0016] FIG. 3 (FIGS. 3A to 3C) shows an example of the relationship between the number of ranges n of the range gate and the imaging distance S. The imaging distance S is the distance from the range gate imaging device 2 to each imaging range b n In the example of FIG. 3, the distance from the range gate imaging device 2 to each imaging range bn The distance to the start position of the imaging distance S1 to S n The imaging distance S is not limited to the distance to the start position of the range, but may be the distance to the middle position of the range.

[0017] For example, in FIG. 3, the imaging distance S1 of the first range is the distance from the range gate imaging device 2 to the start position of the first range, and the distance width from the start position to the end position of the imaging range b1 (hereinafter simply referred to as the distance width) is l1. Similarly, the imaging distance S2 of the second range is the distance from the range gate imaging device 2 to the start position of the second range, and the distance width of the imaging range b2 is l2. Furthermore, the imaging distance S of the nth range is n is the distance from the range gate imaging device 2 to the start position of the nth range, and the imaging range b n The distance between n is.

[0018] In the example of Figure 3, the distance widths l1, l2, ... l n Although an example in which the distances 1 are all equal is shown, the distances 1 may be different from each other.

[0019] In Figure 3A, the imaging range b n 1 shows an example in which the distance 1 in the depth direction of the imaging area CA is uniform and n adjacent imaging ranges b are arranged without any gaps.

[0020] In FIG. 3B, in the depth direction of the imaging area CA, each imaging range b n , and the front and rear range b n 3B shows an example in which an overlapping area (hereinafter simply referred to as an overlapping area) is provided where a part of the imaging area overlaps with the other imaging area. The setting in FIG. 3B is useful when an object spanning multiple ranges is to be judged as a single object.

[0021] In FIG. 3C, in the depth direction of the imaging area CA, each imaging range b n , the front and rear imaging range b nThis shows an example in which a non-imaging area is set between the target and the target. The setting in Figure 3C is useful when the observation range is fixed, for example, when monitoring only the area around the doors on a train platform.

[0022] Fig. 4 shows an example configuration of the range gate imaging device 2. As shown in Fig. 4, it includes a light source 21, a camera 22, a shutter 23, and a control unit 24. The range gate imaging device 2 is configured to perform exposure at a time delayed from the time of irradiation of pulsed light from the light source 21. The distance traveled by the light during the delay time is the imaging distance S of the distance range (imaging range b) captured in the range gate image RG. The distance traveled by the light during the exposure time is the distance width l of the distance range (imaging range b) captured in the range gate image.

[0023] The configuration of the range gate imaging device 2 is not limited to that shown in FIG. 4, and other conventionally known range gate imaging devices may be used.

[0024] The control unit 24 outputs trigger signal 1, trigger signal 2, and trigger signal 3 according to the imaging range b to be imaged. The distance that light travels back and forth during the delay time between trigger signal 1 and trigger signal 2 is the imaging distance S.

[0025] The light source 21 is a pulsed light source, and based on a trigger signal 1 received from the control unit 24, irradiates the imaging area CA with light according to the imaging range b to be imaged.

[0026] The shutter 23 is a global shutter that opens and closes based on a trigger signal 2 received from the control unit 24. The shutter 23 is, for example, a global electronic shutter, a mechanical shutter, or a liquid crystal shutter.

[0027] The camera 22 captures a range gate image RG based on a trigger signal 3 received from the control unit 24. The imaging element of the camera 22 is, for example, a highly sensitive sensor such as an avalanche photodiode.

[0028] Here, the range gate image RG is an image corresponding to the distance between the range gate imaging device 2 and the object to be imaged (corresponding to the amount of exposure delay for the light source 21 when imaging with the range gate imaging device 2). The range gate image RG has a short exposure time, resulting in a coarse texture. For example, the exposure time is 66.7 ns for a distance range of 100 m. The range gate image RG also includes background texture information.

[0029] In other words, the range gate image RG is essentially a binary image. Specifically, the range gate imaging device 2 performs processing to emit light for each imaging range b and capture the timing of the return light. This results in a very short shutter time. Furthermore, as described above, a highly sensitive sensor such as an avalanche photodiode is used. As a result, the image captured by the range gate imaging device 2 is a binary image. Here, a binary image includes an image in which the histogram of pixel values ​​is polarized. For example, when capturing an image using an avalanche photodiode, a histogram polarized by multiplied and unmultiplied pixels, as shown in FIG. 12, is obtained. It also includes a gradation image with a few bits. In other words, in this disclosure, a binary image refers to an image that appears to have significantly fewer gradations than a normal gradation image. Specifically, in this disclosure, the term "binary image" is used to conceptually include not only a completely binary image but also the above-mentioned binary images (images with a polarized histogram and gradation images with a few bits). In other words, the binary image referred to here includes the concept of an image that can be easily binarized when determining whether or not a pixel exists in each pixel region.

[0030] -Gradation imaging device- Returning to Fig. 1, the gradation imaging device 3 is an imaging device that captures a gradation image using background light in a predetermined imaging area CA. The gradation imaging device 3 is not particularly limited and includes a texture imaging device that captures a texture image, a general imaging device such as a digital camera that captures a visible light image (an imaging device using a CMOS sensor or a CCD sensor), an X-ray camera, or a thermal camera. The gradation imaging device 3 outputs the captured gradation image to the calculation unit 4.

[0031] The predetermined imaging area CA here means imaging an imaging area common to the range gate imaging device 2. Note that it is not intended that the imaging range of the range gate imaging device 2 and the imaging range of the gradation imaging device 3 are the same. In other words, as long as the range gate imaging device 2 and the gradation imaging device 3 are configured to be able to image the common imaging area CA, the imaging ranges of the two devices may be different from each other.

[0032] -Arithmetic section- The calculation unit 4 combines object information obtained by searching the range gate image RG received from the range gate imaging device 2 with image information corresponding to the object information in the gradation image received from the gradation imaging device 3, and outputs the combined information.

[0033] As shown in FIG. 2, the calculation unit 4 includes a search processing unit 41, an image corresponding unit 42, and a synthesis unit 43.

[0034] [Search processing section] The search processing unit 41 searches for a detection target within the range gate image RG using a search window VA having a size according to the imaging distance S, and outputs information about the object detected by the search.

[0035] In FIG. 1, a search window VA1 is set in a range gate image RG1 of the first range b1, and a search window VA2 is set in a range gate image RG1 of the (n-1)th range b n-1 The search window VA is set on the range gate image RG. n-1 In this way, in this embodiment, the range gate image RG and the corresponding imaging distance S n Search window VA according ton The size of the range gate image RG is changed. n Specifically, the image is searched for each imaging distance S n As the distance increases, the search window VA n The size also gradually decreases.

[0036] There are no particular limitations on the method for setting the size (horizontal size and vertical size) of the object to be detected (hereinafter simply referred to as the detection target) using the search window VA. For example, (1) one or more default values ​​may be set as preset values, (2) the user may specify the size of the search window VA either during or before operation of the image processing device, or (3) the size may be adjusted automatically. Furthermore, the above setting methods (1) to (3) may be combined. In the above (2), when the user specifies the size of the search window VA, examples include specifying a numerical value or selecting from several options.

[0037] Regarding the setting method by automatic adjustment in (3) above, the specific setting method is not particularly limited, and the following two methods are exemplified.

[0038] For example, as a first method, a range gate image RG is captured in a predetermined imaging range b using a range gate imaging device 2. After that, the range gate image RG is searched by varying the size of a search window VA. Then, the size of the object in each range gate image RG is calculated based on the relationship between the size of the detected object on the image and the distance in the range gate image RG.

[0039] As a second method, for example, a range gate image RG in a predetermined imaging range b is captured using a range gate imaging device 2, and a gradation image is captured using a gradation imaging device 3. Edge extraction (plane differential processing) is performed on the gradation imaging device 3. The range gate image RG in the predetermined imaging range b is compared with the image after the edge extraction, and the area where the same edges are obtained in both images is determined to be the area of ​​the object. Then, the object size is calculated from the imaging distance S of the corresponding range gate image RG and the area size of the object on the range gate image RG.

[0040] The size of the search window VA is set based on the relationship between the size of the detection target set above and the imaging distance S. In this case, the size of the search window VA may be set taking into consideration the shadow cast by the light source 21 of the range gate imaging device 2.

[0041] The effect of using a search window of a size according to the imaging distance in this way will be shown below.

[0042] When the number of pixels in the image is H pixels horizontally and V pixels vertically, and the detection target is imaged at N distances, the search window size that matches the detection target at each distance is defined as: ROI (k), vertical:V ROI (k), where k=1 to N.

[0043] When performing a full search while adjusting the search window size using only a conventional texture image, the image is searched for each window size, so the number of searches is calculated as follows:

[0044]

number

[0045] On the other hand, when scanning a range gate image, only objects that exist within the distance range corresponding to the range gate image are captured in each range gate image. For this reason, the search window size is set horizontally to H on the left and right for the area where the object is captured for each range gate image. EX Pixels, vertically V EX The range of pixels can be searched. The number of searches is calculated using the following formula:

[0046]

number

[0047] The area in which the object is captured can be determined by calculating the center of gravity of each range gate image.

[0048] For example, H=640, V=480, N=3, H ROI (k)=240,120,80,V ROI (k)=180,90,60,H EX =10,V EX = 10, the conventional method requires 558,000 searches, whereas the proposed method can reduce the number of searches to 1,200.

[0049] [Image Correspondence Section] The image corresponding unit 42 outputs image information corresponding to the object information (hereinafter also referred to as "corresponding image information") based on the gradation image captured by the gradation imaging device 3 and the object information output from the search processing unit 41. The corresponding image information is, for example, a cut-out image obtained by cutting out the object information (including the periphery of the object information) from the gradation image, or a background image that does not include the object information.

[0050] Specifically, the image correspondence unit 42 calculates a homography matrix from the image of the range gate imaging means to the image of the gradation imaging device 3 based on the optical and mechanical design parameters, and obtains the corresponding corresponding image information (texture information if a texture imaging device is used) using the homography matrix.

[0051] The homography matrix here is a matrix that defines, when two cameras capture images of a point on a plane in a certain space, the coordinate information of the point captured by one camera is projected onto the coordinates of the other camera. Note that calibration of this homography matrix is ​​performed in advance.

[0052] Based on the calculation of the homography matrix, the image correspondence unit 42 generates corresponding image information (texture information when a texture imaging device is used) by cutting out or clipping an image from the gradation image for an area (hereinafter referred to as the enlarged area) that is enlarged by several pixels in the vertical and horizontal directions from the object information output from the search processing unit 41. Note that the image may be cut out or clipped from the gradation image based on the object information without setting an enlarged area.

[0053] 10, the enlarged region may be further enlarged in the direction in which the shadow of the light source 21 of the range gate imaging device 2 is cast (referred to as the shadow direction). More specifically, the image correspondence unit 42 estimates the region of the shadow cast by the light source 21 on the detection target M (shadow region) based on at least one of the positional relationship between the light source 21 and the camera 22 and the imaging distance S, and further enlarges the enlarged region in accordance with the shadow region. In FIG. 10, the shadow region is illustrated by dot hatching, with J1 representing the crop region initially set by the image correspondence unit 42 and J2 representing the crop region enlarged in the shadow direction of the light source 21. Note that the method of estimating the shadow region by the image correspondence unit 42 is not particularly limited. For example, the shadow region may be estimated by adding thickness information of the detection target, or the shadow region may be estimated based on the imaging distance S at which the target range gate image RG was captured.

[0054] [Synthesis section] The synthesis unit 43 associates the object information output from the search processing unit 41 with the corresponding image information output from the image correspondence unit 42 and outputs them. Specifically, the synthesis unit 43 executes (1) a process of storing texture information (image), region information (numerical value), and distance information (numerical value) for each pixel of an image, and (2) a process of integrating the texture information of the range gate image RG with the texture information of the gradation image. An example of the process (2) above is a process of interpolating color information between a range gate image captured using infrared light and a gradation image captured using visible light. The results of the processes (1) and (2) above are then output to a downstream circuit (program).

[0055] The output of the synthesis unit 43 is used in the above-mentioned subsequent processing (for example, image recognition processing), etc. In this embodiment, the function of the synthesis processing unit is realized by the image corresponding unit 42 and the synthesis unit 43. However, the method for realizing the function of the synthesis processing unit is not limited to this configuration.

[0056] - Operation of image processing device - The operation of the image processing device and the image processing method according to the present disclosure will be described below with reference to Fig. 5. Here, it is assumed that the detection target M is a rectangular parallelepiped, as shown in Fig. 6, and is in the state SV1 in Fig. 6.

[0057] -Step S1- In step S1, the search processing unit 41 refers to the range gate image RG for the set imaging range b. For example, at the start of processing, the range gate imaging device 2 is used to acquire a range gate image RG1 for a first range b1, and the range gate image RG1 is then referenced. Note that at this time, the range gate imaging device 2 may acquire range gate images RG1 for multiple imaging ranges b at once.

[0058] -Step S2- In step S2, the search processing unit 41 sets the size of the search window VA. Here, the search window VA1 is set for the range gate image RG1 of the first range b1. The method for setting the search window VA1 is not particularly limited, but for example, the size of the detection target is set using the search window VA1, and is set based on the relationship between the size of the detection target and the imaging distance S. The setting of the size of the detection target is as described above, and a detailed description thereof will be omitted here.

[0059] -Step S3- In step S3, the search processing unit 41 uses the search window VA to search for a window region that satisfies predetermined conditions within the range gate image RG. More specifically, it determines whether object information that satisfies the predetermined conditions can be obtained. For example, it determines whether the captured object is the detection target M based on the relationship between the size of the captured object within the window region of the range gate image RG and the imaging distance S. As shown in FIG. 6, if the position of the search window VA that satisfies the predetermined conditions is found within the range gate image RG, that position is identified as the window region in which the detection target exists (see RG1 in FIG. 6). The processes from step S1 to step S3 correspond to the first process and the second process.

[0060] -Step S4- In step S4, the search processing unit 41 determines whether a search target has been detected or whether the final imaging range b has been reached. For example, if there is one search target, the determination here is YES, and if there are two or more search targets, the determination is NO. If the determination is YES, the flow proceeds to step S5. On the other hand, if the determination is NO, the flow returns to step S1, and the processing from step S1 to step S4 is repeated. Here, it is assumed that there are two detection targets M, and the processing from step S1 to step S4 is repeated to detect objects M1 and M2.

[0061] -Step S5- In step S5, the search processing unit 41 outputs object information. In this example, the search processing unit 41 outputs information on the objects M1 and M2 as the object information.

[0062] The object information output from the search processing unit 41 includes at least one of pixel information of objects M1, M2 detected as the detection target M imaged in a window area that satisfies predetermined conditions during a search within each range gate image RG, or information on the rectangular area in which the objects are inscribed.

[0063] The pixel information of the objects M1 and M2 includes, for example, coordinate information of the pixels where the objects M1 and M2 exist or coordinate information of the contour pixels of the area where the objects M1 and M2 exist. The rectangular area information inscribed with the objects M1 and M2 includes, for example, the coordinates of any of the four corners of the rectangular area or the coordinates of the center, and dimensional information of the rectangular area (number of horizontal pixels, number of vertical pixels). The object information may also include information on the imaging distance S of a window area that satisfies a predetermined condition. In the example of FIG. 6, in addition to the pixel information or rectangular area information, information on the imaging distance S1 at which the object M1 was detected and information on the imaging distance Sn at which the object M2 was detected are output as the object information.

[0064] -Step S6- In step S6, the image corresponding unit 42 outputs corresponding image information based on the gradation image captured by the gradation image capturing device 3 and the object information output from the search processing unit 41.

[0065] Specifically, the image correspondence unit 42 generates and outputs correspondence image information by cutting out or clipping an image of the enlarged area based on the object information from the grayscale image based on the calculation of the homography matrix described above.

[0066] -Step S7- In step S7, the synthesis unit 43 associates the object information output from the search processing unit 41 with the corresponding image information output from the image correspondence unit 42 and outputs them. Specifically, the synthesis unit 43 executes a process of storing texture information, region information, and distance information for each pixel of one image, and a process of integrating the texture information of the range gate image RG and the texture information of the gradation image, and outputs the results to a subsequent circuit (program).

[0067] As described above, according to this embodiment, a search window of a size corresponding to the imaging distance S of the range gate imaging device 2 is used, thereby eliminating unnecessary searches caused by using a search window size that does not match the size of the detection target M. Furthermore, as described above, the range gate image RG is essentially a binary image, and the amount of calculation required for the search process is smaller than that required for a grayscale image captured by the grayscale imaging device 3. This makes it possible to automatically and quickly extract the image area of ​​the detection target.

[0068] -Variation (1)- 7 shows an example in which the boundary between the first range b1 and the second range b2 is located at the middle position of the search detection object M, that is, the search detection object M straddles the first range b1 and the second range b2. In such a case, when two images are superimposed on adjacent range gate images RG, a continuous object is detected as a single object. In other words, a continuous image is obtained in the adjacent regions on the range gate image RG.

[0069] 7, for example, in step S1, when the search processing unit 41 refers to the range gate image RG, it also refers to the adjacent range gate images RG. Then, for example, if an image of a continuous object is detected in the range gate images RG of the adjacent imaging range b, the process proceeds to step S3 using an image obtained by logically adding the range gate images before and after the object.

[0070] For example, in the example of Fig. 7, the boundary between the object M21 detected on the range gate image RG1 of the first range b1 and the object M22 detected on the range gate image RG2 of the second range b2 has the same length and a shape that is continuous when overlapped. Therefore, the image RG1, which is an image obtained by logically ORing both range gate images RG1 and RG2, is a , and the search window VA in step S3 is calculated. 12 The other operations are the same as those in the above embodiment, and the same effects can be obtained.

[0071] -Variation (2)- The example of Fig. 8 shows an example of operation in the case where an overlapping area is provided in each imaging range bn in the depth direction of the imaging area CA, as shown in Fig. 3B. In the example of Fig. 8, as in the case of Fig. 7, the search detection target M is imaged so as to straddle the boundary between the first range b1 and the second range b2. In such a case, an object M having an overlapping area WS that overlaps each other is detected in adjacent range gate images RG.

[0072] Even in this example, when the search processing unit 41 refers to the range gate image RG in step S1, it also refers to the adjacent range gate images RG before and after. Then, for example, if the image areas of the adjacent range gate images RG of imaging range b before and after overlap, it is determined that they are the same object, and the process proceeds to step S3 using an image obtained by logically adding the range gate images before and after.

[0073] For example, in the example of Fig. 8, there is an overlapping area WS between an object M31 detected on the range gate image RG1 and an object M32 detected on the range gate image RG2. Therefore, an image RG1, which is an image obtained by logically ORing both range gate images RG1 and RG2, is used. b , and the search window VA in step S3 is calculated. 12 The other operations are the same as those in the above embodiment, and the same effects can be obtained.

[0074] -Variation (3)- 9 shows an example in which a stationary object Mx is captured in addition to the detection target M. The stationary object Mx may be, for example, a facility in a factory or a structure attached to a wall or facility. In such a case, it is expected that the same object Mx will be captured in both the first range b1 and the second range b2.

[0075] 9, for example, in step S1, when the search processing unit 41 refers to the range gate image RG, it also refers to the adjacent range gate images RG. Then, for example, if a common stationary object Mx is detected in a plurality of consecutive range gate images RG (above a predetermined threshold) of imaging ranges b, for example, in step S1, a process is performed to remove the common stationary object Mx from each range gate image RG as a background light component. Then, the process of step S3 is performed using the range gate image RG from which the stationary object Mx has been deleted.

[0076] 9, a stationary object Mx is detected in both the range gate image RG1 of the first range b1 and the range gate image RG2 of the second range b2, so a process is executed to remove the stationary object Mx as a background light component from each of the range gate images RG1 and RG2. The other operations are the same as those in the above embodiment, and the same effects are obtained.

[0077] <Second embodiment> 11 is a diagram for explaining the operation of the image processing device according to the second embodiment and the image processing method according to the present disclosure. Note that the configuration and basic operation of the image processing device 1 are the same as those of the first embodiment, and the following description will focus on the differences.

[0078] In this embodiment, if the number of pixels in the range gate image RG in the set distance range in the search in step S3 exceeds the set number of pixels according to the imaging distance S, it is determined in step S4 that the detection target exists in the range gate image RG, that is, that the detection target has been detected. n As the distance increases, the number of pixels is set to gradually decrease.

[0079] Even when using a method such as that of this embodiment, a search is performed using the number of pixels according to the imaging distance S of the range gate imaging device 2, so it is possible to avoid unnecessary searches such as searching for a range gate image RG with a number of pixels that does not match the size of the detection target M. This makes it possible to automatically and quickly extract the image area of ​​the detection target.

[0080] Furthermore, by using the method of this embodiment, it is possible to obtain the effect of reducing the amount of calculations even compared to the method of the first embodiment.

[0081] <Other embodiments> The present disclosure is not limited to the above-described embodiments, and various modifications are possible without departing from the spirit of the present disclosure.

[0082] For example, each embodiment and its modified examples may be combined, or modified examples may be combined to form a new embodiment. Specifically, for example, the second embodiment may be combined with the modified example (3) of the first embodiment to form a new embodiment. [Industrial Applicability]

[0083] The image processing method and image processing device of the present disclosure are extremely useful because they can automatically and quickly extract an image region to be detected. [Explanation of symbols]

[0084] 1. Image processing device 2 Range gate imaging device 3-tone imaging device 41 Search processing unit 43 Synthesis section

Claims

1. An image processing method using an image processing device, a first process of acquiring a range gate image using a range gate imaging device that captures an image within a set distance range for a predetermined imaging area; a second process of searching for a detection target in the range gate image using a search window having a size according to an imaging distance from the range gate imaging device to the set distance range; a third process for, when a window region satisfying a predetermined condition is detected in the range gate image in the second process, synthesizing object information included in the window region with a gradation image acquired by a gradation image pickup device that picks up a gradation image of the predetermined image pickup area, and outputting the synthesized result; the range gate imaging device includes a light source and a camera; an image processing method for estimating a shadow area that may be generated by the light source as the detection target based on at least one of a positional relationship between the light source and the camera or the imaging distance, and setting a size of the search window taking into account the shadow area;

2. 2. The image processing method according to claim 1, wherein the object information includes at least one of pixel information of an object captured in a window area that satisfies the predetermined condition and information of a rectangular area inscribed by the object.

3. The image processing method according to claim 2 , wherein the object information includes information about the imaging distance of the window area that satisfies the predetermined condition.

4. The image processing method according to claim 1 , wherein the gray scale image capture device is a texture image capture device that captures a texture image.

5. 2. The image processing method according to claim 1, wherein, when an object having a size corresponding to the imaging distance is detected within the search window in the search of the second process, it is determined that a window area that satisfies the predetermined condition is present, and the third process is executed.

6. 2. The image processing method according to claim 1, wherein, in the search of the second process, if the number of pixels of a stationary object detected within the search window exceeds a predetermined number corresponding to the imaging distance, it is determined that there is a window area that satisfies the predetermined condition, and the third process is executed.

7. An image processing method using an image processing device, a first process of acquiring a range gate image using a range gate imaging device that captures an image within a set distance range for a predetermined imaging area; a second process of searching for a detection target in the range gate image using a search window having a size according to an imaging distance from the range gate imaging device to the set distance range; a third process for performing a cutout process in which, when a window region satisfying a predetermined condition is detected in the range gate image in the second process, object information included in the window region is acquired, and an area on the gradation image corresponding to the object information is cut out and extracted from the gradation image acquired using a gradation image pickup device that picks up a gradation image of the predetermined imaging area, In the third processing, a homography matrix from the range gate image to the gradation image is calculated, and an enlarged area enlarged from the object information based on the homography matrix is ​​set as a target of the cutout processing; the range gate imaging device includes a light source and a camera; The image processing method estimates a shadow area that can be detected by the light source based on at least one of the positional relationship between the light source and the camera or the imaging distance, and further expands the enlarged area according to the shadow area.

8. 8. The image processing method according to claim 7, wherein the shadow area is estimated by adding thickness information of the object to be detected.

9. 8. The image processing method according to claim 7, wherein the shadow area is estimated based on an imaging distance at which a window area satisfying the predetermined condition is detected in the extraction process.

10. a range gate imaging device that captures a range gate image within a set distance range for a predetermined imaging area; a gradation imaging device that captures a gradation image of the predetermined imaging area; a search processing unit that searches for a detection target in the range gate image using a search window having a size according to an imaging distance from the range gate imaging device to the set distance range; a synthesis processing unit that, when a window area satisfying a predetermined condition is detected in the range gate image, synthesizes object information included in the window area with a gradation image acquired by the gradation imaging device and outputs the synthesized image; the range gate imaging device includes a light source and a camera; estimating a shadow area that may be generated as the detection target by the light source based on at least one of a positional relationship between the light source and the camera or the imaging distance, and setting a size of the search window taking the shadow area into consideration; Image processing device.

11. The image processing device according to claim 10 , wherein the object information includes at least one of pixel information of an object captured in a window area that satisfies the predetermined condition and information of a rectangular area inscribed by the object.

12. The image processing device according to claim 11 , wherein the object information includes information about the imaging distance of the window area that satisfies the predetermined condition.

13. The image processing device according to claim 10 , wherein the gray scale image capture device is a texture image capture device that captures a texture image.

14. The image processing device according to claim 10 , wherein the synthesis processing unit executes a process of synthesizing the object information and the gradation image and outputting the synthesized image when an object having a size according to the imaging distance is detected within the search window during a search by the search processing unit.

15. 11. The image processing device according to claim 10, wherein the synthesis processing unit executes a process of synthesizing the object information and the gradation image and outputting the synthesized image when the number of pixels of a still object detected within the search window in the search by the search processing unit exceeds a predetermined number according to the imaging distance.

16. a range gate imaging device that captures a range gate image within a set distance range for a predetermined imaging area; a gradation imaging device that captures a gradation image of the predetermined imaging area; a search processing unit that searches for a detection target in the range gate image using a search window having a size according to an imaging distance from the range gate imaging device to the set distance range; a processing unit that, when a window area satisfying a predetermined condition is detected in the range gate image, acquires object information included in the window area, and cuts out and extracts an area on the gradation image acquired using the gradation imaging device that corresponds to the object information included in the window area, the processing unit calculates a homography matrix from the range gate image to the gradation image, and sets an enlarged area that is enlarged from the object information based on the homography matrix as a target for the cutout; the range gate imaging device includes a light source and a camera; The processing unit estimates a shadow area that can be created by the light source as the detection target based on at least one of the positional relationship between the light source and the camera or the imaging distance, and further enlarges the enlarged area in accordance with the shadow area.

17. The image processing device according to claim 16 , wherein the processing unit estimates the shadow area by adding thickness information of the detection object.

18. The image processing device according to claim 16 , wherein the processing unit estimates the shadow area based on an imaging distance at which the window area satisfying the predetermined condition is detected.

Images