Control device

Abstract

To prevent the deterioration of stability of object detection in a control device mounted on a moving object.SOLUTION: A control device 200 mounted on a moving object M having a camera 110 and a distance measurement device 120 includes: an information acquisition unit 211 that acquires an image and object information and obtains a group of points from the acquired image; an object determination unit 212 that determines whether an object exists or not using the object information and the group of points; and a start prevention unit 214 that prevents the start of the moving object when the object is determined to be present. The object determination unit determines that the object exists when the group of points is included in a first search area, sets a second search area ARs2 when the first search area does not include the group of points and the difference between the distance indicated by the object information and the distance newly detected by the distance measurement device is less than a threshold value, and determines that the object exists when the group of points is included in the second search area and determines that the object does not exist when no group of points is included in either the first search area or the second search area.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a control device. [Background technology]

[0002] Conventionally, there is known a false start prevention technology that detects the presence or absence of an object in front of the vehicle and the distance between the object and the vehicle, and prevents the vehicle from starting erroneously due to an error in shifting or pedal operation by the driver.The object detection device described in Patent Document 1 captures an image of the area in front of the vehicle using a camera mounted on the vehicle, and detects the presence or absence of an object in front of the vehicle and the distance between the object and the vehicle. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2020-12742 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in an object detection device that combines a distance measuring device such as millimeter-wave radar with a camera, if the ground position of an object is out of the field of view of the camera when the object detection device is started, there will be a difference between the object detection position by the camera and the object detection position by the distance measuring device, and there is a risk that the presence or absence of an object and the distance between the object and the vehicle cannot be detected stably. This problem is not limited to vehicles, but can also occur when moving objects such as aircraft and robots start moving. [Means for solving the problem]

[0005] The present disclosure can be realized in the following forms.

[0006] According to one embodiment of the present disclosure, there is provided a control device (200) mounted on a moving body (M) having a camera (110) and a distance measuring device (120), and including an information acquisition unit (211) that acquires an image captured by the camera in the moving direction of the moving body and object information indicating a distance between the moving body and an object (Ob) present in the moving direction of the moving body detected by the distance measuring device, and acquires a point cloud representing the position of the object from the acquired image, an object determination unit (212) that determines whether the object exists using the object information and the point cloud, and a departure inhibition unit (214) that inhibits the departure of the moving body when it is determined that the object exists, and the object determination unit is configured to detect a predetermined size of the object. If the point cloud is included within a first search area (ARs1) that is set at a position according to the distance indicated by the object information, it is determined that the object exists; if the point cloud is not included within the first search area and the difference between the distance indicated by the object information and the distance newly detected by the ranging device is less than a predetermined threshold, a second search area (ARs2) that is expanded beyond the first search area in a direction away from the moving body is set; if the point cloud is included within the second search area, it is determined that the object exists; and if the point cloud is not included in either the first search area or the second search area, it is determined that the object does not exist.

[0007] According to this aspect of the control device, even if the point cloud acquired from the image is not included in the first search area, if the difference between the distance indicated by the object information and the distance newly detected by the ranging device is equal to or less than a threshold, the control device determines whether or not an object exists using a second search area expanded from the first search area. As a result, even if there is a difference between the object detection position by the camera and the object detection position by the ranging device, if the difference is equal to or less than the threshold, such difference can be tolerated and object detection can be achieved using both the camera and the ranging device, thereby suppressing a decrease in the stability of object detection. [Brief explanation of the drawings]

[0008] [Figure 1]1 is a block diagram showing a schematic configuration of a control device according to an embodiment of the present invention; [Figure 2] 10 is a flowchart illustrating a procedure for object recognition processing according to the present embodiment. [Figure 3] 10 is a flowchart illustrating a procedure for information acquisition processing according to the present embodiment. [Figure 4] FIG. 10 is an explanatory diagram showing an example of the process in step S170. [Figure 5] FIG. 10 is an explanatory diagram showing an example of the process in step S170. [Figure 6] 10 is a flowchart illustrating a procedure for object determination processing according to the present embodiment. [Figure 7] FIG. 10 is an explanatory diagram showing an example of processing in step S210. [Figure 8] FIG. 10 is an explanatory diagram showing an example of a situation in which a deviation in the detected position of an object occurs. [Figure 9] FIG. 10 is an explanatory diagram showing an example of the process in step S240. [Figure 10] FIG. 10 is an explanatory diagram showing the procedure of information update processing according to the present embodiment. [Figure 11] FIG. 4 is an explanatory diagram showing the procedure of the start suppression process according to the present embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] A. Implementation: A-1.Device configuration: As shown in FIG. 1, in this embodiment, the vehicle M includes a camera 110, a distance measuring device 120, and a control device 200.

[0010] The camera 110 captures an image of the area ahead of the vehicle M. In this embodiment, the vehicle M is equipped with a monocular camera as the camera 110. Note that the vehicle M may be equipped with a stereo camera as the camera 110 instead of the monocular camera.

[0011] The distance measuring device 120 detects the presence or absence of an object in the traveling direction of the vehicle M and the distance between the object and the vehicle M. The object includes, for example, a guardrail, a building such as a house, or a wall such as a block wall. In this embodiment, the vehicle M is equipped with a millimeter-wave radar as the distance measuring device 120. Note that the vehicle M may be equipped with a LiDAR (Light Detection and Ranging), an ultrasonic sensor, or the like as the distance measuring device 120, without being limited to the millimeter-wave radar.

[0012] The control device 200 is configured as a computer including a CPU 210, a ROM 220, and a RAM 230. An example of such a computer is an ECU (Electronic Control Unit). In this embodiment, the ROM 220 is configured as a rewritable non-volatile memory. The ROM 220 also has an information storage unit 221 that stores information indicating the presence or absence of an object detected by the distance measuring device 120 and the distance between the object and the vehicle M (hereinafter also referred to as "object information").

[0013] The control device 200 executes a pre-installed program to function as an information acquisition unit 211, an object determination unit 212, an information update unit 213, and a departure suppression unit 214. The information acquisition unit 211 acquires an image captured by the camera 110, a distance detected by the distance measuring device 120, and object information stored in the information storage unit 221, and acquires a point cloud representing the position of the object from the acquired image. The object determination unit 212 determines whether an object exists using the acquired point cloud and object information. If the object determination unit 212 determines that an object exists and the vehicle M is stopped, the information update unit 213 writes the newly acquired object information into the information storage unit 221. If the object determination unit 212 determines that an object exists, the departure suppression unit 214 controls the brakes to suppress the departure of the vehicle M.

[0014] A-2. Object Recognition Processing: The control device 200 repeatedly executes the object recognition process shown in FIG. 2 while the ignition of the vehicle M is on. By executing this process, the control device 200 determines whether or not an object exists in the traveling direction of the vehicle M, and updates information about the object. In the object recognition process, the control device 200 executes an information acquisition process (step S100), an object determination process (step S200), an information update process (step S300), and a start suppression process (step S400). In this embodiment, the control device 200 executes step S300 and step S400 in parallel. Note that the control device 200 may execute step S400 after step S300.

[0015] A-2-1. Information acquisition process: The information acquisition unit 211 executes the information acquisition process shown in Fig. 3. In step S110, the information acquisition unit 211 determines whether or not object information is stored in the information storage unit 221. If the object information is stored in the information storage unit 221 (step S110: Yes), the information acquisition unit 211 acquires the object information from the information storage unit 221 (step S120) and deletes the object information stored in the information storage unit 221 (step S130). By deleting the object information that has been referenced once, it is possible to avoid erroneously using object information that has become detached from the actual situation over time for determination. If the object information is not stored in the information storage unit 221 (step S110: No), the information acquisition unit 211 executes step S160, which will be described later.

[0016] If the vehicle M moves a preset distance after acquiring the object information (step S140: Yes), the information acquisition unit 211 erases the acquired object information (step S150). This is to avoid erroneously using object information that deviates from the actual situation due to the movement of the vehicle M for judgment. If the vehicle M does not move a preset distance (step S140: No), the information acquisition unit 211 retains the object information and executes step S160, which will be described later.

[0017] In step S160, the information acquisition unit 211 acquires the image captured by the camera 110 and the distance detected by the distance measuring device 120. In the following description, the distance acquired in this step will also be referred to as the "current distance."

[0018] In step S170, the information acquisition unit 211 acquires a point cloud representing the position of an object from the acquired image. The information acquisition unit 211 extracts the object from the image using an image analysis technique such as semantic segmentation, and detects the contour of the object to acquire the point cloud.

[0019] Acquisition of a point cloud by the information acquisition unit 211 of this embodiment will be described in more detail. As shown in Fig. 4, the information acquisition unit 211 acquires a point cloud targeting an object Ob present in a processing area ARp of the image IM, excluding a non-processing area ARe indicated by hatching. The information acquisition unit 211 acquires, as a point cloud, a plurality of points P of the object Ob on the boundary line between the object Ob and the ground surface Gr (hereinafter also referred to as the "lower end of the object Ob"). Furthermore, as shown in Fig. 5, when the distance between the object Ob and the vehicle M is short and the lower end of the object Ob is outside the processing area ARp, the object determination unit 212 acquires, as a point cloud, a plurality of points P of the object Ob on the boundary line between the processing area ARp and the non-processing area ARe (hereinafter also referred to as the "lower end of the processing area ARp").

[0020] Each point P has distance information between each point P and the vehicle M. Such distance information is geometrically calculated using the coordinates of each point P in the image IM, the installation position of the camera 110 on the vehicle M, and the angle of view of the camera 110. The information acquisition unit 211 maps the multiple points P having the distance information calculated in this way on a map, and recognizes the position of the object.

[0021] After step S170 shown in FIG. 3 is completed, the information acquisition unit 211 ends the information acquisition process.

[0022] A-2-2. Object detection process: The object determination unit 212 executes the object determination process shown in Fig. 6. In step S210, the object determination unit 212 determines whether or not at least a part of the acquired point cloud is included in a first search area. The "first search area" refers to an area of ​​a predetermined size that is set at a position according to the current distance. In the following description, the determination conditions in this step are also referred to as "normal conditions."

[0023] If it is determined that at least a portion of the point cloud is included in the first search area ARs1 as shown in FIG. 7 (step S210: Yes), the object determination unit 212 determines that "an object exists under normal conditions" as shown in FIG. 6 (step S250).

[0024] On the other hand, if it is determined that the point cloud is not included in the first search area (step S210: No), the object determination unit 212 determines whether object information has been acquired by the information acquisition unit 211 (step S220: Yes) and whether the difference between the distance to the object in the acquired object information (hereinafter referred to as "memorized distance") and the current distance is equal to or less than a preset threshold (step S230). In this embodiment, the object determination unit 212 determines, through such determination, whether the object related to the object information and the currently recognized object are the same object.

[0025] A case where it is determined that the point cloud is not included in the first search area will be described in more detail. As described above, when the bottom end of the object Ob is outside the processing area ARp as shown in FIG. 5, the information acquisition unit 211 acquires, as a point cloud, multiple points P of the object Ob on the bottom end of the processing area ARp. For example, when the vehicle M parks and approaches a wall, which is the object Ob, and stops, the bottom end of the object Ob may be within the processing area ARp until it is halfway thereafter, but may move out of the processing area ARp when it approaches the wall. In such a situation, when point clouds are acquired for multiple consecutive images IM in time series, even if the bottom end of the object Ob eventually moves out of the processing area ARp, the distance between the object Ob and the vehicle M can be correctly detected by utilizing the history of the bottom end of the object Ob, which was initially within the processing area ARp, moving out of the processing area ARp as the vehicle M approaches. However, when the vehicle M changes from an ignition-off state to an ignition-on state and the first image IM acquired shows that the bottom end of the object Ob is outside the processing area ARp, the information acquisition unit 211 erroneously recognizes that the bottom end of the object Ob is located at the bottom end of the processing area ARp. Therefore, as shown in FIG. 8 , the information acquisition unit 211 erroneously recognizes that the object Ob is located at the position of the object Obf, and maps a point cloud having, as distance information, an erroneous distance Df that is greater than the distance Dt from the object Ob, rather than the correct distance Dt. As described above, the point cloud is mapped at a position that is shifted away from the vehicle M from the actual position of the object, and the point cloud falls outside the first search area. Such a shift in the detected position can be geometrically calculated in advance from the installation position of the camera 110 and the angle of view of the camera 110.

[0026] As shown in FIG. 6, if the difference between the stored distance and the current distance is equal to or smaller than the threshold (step S230: Yes), the object determination unit 212 determines whether at least a portion of the point cloud is included in a second search area ARs2 that is expanded from the first search area in a direction away from the vehicle M, as shown in FIG. 9 (step S240). The amount of expansion of the second search area relative to the first search area can be set arbitrarily depending on the amount of deviation of the detected position calculated in advance. In the following description, the determination condition in step S240 is also referred to as a "relaxed condition." If the difference between the stored distance and the current distance is small and the object related to the object information and the currently recognized object can be considered to be the same object, a determination is made that allows for the amount of deviation of the detected position described above, thereby suppressing a decrease in the stability of object detection.

[0027] As described above, in this embodiment, even if a positional deviation occurs during object detection when the ignition of vehicle M is turned on, if the difference between the stored distance and the current distance is small and the object related to the object information and the currently recognized object can be said to be the same object, object determination unit 212 performs determination using the second search area. This allows object determination unit 212 to tolerate such positional deviation and suppresses a decrease in the stability of object detection.

[0028] As shown in FIG. 6, if it is determined that at least a part of the point cloud is included in the second search area (step S240: Yes), the object determination section 212 determines that "an object exists under relaxed conditions" (step S252).

[0029] If the object information has not been acquired (step S220: No), if the difference between the object information and the current distance is greater than a threshold (step S230: No), or if it is determined that the point cloud is not included in the second search area (step S240: No), the object determination unit 212 determines that "no object exists" (step S254).

[0030] After making a determination at any one of the above-described steps S250, S252, and S254, the object determination section 212 ends the object determination process.

[0031] A-2-3. Information update process: The information update unit 213 executes the information update process shown in Fig. 10. In step S310, the information update unit 213 determines whether the vehicle M is stopped. As described above, the determination using the object information is performed when the ignition of the vehicle M is on. In such a determination, it is preferable to use the object information immediately before the ignition is turned off. Therefore, in this embodiment, the condition for writing the object information to the information storage unit 221 is that the vehicle M is in a stopped state in which there is a high possibility that the vehicle M will transition to an ignition-off state.

[0032] If the vehicle M is stopped (step S310: Yes), the information update unit 213 determines whether or not a determination that "an object exists under normal conditions" has been made in the above-described object determination process (step S320). This is because a determination under normal conditions has a higher reliability of the determination result than a determination under relaxed conditions that allow for positional deviation.

[0033] If it is determined that "an object exists under normal conditions" (step S320: Yes) and the object information is not stored in the information storage unit 221 (step S330: No), or if the object information is stored in the information storage unit 221 (step S330: Yes) and the difference between the stored distance and the current distance is greater than or equal to a predetermined threshold (step S340: Yes), the information update unit 213 writes the current distance to the information storage unit 221 (step S350) and terminates the information update process.

[0034] If the vehicle M is not stopped (step S310: No), if it is not determined that an object exists under normal conditions (step S320: No), or if the difference between the stored distance and the current distance is less than the threshold value (step S340: No), the information update unit 213 terminates the information update process without writing the object information to the information storage unit 221.

[0035] A-2-4. Start suppression processing: The start suppression unit 214 performs the start suppression process shown in Fig. 11. When the driver is about to start the vehicle M (step S410: Yes), and when it is determined in the object determination process that "an object exists" under either the normal condition or the relaxed condition (step S420: Yes), the start suppression unit 214 controls the brakes to suppress the start of the vehicle M (step S430), and ends the start suppression process.

[0036] If the vehicle M is stopped (step S410: No), or if the above-mentioned object determination process determines that "no object exists" (step S420: No), the departure suppression unit 214 terminates the departure suppression process without executing step S430.

[0037] According to the control device 200 of the present embodiment described above, even if the point cloud acquired from the image is not included in the first search area, if the difference between the stored distance and the current distance is equal to or less than the threshold, the presence or absence of an object is determined using a second search area expanded from the first search area. As a result, even if there is a difference between the object detection position by the camera 110 and the object detection position by the ranging device 120, if the difference is equal to or less than the threshold, such a difference can be tolerated and object detection can be achieved using both the camera 110 and the ranging device 120, thereby suppressing a decrease in the stability of object detection.

[0038] Furthermore, the control device 200 stores information indicating the presence or absence of an object in the information storage unit 221 in addition to information indicating the distance between the object and the vehicle M, and therefore the information indicating the presence or absence of an object can be used as an activation condition for the start suppression control by the start suppression unit 214. Therefore, the start suppression process can be prevented from becoming complicated compared to, for example, a case where the presence or absence of an object is determined based on the relationship between the detected distance and a threshold value.

[0039] In addition, after acquiring the object information, the information acquisition unit 211 erases the object information stored in the information storage unit 221, thereby preventing the object information that deviates from the actual situation due to the movement of the vehicle M from being mistakenly used for judgment.

[0040] B. Other Embodiments: (B1) In the above embodiment, the control device 200 is mounted on the vehicle M, but the present disclosure is not limited to this. The control device 200 may be mounted on any type of moving body, such as an aircraft or a robot, without being limited to the vehicle M. Even in such a configuration, the same effects as those of the above embodiment can be achieved.

[0041] (B2) In the above embodiment, the control device 200 includes the information storage unit 221, and the information acquisition unit 211 acquires the object information from the information storage unit 221, but the present disclosure is not limited to this. The information acquisition unit 211 may acquire the object information directly from the distance measuring device 120, which includes a non-volatile memory that stores the object information. This configuration also achieves the same effects as the above embodiment.

[0042] (B3) In the above embodiment, the information storage unit 221 stores, as object information, information indicating the presence or absence of an object and the distance between the object and the vehicle M, but the present disclosure is not limited to this. The information storage unit 221 may store only information indicating the distance between the object and the vehicle M. This configuration also achieves the same effects as the above embodiment. In addition, the amount of information stored by the information storage unit 221 can be reduced.

[0043] (B4) In the above embodiment, after acquiring the object information, the information acquisition unit 211 erases the object information stored in the information storage unit 221, but the present disclosure is not limited to this. The information acquisition unit 211 may only overwrite the object information stored in the information storage unit 221. This embodiment also achieves the same effects as the above embodiment.

[0044] The present disclosure is not limited to the above-described embodiments and can be realized in various configurations without departing from the spirit thereof. For example, the technical features in each embodiment corresponding to the technical features in the form described in the Summary of the Invention section can be appropriately replaced or combined to solve some or all of the above-described problems or achieve some or all of the above-described effects. Furthermore, if a technical feature is not described as essential in this specification, it can be appropriately deleted.

[0045] The control device 200 and the methods described herein may be implemented by a special-purpose computer configured with a processor and memory programmed to perform one or more functions embodied in a computer program. Alternatively, the control device 200 and the methods described herein may be implemented by a special-purpose computer configured with a processor configured with one or more dedicated hardware logic circuits. Alternatively, the control device 200 and the methods described herein may be implemented by one or more special-purpose computers configured with a processor and memory programmed to perform one or more functions in combination with a processor configured with one or more hardware logic circuits. Furthermore, the computer program may be stored as instructions executed by a computer on a computer-readable non-transitory storage medium. [Explanation of symbols]

[0046] 110... camera, 120... distance measuring device, 200... control device, 211... information acquisition unit, 212... object determination unit, 214... departure suppression unit, M... vehicle, Ob... object, ARs1... first search area, ARs2... second search area

Claims

1. A control device (200) mounted on a moving body (M) having a camera (110) and a distance measuring device (120), an information acquisition unit (211) that acquires an image captured by the camera in the traveling direction of the moving body and object information indicating a distance between the moving body and an object (Ob) present in the traveling direction of the moving body detected by the distance measuring device, and acquires a point cloud representing the position of the object from the acquired image; an object determination unit (212) that determines whether or not the object exists by using the object information and the point cloud; a start suppression unit (214) that suppresses the start of the moving body when it is determined that the object is present; Equipped with the object determination unit determines that the object is present when the point cloud is included in a first search area (ARs1) of a predetermined size that is set at a position according to the distance indicated by the object information; If the point cloud is not included in the first search area and the difference between the distance indicated by the acquired object information and the distance newly detected by the distance measuring device is equal to or less than a predetermined threshold, a second search area (ARs2) is set that is expanded in a direction away from the moving body, and if the point cloud is included in the second search area, it is determined that the object exists; determining that the object does not exist when the point cloud is not included in either the first search area or the second search area; Control device.

2. The control device according to claim 1, a non-volatile memory (221) for storing the object information; an information updating unit (213) that writes the newly acquired object information into the non-volatile memory when the object determining unit determines that the object exists and the moving body is stopped; Furthermore, The nonvolatile memory further stores information indicating the presence or absence of the object. Control device.

3. The control device according to claim 2, After acquiring the object information, the information acquisition unit erases the object information stored in the nonvolatile memory. Control device.

Images