Observation apparatus and observation method

Abstract

Provided are an observation device and an observation method that enable a vehicle to be appropriately notified, even if recognition results from a plurality of detection devices differ. An observation device (5) notifies a vehicle, the device comprising: a first detection device (50-1) that recognizes an object present in an observation region; a second detection device (50-2) that recognizes an object present in the observation region; and a control unit (53) that, on the basis of a combination of an object recognition result from the first detection device (50-1) and an object recognition result from the second detection device (50-2), determines a recognition result for notification, which is an object recognition result to be included in a notification.

Description

Cross-reference to related applications

[0001] This application claims the priority of Japanese Patent Application No. 2021-010429 (filed on January 26, 2021), and the entire disclosure of the application is incorporated herein by reference for this purpose.

Technical Field

[0002] This disclosure relates to an observation device and an observation method.

Background Art

[0003] In recent years, the development of a safe driving support system has been promoted in which an observation device is installed on the roadside to detect vehicles, pedestrians, etc. and notify approaching vehicles of danger. For example, Patent Document 1 discloses a technique for recognizing an object photographed by a roadside unit, which is a type of observation device, and notifying the outside. Here, in this specification, an object is used as a term including objects, people, and animals.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

[0005] An observation device according to an embodiment of this disclosure is an observation device that notifies a vehicle, and includes a first detection device that recognizes an object existing in an observation area, a second detection device that recognizes the object existing in the observation area, and a control unit that determines a notification recognition result, which is the recognition result of the object to be included in the notification, based on a combination of the recognition result of the object by the first detection device and the recognition result of the object by the second detection device.

[0006] An observation method according to an embodiment of this disclosure is An observation method performed by an observation device that notifies a vehicle, The first step is to recognize objects present in the observation area, A second step of recognizing the object present in the observation area, The process includes a step of determining a notification recognition result, which is the recognition result of the object to be included in the notification, based on a combination of the recognition result of the object obtained in the first step and the recognition result of the object obtained in the second step. [Brief explanation of the drawing]

[0007] [Figure 1] Figure 1 shows an example of the configuration of a communication system equipped with an observation device according to one embodiment. [Figure 2] Figure 2 is an example of a functional block diagram of an observation device according to one embodiment. [Figure 3] Figure 3 shows an example of the configuration of an observation device according to one embodiment. [Figure 4] Figure 4 shows an example of the installation of an observation device according to one embodiment. [Figure 5] Figure 5 is a flowchart showing an example of an observation method according to one embodiment. [Modes for carrying out the invention]

[0008] (Communication system) Figure 1 shows an example configuration of a communication system 1 equipped with an observation device 5 according to one embodiment. The communication system 1 is, for example, a safe driving support communication system for an Intelligent Transport Systems (ITS). A safe driving support communication system is also called a safe driving support system or a safe driving support wireless system.

[0009] Observation device 5 is a device for observing objects on the road, and is, for example, a roadside unit or a surveillance camera. In this embodiment, observation device 5 is a roadside unit and is positioned near an intersection where multiple roads 7 (vehicles) intersect. However, observation device 5 may be positioned on the roadside other than at an intersection. Details of observation device 5 will be described later.

[0010] In communication system 1, the observation device 5 and vehicles 6, such as cars, traveling on the road 7, can communicate wirelessly with each other. Furthermore, multiple vehicles 6 may be able to communicate wirelessly with each other. Communication system 1 may also be able to communicate wirelessly with electronic devices carried by pedestrians 9. These electronic devices may be, for example, mobile devices such as smartphones.

[0011] The observation device 5 can notify vehicle 6 of safe driving support information to assist the driver of vehicle 6 in safe driving. The safe driving support information may include, for example, information regarding the illumination of traffic lights 4, information regarding road regulations, and road alignment information indicating the shape of the intersection where the observation device 5 is located (shape of road 7). The observation device 5 also detects nearby vehicles 6 and pedestrians 9. For example, the observation device 5 can detect a pedestrian 9 crossing a crosswalk 3. The observation device 5 can detect vehicles 6 approaching an intersection. The observation device 5 can include information about the detected vehicles 6 and pedestrians 9 in the safe driving support information and notify vehicle 6. The observation device 5 can also include information notified by vehicle 6 in the safe driving support information and notify other vehicles 6. Furthermore, as described above, the observation device 5 may notify electronic devices carried by pedestrians 9 of this information. In other words, the observation device 5 may notify mobile devices including in-vehicle equipment and mobile terminal devices.

[0012] In this embodiment, the observation device 5 notifies the vehicle 6 of safe driving support information. In this embodiment, the notification also includes a notification recognition result, which is the recognition result of an object present in the observation area. As will be described in detail later, the notification recognition result may be a name indicating whether the object was recognized as a pedestrian 9 or as a vehicle 6 of a specific type (such as a passenger car, truck, motorcycle, or bicycle). Here, the observation area is the area that the observation device 5 observes. The vehicle 6 can receive various information notified from the observation device 5, etc., through its onboard electronic equipment. The electronic equipment installed in the vehicle 6 may be, for example, a car navigation system. Based on the notification from the observation device 5, the electronic equipment installed in the vehicle 6 can support the driver's safe driving by issuing warnings, etc.

[0013] In this way, the communication system 1 assists the driver of vehicle 6 in safe driving. Here, vehicle 6 is not limited to the passenger cars, trucks, motorcycles, and bicycles mentioned above. For example, vehicle 6 may include buses, trams, and the like.

[0014] (Functional block diagram of the observation device) Figure 2 is a functional block diagram of the observation device 5 according to this embodiment. The observation device 5 comprises a plurality of detection devices 50, a communication unit 51, a storage unit 52, and a control unit 53. The plurality of detection devices 50 consists of N detection devices 50, from the first detection device 50-1 to the Nth detection device 50-N, where N is an integer of 2 or more. The plurality of detection devices 50 includes at least the first detection device 50-1 and the second detection device 50-2.

[0015] The detection device 50 is a device that detects an object existing in the observation area and recognizes what the object is based on the detection result. The detection device 50 includes a sensor 501 and an object recognition unit 502. At least a part of the plurality of detection devices 50 may be a visible light camera. The visible light camera may include, for example, a CMOS (complementary metal oxide semiconductor) image sensor capable of imaging in the visible light range as the sensor 501. At least a part of the plurality of detection devices 50 may be an infrared camera. The types of the plurality of detection devices 50 may be the same or different from each other. The infrared camera may include, for example, an infrared sensor as the sensor 501. Here, the sensor 501 is not limited to a CMOS image sensor and an infrared sensor.

[0016] Based on the detection result of the sensor 501, the object recognition unit 502 of the detection device 50 recognizes what the object existing in the observation area is. The object recognition unit 502 may include, for example, one or more processors. For example, when the detection device 50 is a visible light camera or an infrared camera, an object included in the captured image may be extracted, and what the object is may be recognized from features such as the shape of the object. The object recognition unit 502 may use a known object recognition method. For example, the object recognition unit 502 may perform object recognition using a pre-trained deep learning model.

[0017] Each of the plurality of detection devices 50 outputs a recognition result of what the object detected in the observation area is (hereinafter, simply referred to as "recognition result of the object") to the control unit 53. In addition, at least a part of the plurality of detection devices 50 outputs a detection signal detected by the sensor 501 to the control unit 53. In the present embodiment, each of the plurality of detection devices 50 outputs a recognition result of the object to the control unit 53. As another example, a part of the plurality of detection devices 50 may not output a recognition result of the object to the control unit 53, but may output a detection signal detected by the sensor 501 to the control unit 53, and the control unit 53 may perform object recognition based on the detection signal.

[0018] The communication unit 51 is controlled by the control unit 53 to perform wireless communication with the vehicle 6. The communication unit 51 may be composed of a communication circuit and an antenna. The antenna may be, for example, an omnidirectional antenna. The communication unit 51 may perform wireless communication using, for example, the 700 MHz band assigned to ITS. Also, the communication unit 51 may perform wireless communication using, for example, a wireless LAN (Local Area Network).

[0019] The communication unit 51 performs various processes such as amplification processing on the signal received by the antenna, and outputs the processed received signal to the control unit 53. The control unit 53 performs various processes on the input received signal to obtain the information contained in the received signal. Also, the control unit 53 outputs a transmission signal including information to the communication unit 51. The communication unit 51 performs various processes such as amplification processing on the input transmission signal, and wirelessly transmits the processed transmission signal from the antenna.

[0020] In the present embodiment, the communication unit 51 is controlled by the control unit 53 to notify the vehicle 6 of safe driving support information. The notification includes the notification recognition result described later. Also, the notification may include the attention level described later.

[0021] The storage unit 52 may have a function as a memory for storing various information. The storage unit 52 may store, for example, a program executed in the control unit 53, and the result of the process executed in the control unit 53. Also, the storage unit 52 may function as a work memory of the control unit 53. The storage unit 52 can be composed of, for example, semiconductor memories such as ROM and RAM, but is not limited thereto, and can be any storage device. The storage unit 52 may include, for example, a small hard disk drive and an SSD (Solid State Drive).

[0022] In this embodiment, the storage unit 52 stores a first determination table and a second determination table used by the control unit 53 to determine the notification recognition result. The storage unit 52 also stores a third determination table used by the control unit 53 to determine the attention level. Details of the first determination table, the second determination table, and the third determination table will be described later.

[0023] The control unit 53 comprehensively manages the operation of the observation device 5 by controlling other components of the observation device 5. The control unit 53 may include one or more processors. The processors may load programs from accessible memory to implement various functions of the observation device 5. The processors may include at least one of a general-purpose processor that loads a specific program and executes a specific function, and a dedicated processor specialized for a specific process. The dedicated processor may include an Application Specific Integrated Circuit (ASIC). The processor may include a Programmable Logic Device (PLD). The PLD may include a Field-Programmable Gate Array (FPGA). The control unit 53 may include at least one of a System-on-a-Chip (SoC) and a System In a Package (SiP) in which one or more processors cooperate.

[0024] In this embodiment, the control unit 53 performs a process to determine the notification recognition result, which is the recognition result of the object to be included in the notification. In addition, the control unit 53 may calculate the distance to the object and the velocity of the object based on the detection signal from the sensor 501 acquired in order to determine the notification recognition result.

[0025] (Configuration of the observation device) Figure 3 shows an example of the configuration of the observation device 5 according to this embodiment. In the example in Figure 3, the observation device 5 comprises a first detection device 50-1, a second detection device 50-2, and a third detection device 50-3. The observation device 5 may be provided as a roadside observation unit in which these multiple detection devices 50 are attached to a connecting part 57, and the relative positional relationship between the multiple detection devices 50 is defined.

[0026] The connecting portion 57 is a member for attaching multiple detection devices 50. The connecting portion 57 is, for example, a metal frame-shaped member. Here, as shown in Figure 3, a Cartesian coordinate system can be defined for the observation device 5, which is a road observation unit. In this embodiment, the u-axis direction, v-axis direction, and w-axis direction correspond to the directions in which the sides that intersect perpendicularly at the vertices of the connecting portion 57 extend, respectively. As shown in Figure 3, the multiple detection devices 50 may be arranged in the w-axis direction, but are not limited to this. As another example, the multiple detection devices 50 may be arranged in the u-axis direction.

[0027] Figure 4 shows an example of the installation of the observation device 5. The observation device 5 is installed, for example, on a pillar 59 near an intersection where roads 7 to be observed intersect. A connecting part 57 is fixed to the pillar 59, and the orientation of the detection device 50 may be adjusted relative to the connecting part 57. Here, when the observation device 5 is installed to observe the road, the w-axis direction is vertical. In Figure 4, the z-axis direction is vertical, and the x-axis and y-axis indicate horizontal directions perpendicular to the z-axis.

[0028] (Determination of notification recognition result) As described above, the control unit 53 of the observation device 5 performs a process to determine the notification recognition result, which is the recognition result of an object to be included in the notification to the vehicle 6. For example, if a bicycle is approaching the front of the vehicle 6, including the bicycle as a notification recognition result in the notification to the vehicle 6 can prompt the driver to pay attention to the bicycle ahead. Also, for example, if a pedestrian is approaching the front of the vehicle 6, including the pedestrian as a notification recognition result in the notification to the vehicle 6 can prompt the driver to pay attention to the pedestrian ahead. Here, the observation device 5 may be equipped with multiple detection devices 50, as in this embodiment, in order to detect objects present in the observation area without missing any. Each of the multiple detection devices 50 shares part or all of the observation area. Each of the multiple detection devices 50 recognizes an object present in the observation area. If the recognition results for a single object differ among the multiple detection devices 50, confusion may occur if the observation device 5 notifies the driver of the vehicle 6 of the multiple different recognition results as they are. For example, one detection device 50 may recognize an object approaching the front of the vehicle 6 as a bicycle, while another detection device 50 recognizes that object as a pedestrian. In such a case, if both recognition results are notified to the driver of vehicle 6, it will cause confusion as they will not be able to determine whether to pay attention to the bicycle or the pedestrian. Here, whether the objects detected by each detection device 50 are the same object can be determined by the object's position, distance from the observation device 5, or positional relationship. Therefore, the control unit 53 of the observation device 5 according to this embodiment determines the notification recognition result using the method described below and provides an appropriate notification that allows the driver to correctly direct their attention.

[0029] The following describes an example in which a notification recognition result is determined based on a combination of recognition results from two detection devices 50. In the following description, the two detection devices 50 are the first detection device 50-1 and the second detection device 50-2. However, the two detection devices 50 may be any two of the N detection devices 50 from the first detection device 50-1 to the Nth detection device 50-N. Furthermore, the control unit 53 may determine the notification recognition result by executing the method described below in multiple stages. For example, the control unit 53 determines the first stage recognition result based on a combination of the object recognition results from the first detection device 50-1 and the second detection device 50-2. Then, the control unit 53 may determine the second stage recognition result based on a combination of the first stage recognition result and the object recognition result from the third detection device 50-3. By repeating this process, the control unit 53 may determine the final stage recognition result as the notification recognition result. Alternatively, the control unit 53 may determine the recognition result as a notified recognition result based on a combination of the object recognition results from the first detection device 50-1, the second detection device 50-2, and the third detection device 50-3.

[0030] In summary, the observation device 5 first uses a first detection device 50-1 to recognize an object present in the observation area, and a second detection device 50-2 recognizes the same object present in the observation area. Whether or not the objects are identical may be determined based on the object's position, velocity, etc. The recognition result of the first detection device 50-1 and the recognition result of the second detection device 50-2 may be selected from a list of identical recognition results. The control unit 53 acquires the recognition result of the first detection device 50-1 and the recognition result of the second detection device 50-2. Then, the control unit 53 determines the notification recognition result based on these combinations. In the following, the first recognition result refers to the recognition result of the first detection device 50-1, and the second recognition result refers to the recognition result of the second detection device 50-2.

[0031] Next, we will explain the details of the determination table and other components used by the control unit 53 in determining the notification recognition result. When the first recognition result and the second recognition result are the same, the control unit 53 uses the first determination table shown in Table 1 to determine the notification recognition result.

[0032] [Table 1]

[0033] In the first determination table, the leftmost column shows the type of object determined by the first detection device 50-1 and the second detection device 50-2. The types of objects include passenger cars, trucks, motorcycles, bicycles, and pedestrians. Here, the first detection device 50-1 and the second detection device 50-2 determine that the object does not fall under any of these types, and the result is set to "undetermined". In the first determination table, the other columns show the notification recognition result according to the speed of the object. In this embodiment, the speed of the moving object is divided into three categories. The "Speed ​​(Stopped)" column is used when the object is stationary. The "Speed ​​(Slow)" column is used when the speed of the object is slow (for example, less than 50 km / h). The "Speed ​​(Fast)" column is used when the speed of the object is fast (for example, 50 km / h or more). The control unit 53 determines the notification recognition result based on the combination of the type of object indicated by the first recognition result, the type of object indicated by the second recognition result, and the category of the object's speed. Here, the control unit 53 calculates the velocity of the object based on the detection signals acquired from the sensors 501 of the first detection device 50-1 and the second detection device 50-2. The control unit 53 may calculate the velocity of the object based, for example, on the difference (distance traveled) of the object's position in two captured images taken at a predetermined time interval.

[0034] As shown in the first determination table, the control unit 53 uses the object recognition results of the first detection device 50-1 and the second detection device 50-2 almost as the notification recognition result. However, if the recognition results of the two detection devices 50 are indeterminate, the control unit 53 uses "oncoming object" or "obstacle" as the notification recognition result. This allows the driver to be directed to pay the correct attention rather than using the word "indeterminate". Also, the "speed (fast)" column does not include pedestrians. If an object recognized as a pedestrian is recognized as fast, the recognition result may be processed as "indeterminate".

[0035] If the first recognition result and the second recognition result are different, the control unit 53 determines the notification recognition result using the second determination table shown in Tables 2 to 6 below.

[0036] [Table 2]

[0037] [Table 3]

[0038] [Table 4]

[0039] [Table 5]

[0040] [Table 6]

[0041] In the second determination table, the leftmost column indicates the type of object that is the result of the first determination. The types of objects include passenger cars, trucks, motorcycles, bicycles, and pedestrians. If either the first recognition result or the second recognition result is undetermined, the control unit 53 uses the first determination table described above to determine the notification recognition result from the recognition result that is not undetermined. Therefore, the second determination table does not include the case of undetermined. The classifications of "Speed ​​(Stopped)", "Speed ​​(Slow)", and "Speed ​​(Fast)" are the same as in the first determination table.

[0042] As shown in the second determination table, when the first recognition result and the second recognition result differ, the control unit 53 uses a name that encompasses the different objects or a name that is easy to draw the driver's attention as the notification recognition result. For example, if the first recognition result is a passenger car, truck, motorcycle, or bicycle, and the second recognition result does not match, the control unit 53 sets the notification recognition result to an oncoming vehicle. Also, for example, if the first recognition result is a pedestrian and the second recognition result does not match, the control unit 53 sets the notification recognition result to an oncoming vehicle or a stationary object. In this way, when the first recognition result and the second recognition result differ, the control unit 53 also uses a name different from the type of object as the notification recognition result. By using a name that encompasses the different recognized objects or a name that is easy to draw the driver's attention, the control unit 53 can provide appropriate notifications that support safe driving without confusing the driver. Here, the control unit 53 may provide the driver with notifications that place more emphasis on safety as the notification recognition result. For example, if the first recognition result is a pedestrian and the second recognition result is a vehicle such as a passenger car or truck, the control unit 53 may notify the driver that the recognition result is a pedestrian. In a similar case, when notifying a pedestrian of the recognition result, the control unit 53 may notify that the recognition result is an oncoming vehicle. In other words, the second determination table may have different contents depending on the recipient of the notification of the recognition result.

[0043] In this embodiment, the observation device 5 determines the notification recognition result using the second determination table when the distance to an object in the observation area is not short distance. Short distance is a distance shorter than a predetermined distance. The predetermined distance is, for example, 150m, but is not limited to this. For example, the predetermined distance may change depending on the type of recipient of the notification recognition result. Specifically, the predetermined distance may be 70m if the recipient is a smartphone held by a pedestrian, and 150m if the recipient is an electronic device mounted on a vehicle. The control device 53 may calculate the distance to the object based, for example, on the correspondence between the actual distance to the object and its position in the captured image. Such a correspondence may be set when the observation device 5 is installed and stored in the storage unit 52.

[0044] The control unit 53 adopts the recognition result of the more reliable of the first detection device 50-1 and the second detection device 50-2 in order to provide rapid notification when the distance to the object is short. In other words, when the distance to the object is short, the control unit 53 does not use the recognition result of the less reliable object, but determines the notification recognition result based on the recognition result of the more reliable object. Which of the two detection devices 50 is more reliable may be fixed according to the type and performance of the detection device 50. Alternatively, it may vary depending on the situation in which the detection device 50 detects an object. For example, the reliability of the first detection device 50-1 may be fixed as higher than that of the second detection device 50-2. Alternatively, for example, the reliability of the second detection device 50-2 may be set to be higher in poor visibility conditions such as fog or at night, and the reliability of the first detection device 50-1 may be set to be higher in other weather conditions. For example, if the distance to the object is short, and the recognition result of the highly reliable first detection device 50-1 is a pedestrian, but the recognition result of the second detection device 50-2 is something other than a pedestrian, the control unit 53 will set the notification recognition result to a pedestrian.

[0045] Furthermore, in this embodiment, the observation device 5 includes a control unit 53 that determines the attention level based on the distance to the object, the speed of the object, and the notification recognition result determined by the control unit 53 based on the first and second judgment tables, and includes this level in the notification. The attention level is a parameter that represents the degree to which attention should be paid to the object. For example, if the attention level is high, the vehicle 6 needs to take evasive action and decelerate earlier than if the attention level were low. For example, if the attention level is high, the vehicle 6 that receives this as a notification may generate a warning sound or take other actions to more strongly urge the driver's attention.

[0046] The control unit 53 determines the attention level using the third determination table shown in Table 7 below.

[0047] [Table 7]

[0048] For example, if the notification recognition result is a passenger car, and the distance to the object is far (not close) and the object is moving slowly or stopped, the control unit 53 determines the attention level to be low. Also, for example, if the notification recognition result is an obstacle (see Table 1), and the distance to the object is close and the object is moving at a standstill, the control unit 53 determines the attention level to be high. Here, an obstacle that is moving at a standstill is, for example, a fallen tree. The third judgment table shown in Table 7 may contain content according to the recipient of the notification recognition result. For example, an obstacle that is moving at a standstill may have a medium or higher attention level for the vehicle driver, but the attention level may be low when notifying a pedestrian's smartphone.

[0049] (Observation method) Figure 5 is a flowchart showing an observation method according to one embodiment. The observation device 5 determines the notification recognition result to be included in the notification to the vehicle 6 according to the flowchart shown in Figure 5.

[0050] The control unit 53 of the observation device 5 acquires the first recognition result and the second recognition result (step S1).

[0051] The control unit 53 of the observation device 5 calculates the distance to the object and the velocity of the object (step S2).

[0052] If the first recognition result and the second recognition result are the same (Yes in step S3), the control unit 53 of the observation device 5 determines the notification recognition result according to the velocity of the object using the first determination table (step S4).

[0053] The control unit 53 of the observation device 5 determines the notification recognition result from the recognition result that is not uncertain using the first determination table if the first recognition result and the second recognition result are different (No in step S3), and either the first recognition result or the second recognition result is uncertain (Yes in step S5).

[0054] The control unit 53 of the observation device 5 adopts the recognition result of the more reliable object (step S8) if neither the first recognition result nor the second recognition result is uncertain (No in step S5) and the distance to the object is short (Yes in step S7).

[0055] The control unit 53 of the observation device 5 determines the notification recognition result according to the velocity of the object using the second determination table when the distance to the object is long (No. in step S7) (step S9).

[0056] As described above, the observation device 5 and observation method according to this embodiment, with the above configuration and process, can provide appropriate notifications to the vehicle 6 that support safe driving, even when the recognition results of multiple detection devices 50 differ, by determining a name that encompasses the different recognized objects or a name that is easy to draw the driver's attention as the notification recognition result.

[0057] While this disclosure has been described based on the drawings and embodiments, it should be noted that those skilled in the art will find it easy to make various modifications and alterations based on this disclosure. Therefore, it should be noted that these modifications and alterations are included within the scope of this disclosure.

[0058] For example, the type of object determined by the first detection device 50-1 and the second detection device 50-2 may include types of animals. The types of animals may be cats, dogs, raccoons, cows, and deer. In this case, if the determination result of the first detection device 50 is a cat, dog, or raccoon in the first determination table, the notification recognition result may be a small animal. Also, if the determination result of the first detection device 50 is a cow or deer in the first determination table, the notification recognition result may be a large animal. Furthermore, if the determination results of the first detection device 50-1 and the second detection device 50-2 are different types of animals in the second determination table, the notification recognition result may be an animal. However, if the determination result of one detection device 50 is a type of animal, but the determination result of the other detection device 50 is a passenger car, truck, motorcycle, bicycle, or pedestrian, the notification recognition result may be something other than an animal (oncoming vehicle, pedestrian, or stationary object) to strongly alert the driver. [Explanation of symbols]

[0059] 1. Communication System 3 Pedestrian crossing 4. Traffic lights 5. Observation device 6 vehicles 7 road 9 Pedestrians 50 Detection device 50-1 First detection device 50-2 Second detection device 50-3 Third detection device 51 Communications Department 52 Storage section 53 Control Unit 57 Connection part 59 pillars 501 Sensor 502 Object recognition section

Claims

1. An observation device that notifies a moving object, A first detection device that recognizes objects present in the observation area, A second detection device that recognizes the object present in the observation area, The system includes a control unit that determines a notification recognition result, which is the recognition result of the object to be included in the notification, based on a combination of a first recognition result of the object by the first detection device and a second recognition result of the object by the second detection device, Observation device, wherein if the first recognition result and the second recognition result are different, the control unit determines a name that encompasses both the type of object indicated by the first recognition result and the type of object indicated by the second recognition result as the notification recognition result.

2. The observation apparatus according to claim 1, wherein the control unit determines the notification recognition result based on a combination of the type of object indicated by the first recognition result, the type of object indicated by the second recognition result, and the classification of the velocity of the object.

3. The observation device according to claim 1 or 2, wherein the control unit determines the notification recognition result based on the more reliable of the first recognition result and the second recognition result of the object when the distance to the object is shorter than a predetermined distance.

4. The observation apparatus according to any one of claims 1 to 3, wherein the control unit determines a level of attention based on the distance to the object and the velocity of the object and includes it in the notification.

5. The observation device according to any one of claims 1 to 4, wherein the notification recognition result based on a combination of the first recognition result and the second recognition result differs depending on the type of moving object that notifies the notification recognition result.

6. The observation apparatus according to any one of claims 1 to 5, wherein the recognition results of the object by the first detection device and the second detection device are selected from the same candidate recognition results.

7. An observation method performed by an observation device that notifies a moving object, The first step is to recognize objects present in the observation area, A second step of recognizing the object present in the observation area, The process includes a third step of determining a notification recognition result, which is the recognition result of the object to be included in the notification, based on a combination of the recognition result of the object obtained in the first step and the recognition result of the object obtained in the second step, The third step is an observation method in which, if the recognition result of the object performed in the first step differs from the recognition result of the object performed in the second step, the name that encompasses the type of object indicated by the recognition result of the object performed in the first step and the type of object indicated by the recognition result of the object performed in the second step is determined as the notification recognition result.

Images