Observation device, information processing device, and information processing method

Abstract

This observation device includes an observation unit, a storage unit, a communication unit, and a control unit. The observation unit observes objects in an observation region. The storage unit stores a fixed position of the observation unit and the observation region. The communication unit acquires vehicle information including a position of a vehicle from the vehicle. On the basis of the fixed position, the observation region, and the vehicle information, the control unit rearranges the order of the objects in the observation region, to be included in first notification information to be transmitted to an out-of-range vehicle that is in a position outside the observation region and that is heading toward the fixed position, among vehicles transmitting vehicle information, the objects being rearranged in decreasing order of benefit for the out-of-range vehicle.

Description

Observation device, information processing device, and information processing method CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims priority to Japanese Patent Application No. 2022-71904, filed on April 25, 2022, the entire disclosure of which is incorporated herein by reference.

[0002] The present disclosure relates to an observation device, an information processing device, and an information processing method.

[0003] An information provision system has been proposed that generates integrated detection results based on detection information of dynamic objects obtained from multiple detection units installed on the roadside and in the vehicle, and transmits the integrated detection results to the vehicle (see Patent Document 1).

[0004] International Publication No. 2019 / 180978

[0005] An observation device according to a first aspect comprises an observation unit that observes objects within an observation area; a memory unit that stores a fixed position of the observation unit and the observation area; a communication unit that acquires vehicle information from a vehicle including the position of the vehicle; and a control unit that, based on the fixed position, the observation area, and the vehicle information acquired from at least one vehicle via the communication unit, rearranges the order of objects within the observation area to be included in first notification information to be sent to the vehicle, among the vehicles that transmit the vehicle information, in order of the most usefulness to out-of-range vehicles that are located outside the observation area and heading towards the fixed position.

[0006] An information processing device for vehicle use according to a second aspect includes an acquisition unit that acquires first alarm information regarding objects within the observation area of ​​the observation device from the observation device, acquires first vehicle information including the position of the vehicle and detection information of surrounding objects from the vehicle on which the device is installed, and acquires second vehicle information including the position of the vehicle from other vehicles other than the vehicle; and a control unit that identifies the nearest vehicle based on the first vehicle information and the second vehicle information, and rearranges the order of objects in the first alarm information and the detection information to include them in the second alarm information to be sent to the identified vehicle in order of their usefulness to the identified vehicle.

[0007] An information processing method according to a third aspect includes storing a fixed position of an observation unit and an observation area observed by the observation unit, acquiring vehicle information from a vehicle including the position of the vehicle, and, based on the fixed position, the observation area, and the vehicle information acquired from at least one vehicle via the communication unit, rearranging the order of objects within the observation area to be included in first notification information to be sent to the vehicle in order of their usefulness to out-of-range vehicles that are located outside the observation area of ​​the vehicle that transmitted the vehicle information and are heading towards the fixed position.

[0008] An information processing method according to a fourth aspect includes: acquiring first notification information regarding an object within an observation area of ​​an observation device from the observation device; acquiring first vehicle information including the position of the first vehicle from a first vehicle; acquiring second vehicle information including the position of the first vehicle from another vehicle other than the first vehicle; detecting surrounding objects as detection information; identifying the nearest vehicle based on the first vehicle information and the second vehicle information; and rearranging the order of objects in the first notification information and the detection information to be included in second notification information to be sent to the identified vehicle in order of their usefulness to the vehicle.

[0009] Fig. 1 is a diagram showing an example of the configuration of a communication system including an observation device according to an embodiment. Fig. 1 is a block diagram showing a schematic configuration of the observation device of Fig. 1. Fig. 2 is a block diagram showing a schematic configuration of the vehicle of Fig. 1. Fig. 3 is a state diagram for explaining an out-of-range vehicle relative to the observation device. Fig. 4 is a block diagram showing a schematic configuration of the information processing device of Fig. 3. Fig. 5 is a flowchart for explaining a first notification process executed by the control unit of Fig. 2. Fig. 6 is a flowchart for explaining a second notification process executed by the control unit of Fig. 5.

[0010] Hereinafter, an embodiment of an observation device to which the present disclosure is applied will be described with reference to the drawings.

[0011] 1 shows an example of the configuration of a communication system 11 including an observation device 10 according to an embodiment. The communication system 11 is, for example, a safe driving support communication system of an intelligent transport system (ITS). The safe driving support communication system is also called a safe driving support system or a safe driving support wireless system.

[0012] The observation device 10 may be a device that observes an object to be observed, such as a vehicle, an object, or a person on a road. The observation device 10 is, for example, a roadside device, a surveillance camera device, or the like. In this embodiment, the observation device 10 is a roadside device that may be placed near an intersection where multiple roads 12 (roadways) intersect and observe the road surface. For example, the observation device 10 may observe the road surface in the extension direction of the roads 12 connected to the intersection. The observation device 10 may also be placed on the side of the road other than at the intersection.

[0013] In the communication system 11, the observation device 10 and a vehicle 13 such as an automobile traveling on a road 12 may communicate wirelessly with each other. A plurality of vehicles 13 may communicate wirelessly with each other.

[0014] The observation device 10 may transmit first notification information to the vehicle 13 to assist the driver of the vehicle 13 in safe driving. The first notification information includes information about an object to be observed that is present on the road being observed. The information about the object to be observed may include the presence or absence of the object, the type of object present, the distance to the observation device 10, the moving speed, and the traveling direction. The first notification information may include the position of the observation device 10. The position of the observation device 10 may be the same as the fixed position of the observation unit described below.

[0015] The vehicle 13 may periodically transmit vehicle information to the observation device 10 or the like. The vehicle information may include, for example, the position, speed, acceleration, and direction of travel of the vehicle 13. The vehicle 13 may acquire first notification information provided by the observation device 10 or the like using an on-board information processing device mounted on the vehicle 13. The on-board information processing device may be, for example, a car navigation system. The on-board information processing device may support the driver's safe driving by providing a notification such as a warning to the driver based on the first notification information provided by the observation device 10. The notification to the driver may include the type, position, speed, direction of travel, etc. of objects such as other vehicles 13 and pedestrians near the intersection the vehicle 13 is traveling through. As described above, the communication system 11 may support the driver of the vehicle 13 in safe driving. The vehicle 13 is not limited to an automobile. For example, the vehicle 13 may include a motorcycle, a bus, a streetcar, or a bicycle.

[0016] The following describes in detail the observation device 10. As shown in Figure 2, the observation device 10 includes an observation unit 14, a storage unit 15, a communication unit 16, and a control unit 17.

[0017] The observation unit 14 observes an object within a predetermined observation area. The predetermined observation area may be determined based on the measurement accuracy of the object according to the distance from the observation unit 14. The observation unit 14 may include at least one sensor. The sensor may be, for example, a camera such as a visible light camera or an FIR camera that captures far-infrared images, or a ranging sensor such as a millimeter-wave radar, LiDAR, or ultrasonic sonar.

[0018] The visible light camera can capture a normal image (RGB image) of the subject. The FIR camera can capture a far-infrared image according to temperature and can detect the temperature of the subject being observed. The distance measurement sensor may emit a projected wave and measure the distance to a reflection point on the object based on the time from when the projected wave is emitted to when the reflected wave from the object is received.

[0019] The sensor may generate detection information based on the detection of an object. In a configuration in which the sensor is a camera, the detection information may be an image. In a configuration in which the sensor is a distance measuring sensor, the detection information may be a distance distribution including distance values ​​for each direction in which a reflected wave is generated in response to a projected wave such as a millimeter wave, near-infrared light, or sound wave. The sensor may generate the detection information at a predetermined cycle. The generation cycle of the detection information is, for example, 100 ms.

[0020] The observation area in the observation unit 14 may be an area extending in a specific direction from the observation unit 14. The observation area oa may be an area extending within an azimuth of ±90 degrees from a central azimuth when viewed from vertically above. Note that even in an area extending within an azimuth of ±90 degrees, an area beyond a predetermined distance from the observation unit 14 does not have to be included in the observation area oa. The predetermined distance is, for example, the distance at which the observation unit 14 can be observed. The observation area is determined based on the installation position and installation posture at which the observation unit 14 is installed. The observation unit 14 may be installed so that an area of ​​the road 12 connected to the intersection extending in one direction from the intersection is included in the observation area.

[0021] The storage unit 15 includes any storage device, such as a random access memory (RAM) or a read-only memory (ROM), and may store various programs that cause the control unit 17 to function and various information that the control unit 17 uses.

[0022] The storage unit 15 stores the fixed position and observation area of ​​the observation unit 14. The fixed position may be indicated in the form of, for example, latitude and longitude. The observation area may be indicated in the form of, for example, latitude and longitude.

[0023] In a configuration in which the observation unit 14 includes a camera, when a pixel constituting an image acquired from the camera indicates an object on the road surface or floor surface, the storage unit 15 may store a conversion formula or conversion table for converting the coordinates of the pixel in the two-dimensional coordinate system into the coordinates in the world coordinate system. Using the conversion formula or conversion table, the position in the world coordinate system of an object on the road surface or floor surface included in the image acquired from the camera and the distance to the observation unit 14 can be calculated.

[0024] The communication unit 16 may be controlled by the control unit 17 to perform wireless communication with the vehicle 13. The communication unit 16 may perform wireless communication via broadcast communication. The communication unit 16 may acquire vehicle information, for example, by receiving vehicle information from the vehicle 13. The communication unit 16 may acquire surrounding information from the vehicle 13. The surrounding information may include information about objects around the vehicle 13 detected by the vehicle 13, as described below. The surrounding information may include information indicating the range of free space, as described below. Furthermore, the communication unit 16 may, for example, transmit first alarm information to the vehicle 13.

[0025] The communication unit 16 may be configured with a communication circuit and an antenna. The antenna may be, for example, an omnidirectional antenna. The communication unit 16 may perform wireless communication using, for example, the 700 MHz band allocated to ITS. Alternatively, the communication unit 16 may perform wireless communication using, for example, a wireless LAN (Local Area Network).

[0026] The communication unit 16 may perform various processes such as amplification on the signal received by the antenna, and output the processed received signal to the control unit 17. The control unit 17 may perform various processes on the input received signal to acquire information contained in the received signal. The communication unit 16 may perform various processes such as amplification on the information acquired from the control unit 17, and wirelessly transmit the processed transmission signal from the antenna.

[0027] The control unit 17 includes one or more processors and memories. The processor may include a general-purpose processor that loads a specific program to execute a specific function, and a dedicated processor specialized for a specific process. The dedicated processor may include an application-specific integrated circuit (ASIC). The processor may include a programmable logic device (PLD). The PLD may include a field-programmable gate array (FPGA). The control unit 17 may be either a system-on-a-chip (SoC) or a system in a package (SiP), in which one or more processors work together.

[0028] The control unit 17 may control the communication unit 16 to broadcast the first notification information periodically, for example, at a cycle of 100 ms. Therefore, the control unit 17 may periodically generate the first notification information. The generation of the first notification information by the control unit 17 will be described below.

[0029] The control unit 17 may acquire information about an object to be observed within the observation area based on detection information acquired from the observation unit 14. The information may include the presence or absence, type, position, and detection time of the object. Furthermore, the control unit 17 may calculate a change in the position of the object based on multiple pieces of detection information detected consecutively by the observation unit 14, and calculate the traveling direction and moving speed based on the change.

[0030] For example, in a configuration in which the detection information is an image, the control unit 17 detects the presence or absence and type of an object based on an estimation model using pattern matching and machine learning. The control unit 17 also detects the area of ​​the road surface in the image and converts the position of the detected object on the road surface into a position in the world coordinate system, thereby detecting the position of the object.

[0031] Alternatively, for example, in a configuration in which the detection information is a distance distribution, the control unit 17 detects the presence or absence and type of an object based on the size and shape of a group of approximately equidistant azimuths.In a configuration in which the detection information is a distance distribution, the control unit 17 detects the position of an object based on the distance distribution of an object recognized as a single object.

[0032] When the control unit 17 has acquired peripheral information (described later), it determines whether the peripheral information includes information about an object located within the observation area. When the peripheral information includes information about an object located within the observation area, the control unit 17 may extract the information to acquire information about the object located within the observation area. The extracted information may include the type, position, direction of travel, and speed of travel of the object, as well as the time point at which the information about the object was detected.

[0033] The control unit 17 may accumulate information about objects located within the observation area from at least one of the detection information and the peripheral information acquired between the generation of the first alarm signal in the previous cycle and the generation of the first alarm signal in the current cycle. The control unit 17 may merge information about the same object among the accumulated information about the object. For example, when the same object is included in the detection information and the peripheral information, the control unit 17 may use the information with higher detection accuracy as the information about the object. The level of detection accuracy may be determined by any method. Furthermore, when information about the same object is detected at different times in the same cycle, the control unit 17 may apply the latest information as the information about the object.

[0034] The control unit 17 generates a first notification signal by rearranging the order of the objects in the observation area acquired as described above, as will be described below.

[0035] To rearrange the order, the control unit 17 selects an out-of-range vehicle based on the fixed position and observation area of ​​the observation unit 14, and vehicle information acquired from at least one vehicle 13 via the communication unit 16. As shown in Figure 4, an out-of-range vehicle 13or is a vehicle 13 that is located outside the observation area oa of the observation unit 14 and heading towards the fixed position, among the vehicles 13 that transmit vehicle information to the observation device 10.

[0036] Specifically, the control unit 17 compares the position of the vehicle that transmitted the vehicle information with the observation area oa based on the observation area oa and the vehicle information, and determines whether the vehicle is an out-of-range vehicle 13or by recognizing the position and traveling direction of the vehicle that transmitted the vehicle information based on the fixed position and the vehicle information. For example, the control unit 17 may determine whether the vehicle that transmitted the vehicle information is an out-of-range vehicle 13or based on whether the observation area oa is included in the vehicle and whether the fixed position is located in the traveling direction from the position of the vehicle.

[0037] The control unit 17 selects a vehicle determined to be an out-of-range vehicle 13or. The control unit 17 may calculate a priority for the selected out-of-range vehicle 13or. The control unit 17 may calculate the priority of each out-of-range vehicle 13or based on the fixed position of the observation unit 14 and the vehicle information of the out-of-range vehicle 13or. Specifically, the control unit 17 calculates the priority such that the priority of the out-of-range vehicle 13or that approaches the fixed position of the observation unit 14 first is increased among the out-of-range vehicles 13or. Furthermore, the control unit 17 may calculate the priority such that the priority of the out-of-range vehicle 13or located on a road 12 that intersects with the road 12 that overlaps with the observation area oa is increased.

[0038] The control unit 17 calculates the time it takes for the vehicle 13 to move to the closest position to the observation unit 14, based on, for example, the position, speed, acceleration, and direction of travel of the vehicle 13 in the vehicle information, and the fixed position of the observation unit 14. The control unit 17 may use this time to identify the out-of-range vehicle 13or that approaches the fixed position the fastest. The control unit 17 may calculate a priority such that the vehicle that approaches the fixed position the fastest has a higher priority.

[0039] The control unit 17 may further select some of the out-of-range vehicles 13 or out-of-range vehicles 13 based on priority from among the vehicles selected as the out-of-range vehicles 13. For example, the control unit 17 may select the out-of-range vehicles 13 with the highest priority or some of the out-of-range vehicles from among all the out-of-range vehicles. Specifically, the control unit 17 may select one out-of-range vehicle as some of the out-of-range vehicles. More specifically, the control unit 17 may select the out-of-range vehicle with the highest priority.

[0040] The control unit 17 rearranges the order of the objects in the observation area oa in order of increasing usefulness to the selected out-of-range vehicle 13or. When the selected out-of-range vehicle 13or moves near a fixed position, the control unit 17 may rearrange the order of the objects in the observation area oa in order of increasing usefulness to the out-of-range vehicle 13or. The time when the out-of-range vehicle 13or moves near a fixed position and the level of usefulness will be described below.

[0041] The control unit 17 may predict the time when each of the selected out-of-range vehicles 13or will come closest to a fixed position. The control unit 17 predicts the time by calculating the time when each of the out-of-range vehicles 13or will come closest to a fixed position based on, for example, the position, speed, acceleration, and traveling direction of the out-of-range vehicle 13or in the vehicle information and the fixed position of the observation unit 14.

[0042] The control unit 17 may evaluate the usefulness of each object in the observation area oa to be included in the first alarm signal. The control unit 17 may evaluate the usefulness so that the closer the predicted position of each object at the time of prediction is to a fixed position, the higher the usefulness. The control unit 17 may calculate the predicted position based on the position, traveling direction, moving speed, and detection time of the object included in the detection signal and the surrounding information.

[0043] The control unit 17 may increase the usefulness of an object that has not yet passed the fixed position at the predicted time. In other words, the control unit 17 does not need to increase the usefulness or evaluate the usefulness of an object that passes the fixed position between the detection time and the predicted time. The control unit 17 may not include objects with low usefulness or objects for which the usefulness is not evaluated in the first notification information.

[0044] The control unit 17 generates the first notification information by rearranging the objects in the observation area OA in order of usefulness. The control unit 17 does not need to include in the first notification information objects that have already passed the fixed position at the time when the control unit 17 predicts that the out-of-range vehicle 13OR will be closest to the fixed position. The out-of-range vehicle 13OR can determine which information is most useful to the out-of-range vehicle 13OR from the order of the objects in the observation area OA included in the received first notification information. Then, when assisting the driver in safe driving or performing autonomous driving, the out-of-range vehicle 13OR can perform processing such as preferentially using information in the order of usefulness (e.g., up to the fifth place) and not using (e.g., discarding) information in the order of usefulness, thereby improving safety and reducing processing load.

[0045] If the peripheral information provided by the out-of-range vehicle 13OR includes a free space range, the control unit 17 may rearrange the objects in the observation area OA based on the free space range. Specifically, the control unit 17 calculates the average area of ​​the free space from the free space range included in the peripheral information provided by the out-of-range vehicle 13OR traveling in the same direction. If the calculated average value is equal to or less than the area threshold, the control unit 17 may rearrange the objects in the observation area OA also according to the type of object in the observation area OA. The control unit 17 may rearrange the objects so that the slower the general movement speed of the object type is, the higher the ranking. For example, the control unit 17 may rank pedestrians, bicycles, motorcycles, and automobiles in that order. The control unit 17 may rearrange the objects according to their type and, within the same object, according to the aforementioned order of usefulness, and generate the first alarm information.

[0046] When the average value of the area of ​​the free space is equal to or less than the area threshold, the control unit 17 may determine that congestion has occurred on the road 12 that constitutes the free space. When determining that congestion has occurred, the control unit 17 may include information that congestion has occurred on the road 12 in the first notification information.

[0047] The control unit 17 may include the fixed position of the observation unit 14 in the first notification information. The control unit 17 may include the fixed position in a dedicated area or a free area in the communication standard used for wireless communication with the vehicle 13.

[0048] As shown in FIG. 3, the vehicle 13 may include a sensor 18, a position detection device 19, a communication unit 20, and an on-board information processing device 21.

[0049] The sensor 18 may include a speed sensor, an acceleration sensor, a steering angle sensor, a timer, a camera, and a distance measurement sensor. The speed sensor detects the moving speed of the vehicle 13 on which the sensor 18 is installed. The acceleration sensor detects the acceleration of the vehicle 13 on which the sensor 18 is installed. The steering angle sensor detects the steering angle of the vehicle 13 on which the sensor 18 is installed. The timer detects the time. The camera captures the scene around the vehicle 13 on which the sensor 18 is installed and generates an image. The distance measurement sensor detects the distance to various object points within the distance measurement direction. The sensor 18 may provide the detected information to the information processing device 21.

[0050] The position detection device 19 detects the position of the vehicle 13 on which it is installed. The position detection device 19 is, for example, a positioning device such as a Global Navigation Satellite System (GNSS).

[0051] The communication unit 20 may perform wireless communication with the observation device 10 and other vehicles 13 of the vehicle 13 on which it is installed. The communication unit 20 may perform wireless communication by broadcast communication. The communication unit 20 may acquire first notification information from the observation device 10. The communication unit 20 may acquire vehicle information and surrounding information from other vehicles 13. The communication unit 20 may transmit the vehicle information and surrounding information to the observation device 10. The communication unit 20 may transmit second notification information, which will be described later, to other vehicles 13.

[0052] The communication unit 20 may be configured with a communication circuit and an antenna. The antenna may be, for example, an omnidirectional antenna. The communication unit 20 may perform wireless communication using, for example, the 700 MHz band allocated to ITS. Alternatively, the communication unit 20 may perform wireless communication using, for example, a wireless LAN.

[0053] The communication unit 20 may perform various processes such as amplification on a signal received by the antenna, and output the processed received signal to the information processing device 21. The information processing device 21 may perform various processes on the input received signal to acquire information contained in the received signal. The communication unit 20 may perform various processes such as amplification on information acquired from the information processing device 21, and wirelessly transmit the processed transmission signal from the antenna.

[0054] As shown in FIG. 5, the information processing device 21 includes an acquisition unit 22 and a control unit 23 .

[0055] The acquisition unit 22 acquires first notification information from the observation device 10 via the communication unit 20. The acquisition unit 22 acquires detection information indicating the detection results of objects around the vehicle 13 on which the information processing device 21 is installed, specifically from the sensor 18. The acquisition unit 22 acquires first vehicle information from the vehicle 13 on which the information processing device 21 is installed (hereinafter, also referred to as the "first vehicle"). Specifically, the acquisition unit 22 acquires, as the first vehicle information, the travel speed and steering angle from the sensor 18 in the vehicle 13 and the position of the vehicle 13 from the position detection device 19 in the vehicle 13. The acquisition unit 22 may acquire second vehicle information of the vehicle 13 from another vehicle 13 (hereinafter, also referred to as the "second vehicle") around the vehicle 13 on which the information processing device 21 is installed, via the communication unit 20.

[0056] The acquisition unit 22 may be a communication interface that communicates information with an external device of the information processing device 21 .

[0057] The control unit 23 includes one or more processors and memories. The processor may include a general-purpose processor that loads a specific program to execute a specific function, and a dedicated processor specialized for a specific process. The dedicated processor may include an application-specific IC. The processor may include a programmable logic device. The PLD may include an FPGA. The control unit 23 may be either an SoC in which one or more processors work together.

[0058] The control unit 23 may provide the first vehicle information to the observation device 10 via the communication unit 20.

[0059] The control unit 23 may acquire information about an object to be observed around the vehicle 13 equipped with the information processing device 21, based on the detection information acquired via the acquisition unit 22. The information may include the presence or absence, type, position, and detection time of the object. Furthermore, the control unit 23 may calculate a change in the position of the object based on multiple pieces of detection information detected consecutively by the sensor 18, and calculate the traveling direction and movement speed based on the change.

[0060] In a configuration in which the detection information is at least one of an image and a distance distribution, the control unit 23 may detect the presence / absence, type, and position of an object, similar to the control unit 17 of the observation device 10. Note that the position and observation direction of the sensor 18 of the vehicle 13 are different from the fixed position and observation direction of the observation unit 14, and therefore the objects acquired as information by the control unit 23 may include an object obbl located in a blind spot from the observation unit 14 within the observation area oa, as shown in FIG.

[0061] The control unit 23 may calculate the range of the free space based on the detection information. In a configuration in which the detection information is an image, the control unit 23 may estimate the free space based on color matching and an estimation model using machine learning, and calculate the range of the free space. In a configuration in which the detection information is a distance distribution, the control unit 23 may estimate the range of the free space by determining a group of observation directions that have the same distance distribution as the distance to the road surface, which is determined by design, for each observation direction toward the ground, and calculate the range of the free space.

[0062] The control unit 23 may generate surrounding information including objects as information acquired based on the detection information. Therefore, the surrounding information includes information about objects around the vehicle 13 detected by the vehicle 13. The surrounding information may also include an object obbl located in a blind spot from the observation unit 14 within the observation area oa. The control unit 23 may include a range of free space in the surrounding information. The control unit 23 may provide the surrounding information to the observation device 10 via the communication unit 20. The control unit 23 may provide the surrounding information to surrounding vehicles 13 via the communication unit 20.

[0063] When the control unit 23 detects an object as information based on the detection information, the control unit 23 may generate information according to the functions installed in the vehicle 13. When the control unit 23 acquires first alarm information from the observation device 10, the control unit 23 may generate information according to the functions of the equipment installed in the vehicle 13. When the control unit 23 acquires surrounding information from surrounding vehicles 13, the control unit 23 may generate information according to the functions of the equipment installed in the vehicle 13.

[0064] For example, in a configuration used to alert the driver to a surrounding object, the control unit 23 may generate a warning based on at least one of object information based on the detection information, the first notification information, and the surrounding information. The control unit 23 may provide the warning to an output device such as a display or a speaker mounted on the vehicle 13.

[0065] When the control unit 23 acquires the first notification information, the control unit 23 may generate the second notification information. The generation of the second notification information by the control unit 23 will be described below.

[0066] The control unit 23 may accumulate information about the object to be observed from the first alarm information acquired between the time when the previous second alarm information was provided to the surrounding vehicles 13 and the time when the first alarm information is acquired, and from at least one of the detection information and the peripheral information acquired from the surrounding vehicles 13. The control unit 23 may merge information about the same object among the accumulated information about the object. For example, when the same object is included in the first detection information, the detection information, and the peripheral information, the control unit 23 may use information with higher detection accuracy as information about the object. Furthermore, when information about the same object is detected at different times, the control unit 23 may apply the latest information as information about the object.

[0067] The control unit 23 generates a second alarm signal by rearranging the order of objects in the observation area oa acquired as described above, as described below. To rearrange the order, the control unit 23 identifies the vehicle 13 closest to the first vehicle 13 based on the first vehicle information and the second vehicle information. The control unit 23 may exclude the second vehicle 13 that is closer than the first vehicle 13 to the fixed position of the observation unit 14 included in the first alarm information from the specified vehicle 13 closest to the first vehicle 13.

[0068] The control unit 23 rearranges the order of the objects in the first alarm information and the objects in the detection information in order of decreasing usefulness to the identified vehicle 13. The control unit 23 may rearrange the objects, including the objects in the peripheral information, in order of decreasing usefulness to the identified vehicle 13.

[0069] The control unit 23 may evaluate, based on any criteria, the usefulness of each object in the first alarm information and the detection information for the identified vehicle 13. For example, the control unit 23 may evaluate the usefulness of each object so that the closer it is to the first vehicle 13, the higher the usefulness. The control unit 23 generates the second alarm information by rearranging and including the objects in the first alarm information and the detection information in order according to the level of usefulness.

[0070] When the first notification information includes information about a traffic jam, the control unit 23 may determine, based on the first vehicle information, whether the first vehicle 13 is located on a road 12 where a traffic jam is occurring. When the first vehicle 13 is located on a road 12 where a traffic jam is occurring, the control unit 23 may also rearrange the objects in the first notification information and the detection information according to the type of each object. The control unit 23 may rearrange the objects so that the slower the general movement speed of the object type is, the higher the ranking. The control unit 23 may generate the second notification information by rearranging the objects according to the type of object and, within the same object, rearranging them in the order of the aforementioned usefulness. Furthermore, when the first notification information includes information about a traffic jam, the control unit 23 may limit the number of object information items to be included in the second notification information. When limiting the information to be included, the control unit 23 may notify other vehicles 13 of the information restriction via individual wireless communication, and may set the restriction to be requested. Furthermore, when limiting the information to be included, the observation device 10 may include in the first notification information, along with information indicating that a traffic jam has occurred, a request to limit the number of objects to be included in the second information, and a specification of the number of objects to be included.

[0071] The control unit 23 may control the communication unit 20 to transmit the generated second alarm information to surrounding vehicles 13 by broadcast communication. The vehicle 13 closest to the first vehicle 13 can determine which information is more useful from the order of objects included in the received second alarm information. Then, when assisting the driver in safe driving or performing automated driving, it becomes possible to perform processing such as preferentially using information in the order of more usefulness (e.g., up to the fifth place) and not using (e.g., discarding) information in the order of less usefulness, thereby improving safety and reducing processing load.

[0072] Next, the first notification process executed by the control unit 17 of the observation device 10 in this embodiment will be described with reference to the flowchart of Fig. 6. The first notification process starts periodically.

[0073] In step S100, the control unit 17 accumulates information about the object to be observed located within the observation area oa based on the detection information and the peripheral information. After accumulation, the process proceeds to step S101.

[0074] In step S101, the control unit 17 merges the information about the objects accumulated in step S100. After merging, the process proceeds to step S102.

[0075] In step S102, the control unit 17 identifies the out-of-range vehicle 13or based on the acquired vehicle information. After the identification, the process proceeds to step S103.

[0076] In step S103, the control unit 17 calculates the time required for the out-of-range vehicle 13or identified in step S102 to move to the closest position to the fixed position. After the calculation, the process proceeds to step S104.

[0077] In step S104, the control unit 17 calculates the priority of each of the out-of-range vehicles 13 or identified in step S102 based on the time calculated in step S103. After the calculation, the process proceeds to step S105.

[0078] In step S105, the control unit 17 selects some of the out-of-range vehicles 13or identified in step S102 based on the priority calculated in step S105. After the selection, the process proceeds to step S106.

[0079] In step S106, the control unit 17 calculates the predicted position of each object merged in step S101 at the time when the out-of-range vehicle 13 selected in step S105 or the object moves to the closest position to the fixed position. Furthermore, the control unit 17 evaluates the usefulness of each object based on each predicted position and the fixed position. After the evaluation, the process proceeds to step S107.

[0080] In step S107, the control unit 17 calculates the average value of the area of ​​the free space from the range of the free space included in the surrounding information acquired from the out-of-range vehicle 13or identified in step S102. Furthermore, the control unit 17 determines whether the average value is equal to or less than the area threshold. If the average value is equal to or less than the area threshold, the process proceeds to step S108. If the average value is not equal to or less than the area threshold, the process proceeds to step S109.

[0081] In step S108, the control unit 17 adds the sorting according to the type of object to the sorting conditions when generating the first alarm information. After adding the sorting conditions, the process proceeds to step S109.

[0082] In step S109, the control unit 17 generates first alarm information for each out-of-range vehicle 13or by rearranging the information about the objects merged in step S102 based on the usefulness evaluated in step S106 and, if a rearrangement condition has been added in step S108, the condition. After the generation, the process proceeds to step S110.

[0083] In step S110, the control unit 17 controls the communication unit 16 to transmit the first notification information generated in step S109 by broadcast communication. After the first notification information is transmitted, the first notification process ends.

[0084] Next, the second notification process executed by the control unit 23 of the information processing device 21 in this embodiment will be described with reference to the flowchart of Fig. 7. The second notification process starts periodically.

[0085] In step S200, the control unit 23 accumulates information about the object to be observed based on the detection information and surrounding information acquired from the surrounding vehicles 13. After accumulation, the process proceeds to step S201.

[0086] In step S201, the control unit 23 waits for the generation cycle of the second notification process. After waiting, the process proceeds to step S202.

[0087] In step S202, the control unit 23 determines whether or not the first notification information has been acquired. If the first notification information has been acquired, the process proceeds to step S203. If the first notification information has not been acquired, the second notification process ends.

[0088] In step S203, the control unit 23 merges the information about the object accumulated in step S200 with the information about the object included in the first alarm information whose acquisition was confirmed in step S202. After merging, the process proceeds to step S204.

[0089] In step S204, the control unit 23 identifies the other vehicle 13 that is closest to the vehicle 13 equipped with the information processing device 21, based on the first vehicle information and the second vehicle information acquired after the start of the second notification process. After the identification, the process proceeds to step S205.

[0090] In step S205, the control unit 23 evaluates the usefulness of each object based on the position of the vehicle 13 in the first vehicle information and the position of each object merged in step S203. After the evaluation, the process proceeds to step S206.

[0091] In step S206, the control unit 23 determines whether the vehicle 13 equipped with the information processing device 21 is located on the congested road 12, based on the information on the congested road 12 included in the first notification information whose acquisition was confirmed in step S202. If the vehicle 13 is located on the congested road 12, the process proceeds to step S207. If the vehicle 13 is not located on the congested road 12, the process proceeds to step S208.

[0092] In step S207, the control unit 23 adds the sorting according to the type of object to the sorting conditions when generating the second alarm information. After adding the sorting conditions, the process proceeds to step S208.

[0093] In step S208, the control unit 23 generates second notification information by rearranging the information about the object merged in step S203 based on the usefulness evaluated in step S205 and, if a rearrangement condition has been added in step S207, the condition. After the generation, the process proceeds to step S209.

[0094] In step S209, the control unit 23 controls the communication unit 20 to transmit the second notification information generated in step S210 by broadcast communication. After the second notification information is transmitted, the second notification process ends.

[0095] The observation device 10 of this embodiment configured as described above rearranges the order of objects within the observation area OA to be included in the first notification information to be transmitted to the vehicle 13, based on the fixed position, the observation area OA, and vehicle information acquired from at least one vehicle 13 via the communication unit 16, in order of the most usefulness to an out-of-range vehicle 13OR located outside the observation area OA and heading toward the fixed position, among the vehicles 13 transmitting vehicle information. With this configuration, the observation device 10 can provide nearby vehicles 13 with information that places a low burden on the out-of-range vehicle 13OR that is likely to use the first notification information at the time of acquiring the first notification information. As a result, the observation device 10 reduces the processing load of the vehicle 13 for confirming whether the first notification information is needed.

[0096] Furthermore, the observation device 10 of this embodiment calculates the priority of the out-of-range vehicles 13or, selects some of the out-of-range vehicles 13or based on the priority, and rearranges the order of objects in the observation area oa to be included in the first notification information to be sent to the vehicle 13 in order of the most usefulness to the out-of-range vehicles 13or when some of the out-of-range vehicles 13or move near the fixed position. With this configuration, the observation device 10 can provide nearby vehicles 13 with information that requires a low burden of confirmation of necessity for out-of-range vehicles 13or that have a high priority for using the first notification information at the time of acquiring the first notification information. As a result, the observation device 10 further reduces the processing load of necessity confirmation on the vehicle 13.

[0097] Furthermore, the observation device 10 of this embodiment increases the priority of the out-of-range vehicle 13or that approaches the fixed position the fastest among the out-of-range vehicles 13or. With this configuration, the observation device 10 can generate first notification information suitable for the vehicle 13 that requires the shortest time to process information. Therefore, the observation device 10 can provide first notification information suitable for the vehicle 13 for which a reduction in the processing load of necessity confirmation is particularly required.

[0098] Furthermore, the observation device 10 of this embodiment predicts the time when some out-of-range vehicles 13or will be closest to a fixed position for each out-of-range vehicle 13or, and the closer the vehicle is to the fixed position at the predicted time, the more useful the object to be included in the first notification information. Generally, the closer an object is to the fixed position at the predicted time, the more attention it will attract to vehicles 13 passing near the fixed position. In response to such an event, the observation device 10 configured as described above can include objects in the first notification information as information on the objects in order of the most attention it attracts to some out-of-range vehicles 13or.

[0099] Furthermore, in the observation device 10 of this embodiment, the object for which usefulness is enhanced is an object that has not yet passed the fixed position at the time of prediction. Objects that have already passed the fixed position at the time of prediction generally attract less attention from some out-of-range vehicles than objects that have not yet passed. In response to such an event, the observation device 10 configured as described above can include objects in the first alarm information in order of increasing attention to some out-of-range vehicles 13or.

[0100] Furthermore, the observation device 10 of this embodiment does not include in the first notification information any objects that have already passed through the fixed position at the time of the prediction. With this configuration, the observation device 10 prevents the addition of unnecessary information and reduces the processing load on the vehicle 13 by excluding information that may be unnecessary for some out-of-range vehicles 13or from the first notification information.

[0101] Furthermore, the observation device 10 of this embodiment acquires objects within the observation area oa to be included in the first notification information by detection by the observation unit 14 and extraction from surrounding information acquired from the vehicle 13. With this configuration, the observation device 10 can acquire information on objects detected from multiple viewpoints. Therefore, the observation device 10 can reduce the number of missed detections of objects that are provided as information to the vehicle 13.

[0102] Furthermore, in the observation device 10 of this embodiment, the surrounding information includes an object obbl that is located within the observation area oa in a blind spot from the observation unit 14. With this configuration, the observation device 10 can include information about an object that cannot be detected by the observation unit 14 in the first alarm information.

[0103] The information processing device 21 of this embodiment configured as described above identifies the nearest vehicle 13 based on the first vehicle information and the second vehicle information, and rearranges the order of objects in the first notification information and the detection information to be included in the second notification information to be transmitted to the identified vehicle 13 in order of the most usefulness to the vehicle 13. With this configuration, the information processing device 21 can provide nearby vehicles 13 that do not have the ability to communicate with the observation device 10 with information that places a low burden on the vehicle 13 to confirm whether it is necessary.

[0104] In one embodiment, (1) the observation device comprises an observation unit that observes objects within an observation area; a memory unit that stores a fixed position of the observation unit and the observation area; a communication unit that acquires vehicle information from a vehicle including the position of the vehicle; and a control unit that, based on the fixed position, the observation area, and the vehicle information acquired from at least one vehicle via the communication unit, rearranges the order of objects within the observation area to be included in first alarm information to be sent to the vehicle in order of the most usefulness to out-of-range vehicles that are located outside the observation area and heading towards the fixed position, among the vehicles that send the vehicle information.

[0105] (2) In the observation device of (1) above, the control unit can calculate the priority of the out-of-range vehicles, select some of the out-of-range vehicles based on the priority, and rearrange the order of objects in the observation area to be included in the first alarm information to be sent to the vehicle in order of the most usefulness to the out-of-range vehicles when the out-of-range vehicles move near the fixed position.

[0106] (3) In the observation device of (2) above, the vehicle information may include the speed and direction of travel of the vehicle transmitting the vehicle information, and the control unit may increase the priority of the out-of-range vehicle that approaches the fixed position most quickly among the out-of-range vehicles.

[0107] (4) In the observation device of (3) above, the control unit predicts the time when each of the out-of-range vehicles will be closest to the fixed position, and the closer the vehicle is to the fixed position at the predicted time, the more useful the object to be included in the first notification information can be.

[0108] (5) In the observation device of (4) above, the object whose usefulness is increased by the control unit may be an object that has not yet passed through the fixed position at the time of the prediction.

[0109] (6) In the observation device of (4) or (5) above, the control unit may exclude from the first notification information an object that has already passed the fixed position at the time of the prediction.

[0110] (7) In any of the observation devices (1) to (6) above, the control unit can obtain objects within the observation area to be included in the first alarm information by detection by the observation unit and extraction from surrounding information obtained from the vehicle.

[0111] (8) In the observation device of (7) above, the peripheral information may include an object located in a blind spot from the observation unit within the observation area.

[0112] In one embodiment, (9) an information processing device for vehicle installation includes: an acquisition unit that acquires first alarm information regarding objects within the observation area of ​​the observation device from the observation device, acquires first vehicle information including the position of the vehicle and detection information of surrounding objects from the vehicle on which the device is installed, and acquires second vehicle information including the position of the vehicle from other vehicles other than the vehicle; and a control unit that identifies the nearest vehicle based on the first vehicle information and the second vehicle information, and rearranges the order of objects in the first alarm information and the detection information to include them in the second alarm information to be sent to the identified vehicle in order of their usefulness to the identified vehicle.

[0113] In one embodiment, (10) an information processing method includes storing a fixed position of an observation unit and an observation area observed by the observation unit, acquiring vehicle information from a vehicle including the position of the vehicle, and, based on the fixed position, the observation area, and the vehicle information acquired from at least one vehicle, rearranging the order of objects within the observation area to be included in first notification information to be sent to the vehicle in order of their usefulness to out-of-range vehicles that are located outside the observation area among the vehicle that transmitted the vehicle information and are heading toward the fixed position.

[0114] In one embodiment, (11) an information processing method includes: acquiring first notification information regarding an object within an observation area of ​​an observation device from the observation device; acquiring first vehicle information including the position of the first vehicle from a first vehicle; acquiring second vehicle information including the position of the first vehicle from another vehicle other than the first vehicle; detecting surrounding objects as detection information; identifying the nearest vehicle based on the first vehicle information and the second vehicle information; and rearranging the order of objects in the first notification information and the detection information to be included in second notification information to be sent to the identified vehicle in order of most usefulness to the vehicle.

[0115] The above has described embodiments of the observation device 10 and the information processing device 21. However, embodiments of the present disclosure can also be embodied as a method or program for implementing the device, as well as a storage medium on which a program is recorded (for example, an optical disk, a magneto-optical disk, a CD-ROM, a CD-R, a CD-RW, a magnetic tape, a hard disk, or a memory card).

[0116] Furthermore, the implementation form of the program is not limited to application programs such as object code compiled by a compiler or program code executed by an interpreter, but may also be in the form of a program module incorporated into an operating system. Furthermore, the program may or may not be configured so that all processing is performed solely by the CPU on the control board. The program may also be configured so that part or all of it is executed by another processing unit mounted on an expansion board or expansion unit added to the board as needed.

[0117] The drawings illustrating the embodiments of the present disclosure are schematic, and the dimensional ratios and the like in the drawings do not necessarily correspond to the actual ones.

[0118] Although the embodiments of the present disclosure have been described based on the drawings and examples, it should be noted that those skilled in the art could make various modifications or alterations based on the present disclosure. Therefore, it should be noted that these modifications or alterations are included in the scope of the present disclosure. For example, the functions included in each component can be rearranged so as not to cause logical inconsistencies, and multiple components can be combined or divided into one.

[0119] All of the features described in this disclosure and / or all steps of all of the disclosed methods or processes may be combined in any combination except combinations in which these features are mutually exclusive. Furthermore, each feature described in this disclosure may be replaced by an alternative feature serving the same, equivalent, or similar purpose, unless expressly denied. Thus, unless expressly denied, each disclosed feature is only one example of a generic series of identical or equivalent features.

[0120] Furthermore, embodiments of the present disclosure are not limited to the specific configurations of any of the above-described embodiments, but rather extend to any novel feature or combination thereof described herein, or any novel method or process step or combination thereof described herein.

[0121] In the present disclosure, descriptions such as "first" and "second" are identifiers for distinguishing the configuration. Configurations distinguished by descriptions such as "first" and "second" in the present disclosure can have their numbers exchanged. For example, the first notification information can exchange the identifiers "first" and "second" with the second notification information. The exchange of identifiers is performed simultaneously. The configurations remain distinguished even after the identifier exchange. Identifiers may be deleted. Configurations from which identifiers have been deleted are distinguished by symbols. The descriptions of identifiers such as "first" and "second" in the present disclosure should not be used solely to interpret the order of the configurations or to justify the existence of identifiers with smaller numbers.

[0122] REFERENCE SIGNS LIST 10 Observation device 11 Communication system 12 Road 13 Vehicle 13 or out-of-range vehicle 14 Observation unit 15 Storage unit 16 Communication unit 17 Control unit 18 Sensor 19 Position detection device 20 Communication unit 21 Information processing device 22 Acquisition unit 23 Control unit oa Observation area obbl Object located in blind spot

Claims

1. an observation unit for observing an object within an observation area; A storage unit that stores a fixed position of the observation unit and the observation area; A communication unit that acquires vehicle information including a position of the vehicle from the vehicle; and a control unit that rearranges an order of objects within the observation area to be included in first notification information to be transmitted to the vehicle, based on the fixed position, the observation area, and the vehicle information acquired from at least one vehicle, in order of increasing usefulness to an out-of-range vehicle that is located outside the observation area and heading toward the fixed position, among the vehicles transmitting the vehicle information. Observation equipment.

2. The observation device according to claim 1, The control unit calculates a priority of the out-of-range vehicles, selects some of the out-of-range vehicles based on the priority, and rearranges an order of objects in the observation area to be included in the first notification information in order of the usefulness to the out-of-range vehicles when the some of the out-of-range vehicles move near the fixed position. Observation equipment.

3. The observation device according to claim 2, The vehicle information includes a speed and a traveling direction of a vehicle transmitting the vehicle information, The control unit increases the priority of an out-of-range vehicle that approaches the fixed position earliest among the out-of-range vehicles. Observation equipment.

4. The observation device according to claim 3, The control unit predicts a time when the part of the out-of-range vehicles will be closest to the fixed position for each out-of-range vehicle, and increases the usefulness of the object that is closer to the fixed position at the predicted time. Observation equipment.

5. The observation device according to claim 4, The object for which the usefulness is increased by the control unit is an object that has not yet passed through the fixed position at the predicted time. Observation equipment.

6. The observation device according to claim 4 or 5, The control unit does not include, in the first notification information, an object that has already passed the fixed position at the predicted time point. Observation equipment.

7. The observation device according to any one of claims 1 to 4, The control unit obtains an object in the observation area to be included in the first notification information by detection by the observation unit and extraction from surrounding information obtained from the vehicle. Observation equipment.

8. The observation device according to claim 7, The peripheral information includes an object located in a blind spot from the observation unit within the observation area. Observation equipment.

9. an acquisition unit that acquires from the observation device first notification information related to objects within an observation area of ​​the observation device, acquires from the vehicle on which the observation device is mounted first vehicle information including a position of the vehicle and detection information of surrounding objects, and acquires second vehicle information including the position of the vehicle from other vehicles other than the vehicle; a control unit that identifies a nearest vehicle based on the first vehicle information and the second vehicle information, and rearranges an order of objects in the first notification information and the detection information to include the nearest vehicle in second notification information to be transmitted to the vehicle in order of the degree of benefit to the identified vehicle. An in-vehicle information processing device.

10. storing a fixed position of the observation unit and an observation area by the observation unit; Acquire vehicle information from a vehicle, the vehicle information including a position of the vehicle; Based on the fixed position, the observation area, and the vehicle information acquired from at least one vehicle, a sequence of objects within the observation area to be included in first notification information to be transmitted to the vehicle is rearranged in order of increasing usefulness to an out-of-range vehicle that is located outside the observation area among the vehicles that transmitted the vehicle information and is heading toward the fixed position. Information processing methods.

11. Obtaining first notification information related to an object in an observation area of ​​an observation device from the observation device; Obtaining first vehicle information from a first vehicle, the first vehicle information including a location of the first vehicle; acquiring second vehicle information from a vehicle other than the first vehicle, the second vehicle information including a position of the vehicle; Detect surrounding objects as detection information, Identifying a closest vehicle based on the first vehicle information and the second vehicle information; The order of the objects in the first notification information and the detection information is rearranged so that the objects are included in second notification information to be transmitted to the vehicle in the order of the degree of benefit to the vehicle. Information processing methods.