Support system, support device, support method, and support program

Abstract

To provide a support system for improving attention-arousing performance to a moving body.SOLUTION: In an assistance system for assisting driving by an operator of a host vehicle on which an optical sensor that scans an outside world by light irradiation and a radar sensor that observes the outside world by radio wave irradiation are mounted, a processor acquires scanning point group data Ds representing a plurality of scanning points by the optical sensor and observation point group data Do representing a plurality of observation points by the radar sensor, matches the scanning points of the scanning point group data Ds with the observation points of the observation point group data Do, and sets scanning points matched with observation points for which speed values Vo during movement are recognized, as follows: And displaying and outputting the display point cloud data Dd as the scanning point cloud data Ds colored with the caution color Cw for the moving object having the velocity value Vo.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present disclosure relates to an assistance technology that assists vehicle driving. [Background technology]

[0002] The technology disclosed in Patent Document 1 provides remote control support for the driving of a vehicle equipped with a laser sensor, a type of optical sensor that scans the outside world by irradiating light. This technology performs coloring processing on the laser measurement point cloud obtained by the laser sensor based on images captured by a color digital camera mounted on the vehicle. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2020-170293 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in the technology disclosed in Patent Document 1, the laser measurement point cloud is displayed in the characteristic color of the target, regardless of whether the target represented by the laser measurement point cloud is a moving object that should be particularly warned about by the vehicle. As a result, there is a concern that the ability to draw attention to the moving object is reduced, leading to a viewer of the display overlooking the moving object.

[0005] An object of the present disclosure is to provide an assistance system that enhances attention to a moving object. Another object of the present disclosure is to provide an assistance device that enhances attention to a moving object. Yet another object of the present disclosure is to provide an assistance method that enhances attention to a moving object. Yet another object of the present disclosure is to provide an assistance program that enhances attention to a moving object. [Means for solving the problem]

[0006] The technical means of the present disclosure for solving the problems will be described below. Note that the claims and the reference characters in parentheses in this section indicate the correspondence with the specific means described in the embodiments described later in detail, and do not limit the technical scope of the present disclosure.

[0007] A first aspect of the present disclosure is An assistance system for assisting an operator of a host vehicle (2) equipped with a processor (12), an optical sensor (52s) that scans the outside world by irradiating light, and a radar sensor (52o) that observes the outside world by irradiating radio waves, Acquiring scanning point cloud data (Ds) representing a plurality of scanning points by an optical sensor and observation point cloud data (Do) representing a plurality of observation points by a radar sensor; Matching scanning points of the scanning point cloud data with observation points of the observation point cloud data; The display device is configured to display and output display point cloud data (Dd) as scanning point cloud data in which scanning points matched with observation points at which the moving velocity value (Vo) is recognized are colored with an attention color (Cw) for the moving object (3) of the velocity value.

[0008] A second aspect of the present disclosure is An assistance device for assisting an operator of a host vehicle in driving the host vehicle, the assistance device including a processor (12), an optical sensor (52s) for scanning the outside world by irradiating light, and a radar sensor (52o) for observing the outside world by irradiating radio waves, the assistance device being mountable on a host vehicle (2), Acquiring scanning point cloud data (Ds) representing a plurality of scanning points by an optical sensor and observation point cloud data (Do) representing a plurality of observation points by a radar sensor; Matching scanning points of the scanning point cloud data with observation points of the observation point cloud data; The display device is configured to display and output display point cloud data (Dd) as scanning point cloud data in which scanning points matched with observation points at which the moving velocity value (Vo) is recognized are colored with an attention color (Cw) for the moving object (3) of the velocity value.

[0009] A third aspect of the present disclosure is An assistance method executed by a processor (12) to assist an operator of a host vehicle (2) equipped with an optical sensor (52s) that scans the outside world by irradiating light and a radar sensor (52o) that observes the outside world by irradiating radio waves, the method comprising: Acquiring scanning point cloud data (Ds) representing a plurality of scanning points by an optical sensor and observation point cloud data (Do) representing a plurality of observation points by a radar sensor; Matching scanning points of the scanning point cloud data with observation points of the observation point cloud data; The method includes displaying and outputting display point cloud data (Dd) as scanning point cloud data in which scanning points matched with observation points at which the moving velocity value (Vo) is recognized are colored with an attention color (Cw) for the moving object (3) of the velocity value.

[0010] A fourth aspect of the present disclosure is An assistance program is stored in a storage medium (10) to assist an operator of a host vehicle (2) equipped with an optical sensor (52s) that scans the outside world by irradiating light and a radar sensor (52o) that observes the outside world by irradiating radio waves, the assistance program including instructions for causing a processor (12) to execute the assistance, Acquiring scanning point cloud data (Ds) representing a plurality of scanning points by an optical sensor and observation point cloud data (Do) representing a plurality of observation points by a radar sensor; Matching scanning points of the scanning point cloud data with observation points of the observation point cloud data; The command includes instructions to execute the following: and to display and output the display point cloud data (Dd) as scanning point cloud data in which the scanning points matched with the observation points at which the velocity value (Vo) during movement is recognized are colored with the attention color (Cw) for the moving object (3) of the velocity value.

[0011] In these first to fourth aspects, scanning point cloud data representing multiple scanning points acquired by an optical sensor mounted on a host vehicle and observation point cloud data representing multiple observation points acquired by a radar sensor mounted on the host vehicle are matched after acquisition. Therefore, according to the first to fourth aspects, the display point cloud data is output and displayed as scanning point cloud data in which scanning points matched with observation points whose moving speed values ​​are recognized are colored in a warning color for moving objects with that speed value. This allows the operator of the host vehicle to visually check the display point cloud data and pay attention to the presence of moving objects moving at speed values ​​corresponding to the warning-colored scanning points. Therefore, it is possible to increase the ability to attract attention to moving objects. [Brief explanation of the drawings]

[0012] [Figure 1] FIG. 1 is a block diagram showing the overall configuration of a first embodiment. [Figure 2] FIG. 2 is a schematic diagram showing a traveling environment of a host vehicle to which the first embodiment is applied. [Figure 3] FIG. 1 is a block diagram showing a functional configuration of a support system according to a first embodiment. [Figure 4] 1 is a flowchart showing a support flow according to a first embodiment. [Figure 5] 1 is a graph for explaining a support flow according to the first embodiment. [Figure 6] FIG. 2 is a schematic diagram for explaining a support flow according to the first embodiment. [Figure 7] FIG. 10 is a schematic diagram for explaining a support flow according to a second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0013] Hereinafter, multiple embodiments of the present disclosure will be described with reference to the drawings. Note that corresponding components in each embodiment are designated by the same reference numerals, and redundant description may be omitted. Furthermore, when only a portion of the configuration is described in each embodiment, the configuration of another previously described embodiment may be applied to the remaining portions of the configuration. Furthermore, in addition to the combinations of configurations explicitly stated in the description of each embodiment, configurations of multiple embodiments may be partially combined together even if not explicitly stated, provided that there is no particular problem with the combination.

[0014] (First embodiment) The assistance system 1 of the first embodiment shown in FIG. 1 performs assistance processing to assist the operator of a host vehicle 2 in driving. The host vehicle 2 may be any of a car, truck, bus, autonomous robot, etc., which is a vehicle that can travel on a road, as exemplified in FIG. 2. From a perspective centered on the host vehicle 2, the host vehicle 2 can be said to be an ego-vehicle. Also, from a perspective centered on the host vehicle 2, a moving object 3 can be said to be another road user. Here, the moving object 3 may be at least one of a car, truck, bus, autonomous robot, motorcycle, bicycle, pedestrian, animal, etc., which is a road user that can travel on a road, as exemplified in FIG. 2.

[0015] The operator of the host vehicle 2 may be a driver who is on board the host vehicle 2 and is capable of performing manual driving operations. The operator of the host vehicle 2 may also be a remote operator who is capable of performing manual driving operations or driving commands remotely from an external center outside the host vehicle 2. In either case, the host vehicle 2 is provided with an autonomous driving mode that is classified according to the degree of intervention from the operator in the dynamic driving task. The autonomous driving mode may be realized by autonomous driving control, such as conditional driving automation, high driving automation, or full driving automation, in which the system performs all dynamic driving tasks when activated. The autonomous driving mode may also be realized by advanced driving assistance control, such as driving assistance or partial driving automation, in which the operator performs some or all of the dynamic driving tasks. The autonomous driving mode may be realized by either autonomous driving control or advanced driving assistance control, or by a combination of these, or by switching between them.

[0016] The host vehicle 2 is equipped with a sensor system 5, a communication system 6, a map database 7, and an information presentation system 8, all of which are shown in Figures 1 and 3, along with at least a part of the assistance system 1. Specifically, the sensor system 5 acquires sensing information about the external and internal worlds of the host vehicle 2 that can be used by the assistance system 1. To this end, the sensor system 5 includes an internal sensor 50 and an external sensor 52.

[0017] The internal sensor 50 generates internal information as sensing information from the internal environment of the host vehicle 2. The internal sensor 50 may be a physical quantity detection type that detects a specific physical quantity of motion in the internal environment of the host vehicle 2. The physical quantity detection type internal sensor 50 is, for example, at least one of a driving speed sensor, an acceleration sensor, an inertial sensor, etc. The internal sensor 50 may be an occupant detection type that detects a specific state of an occupant in the internal environment of the host vehicle 2. The occupant detection type internal sensor 50 is, for example, at least one of a Driver Status Monitor (registered trademark), a biological sensor, a seating sensor, an actuator sensor, an in-vehicle equipment sensor, etc.

[0018] The external sensor 52 generates external information as sensing information from the external environment that is the driving environment in which the host vehicle 2 is traveling. The external sensor 52 may be a target detection type that detects targets present in the external world of the host vehicle 2. As the target detection type external sensor 52, the host vehicle 2 is equipped with at least one pair of an optical sensor 52s and a radar sensor 52o. The pair of the optical sensor 52s and the radar sensor 52o are positioned on the host vehicle 2 in a mounted state in which their field of view overlap each other. Here, the pair of the optical sensor 52s and the radar sensor 52o may be positioned on the host vehicle 2 in a mounted state in which the field of view of the radar sensor 52o extends within the field of view of the optical sensor 52s in a horizontal plane view. The pair of the optical sensor 52s and the radar sensor 52o may be positioned on the host vehicle 2 in a mounted state in which the field of view of the radar sensor 52o extends within the field of view of the radar sensor 52o in a horizontal plane view. In addition, when viewed in a horizontal plane, the optical sensor 52s and the radar sensor 52o may be positioned on the host vehicle 2 in a state where their viewing angles are horizontally shifted and partially overlap each other. In addition to the pair of the optical sensor 52s and the radar sensor 52o, the host vehicle 2 may also be equipped with, for example, a camera and / or a sonar as the target detection type external sensor 52.

[0019] 1 and 3 is, for example, a LiDAR (Light Detection and Ranging / Laser Imaging Detection and Ranging) that scans a specific direction (forward in this embodiment) by irradiating light in the outside world of the host vehicle 2. The optical sensor 52s has a light-emitting unit 520s, a scanning unit 521s, and a light-receiving unit 522s.

[0020] The light-emitting unit 520s is mainly composed of a light-emitting element, such as a laser diode, that emits directional laser light in the infrared range. The light-emitting unit 520s projects light in the form of an intermittent pulse beam toward the outside world of the host vehicle 2. The scanning unit 521s is mainly composed of a scanning mirror. The scanning unit 521s reflects the light emitted from the light-emitting unit 520s according to the rotation angle of the scanning mirror, thereby optically scanning the outside world of the host vehicle 2 with the light.

[0021] The light receiving unit 522s is configured by combining an integrated circuit with a light receiving element, such as a SPAD (Single Photon Avalanche Diode), that is highly sensitive to irradiated light. The light receiving unit 522s receives reflected light from targets in the external world after the reflected light is re-reflected by the scanning unit 521s. The light receiving unit 522s outputs scanning point cloud data Ds representing multiple scanning points obtained by scanning targets in the external world based on light receiving signals generated for each of multiple pixels that receive the reflected light in the light receiving element. The output scanning point cloud data Ds includes at least position information for each scanning point as three-dimensional or two-dimensional information. The data values ​​constituting the position information may be distance values ​​and angle values ​​for each scanning point, or may be coordinate values ​​that can be converted into distance values ​​and angle values ​​for each scanning point.

[0022] The radar sensor 52o is, for example, a millimeter wave radar that uses radio waves to observe a specific direction (forward in this embodiment) outside the host vehicle 2. The radar sensor 52o has a transmitting / receiving antenna unit 520o and a transmitting / receiving processing unit 521o.

[0023] The transmitting / receiving antenna unit 520o is mainly composed of an antenna array such as a microstrip antenna. The transmitting / receiving processing unit 521o is composed of an IC chip such as a DSP (Digital Signal Processor) combined with an RF (Radio Frequency) circuit. The transmitting antenna section in the transmitting / receiving antenna unit 520o converts the transmission signal modulated by the transmitting / receiving processing unit 521o into radio waves, thereby generating a transmission wave to be transmitted to the outside of the host vehicle 2. The receiving antenna section in the transmitting / receiving antenna unit 520o receives a reflected wave of the transmission wave from a target in the outside world and converts it into a received signal.

[0024] The transmission / reception processing unit 521o mixes the converted received signal with the transmitted signal and then performs FFT (Fast Fourier Transform) analysis to output observation point cloud data Do representing multiple observation points at which targets in the external world are observed. The output observation point cloud data Do includes at least position information and velocity information for each observation point as three-dimensional or two-dimensional information. The data values ​​constituting the position information may be distance values ​​and angle values ​​for each observation point, or may be coordinate values ​​convertible to distance values ​​and angle values ​​for each observation point. The data values ​​constituting the velocity information may be velocity values ​​as vector quantities for each observation point, or may be velocity values ​​as scalar quantities for each observation point.

[0025] The communication system 6 acquires communication information usable by the assistance system 1 via wireless communication. The communication system 6 may be a positioning type that receives positioning signals from GNSS (Global Navigation Satellite System) satellites present in the external world of the host vehicle 2. The positioning type communication system 6 is, for example, a GNSS receiver. The communication system 6 may be a V2X type that transmits and receives communication signals to and from a V2X system present in the external world of the host vehicle 2. The V2X type communication system 6 is, for example, at least one of a DSRC (Dedicated Short Range Communications) communication device and a cellular V2X (C-V2X) communication device. The communication system 6 may be a terminal communication type that transmits and receives communication signals to and from a terminal present in the internal world of the host vehicle 2. The terminal communication type communication system 6 is, for example, at least one of a Bluetooth (registered trademark) device, a Wi-Fi (registered trademark) device, an infrared communication device, etc.

[0026] The map database 7 stores map information that can be used by the assistance system 1. The map database 7 includes at least one type of non-transitory tangible storage medium, such as a semiconductor memory, a magnetic medium, or an optical medium. The map database 7 may be a database of a locator that estimates the self-state quantities including the self-position of the host vehicle 2. The map database 7 may be a database of a navigation unit that navigates the driving route of the host vehicle 2. The map database 7 may be configured by combining multiple types of these databases.

[0027] The map database 7 acquires and stores the latest map information, for example, by communicating with an external center via a V2X-type communication system 6. Here, the map information is converted into two-dimensional or three-dimensional data representing the driving environment in which the host vehicle 2 travels. In particular, digital data of a high-precision map is preferably used as the three-dimensional map data. The map information may include road information representing at least one of the following: the position, shape, and road surface condition of the road itself. The map information may also include marking information representing at least one of the following: the position and shape of signs and lane markings attached to the road. The map information may also include structure information representing at least one of the following: the position and shape of buildings and traffic lights facing the road.

[0028] The information presentation system 8 presents notification information to the operator of the host vehicle 2. The information presentation system 8 may be a visual stimulation type that stimulates the operator's vision through a display. When the operator is a driver inside the host vehicle 2, the visual stimulation type information presentation system 8 is, for example, at least one of a HUD (Head-Up Display), an MFD (Multi-Function Display), a combination meter, a navigation unit, etc. When the operator is a remote operator outside the host vehicle 2, the visual stimulation type information presentation system 8 is, for example, at least one of a liquid crystal panel, an organic EL panel, an XR unit, etc.

[0029] As shown in Fig. 1, the assistance system 1 is configured to include at least one dedicated computer. The assistance system 1 is connected to a sensor system 5, a communication system 6, a map database 7, and an information presentation system 8 via at least one of, for example, a LAN (Local Area Network) line, a wire harness, an internal bus, or a wireless communication line. When the assistance system 1 is configured with multiple dedicated computers, the connections between these dedicated computers are similar.

[0030] The dedicated computer constituting the assistance system 1 may be a driving control ECU (Electronic Control Unit) that controls the driving of the host vehicle 2. The dedicated computer constituting the assistance system 1 may be a locator ECU that estimates the self-state quantity of the host vehicle 2. The dedicated computer constituting the assistance system 1 may be a navigation ECU that navigates the driving route of the host vehicle 2 based on map information in a map database 7. The dedicated computer constituting the assistance system 1 may be an actuator ECU that controls the driving actuator of the host vehicle 2. The dedicated computer constituting the assistance system 1 may be an HCU (Human Machine Interface (HMI) Control Unit) that controls the presentation of information from an information presentation system 8 in the host vehicle 2. The dedicated computer constituting the assistance system 1 may be a communication ECU that controls the communication system 6 in the host vehicle 2. The dedicated computer constituting the assistance system 1 may be a computer other than the host vehicle 2 that constitutes an external center or mobile terminal that can communicate via, for example, a V2X type communication system 6.

[0031] The dedicated computer constituting the assistance system 1 has at least one memory 10 and one processor 12. The memory 10 is at least one type of non-transitory tangible storage medium, such as a semiconductor memory, a magnetic medium, or an optical medium, that non-temporarily stores computer-readable programs, data, and the like. Here, "storage" may refer to accumulation in which data is retained even when the host vehicle 2 is powered off, or may refer to temporary storage in which data is erased when the host vehicle 2 is powered off. The processor 12 includes at least one type of core, such as a central processing unit (CPU), a graphics processing unit (GPU), a reduced instruction set computer (RISC)-CPU, a data flow processor (DFP), or a graph streaming processor (GSP).

[0032] In the assistance system 1, the processor 12 executes a plurality of instructions included in an assistance program stored in the memory 10 to assist the operator of the host vehicle 2 in driving. In this way, the assistance system 1 constructs a plurality of function blocks to assist the operator of the host vehicle 2 in driving. The plurality of function blocks constructed in the assistance system 1 include a data acquisition block 100, a matching block 110, and a display output block 120, as shown in FIG. 3 .

[0033] The assistance method in which the assistance system 1 assists the operator of the host vehicle 2 in driving is executed in accordance with the assistance flow shown in Fig. 4 by cooperation of these blocks 100, 110, and 120. This assistance flow is executed repeatedly while the host vehicle 2 is running. Note that each "S" in this assistance flow represents each step executed by multiple commands included in the assistance program.

[0034] In S10 of Fig. 4, the data acquisition block 100 (see Fig. 3) acquires the scanning point cloud data Ds and the observation point cloud data Do of the current frame from the optical sensor 52s and the radar sensor 52o, respectively. At this time, it is preferable that the scanning point cloud data Ds and the observation point cloud data Do be data that were generated within an allowable time difference that is permissible for synchronization on the control time axis. Note that if the generation times of the scanning point cloud data Ds and the observation point cloud data Do differ by more than the allowable time difference, synchronization may be achieved by correcting the data value of one of the data Ds, Do to match the other.

[0035] In S20, matching block 110 (see FIG. 3) selects observation points in the observation point cloud data Do acquired in S10 to be matched with the scanning point cloud data Ds acquired in S10. At this time, the observation points to be matched with the scanning point cloud data Ds are set to those observation points in the observation point cloud data Do that have a recognized velocity value Vo during movement.

[0036] Specifically, the matching block 110 in S20 selects an observation point at which a velocity value Vo exceeds both the first threshold Vo1 and the second threshold Vo2 as a main matching target with the scanning point cloud data Ds. At the same time, the matching block 110 in S20 selects an observation point at which a velocity value Vo exceeds the first threshold Vo1 but is equal to or less than the second threshold Vo2 as a sub-matching target with the scanning point cloud data Ds. Here, the first threshold Vo1 is defined as the lower limit value, such as 0.1 m / s, at which the velocity value Vo can be recognized as moving. Meanwhile, the second threshold Vo2 is defined as a value that exceeds the first threshold Vo1 and increases in accordance with the distance value Lo of the observation point in the observation point cloud data Do, as shown in FIG. 5 .

[0037] Under these definitions, at least one observation point representing a moving object 3 moving at a velocity value Vo that exceeds the first threshold Vo1 and exceeds the second threshold Vo2 is subjected to the matching process in S30 shown in Fig. 4 as a main matching target with the scanning point cloud data Ds. At the same time, at least one observation point representing a moving object 3 moving at a velocity value Vo that exceeds the first threshold Vo1 and is equal to or less than the second threshold Vo2 is subjected to the matching process in S30 as a sub-matching target with the scanning point cloud data Ds.

[0038] In S30, the matching block 110 (see FIG. 3) performs a matching process to match each observation point selected as a main matching target and a sub-matching target from the observation point cloud data Do in S20 with one of the scanning points in the scanning point cloud data Ds. Based on the position information of each data Do and Ds, the matching process associates each observation point of the main matching target and the sub-matching target with a scanning point that is in a distance relationship within an allowable range. The matching process may be performed by dividing the field of view of each sensor 52o and 52s into a plurality of three-dimensional voxels or two-dimensional grids that correspond to the dimensions of the data values ​​in each data Do and Ds.

[0039] In S40, the display output block 120 (see FIG. 3) separates and generates three types of data Dsw, Dse, and Dss from the scanning point cloud data Ds on which the matching process was performed in S30. Specifically, the display output block 120 in S40 extracts from the scanning point cloud data Ds all scanning points that are matched with the observation points of the main matching target in S20 and represent moving objects 3 moving at a speed value Vo exceeding the second threshold value Vo2, thereby generating the attention moving object data Dsw. At the same time, the display output block 120 in S40 extracts from the scanning point cloud data Ds all scanning points that are matched with the observation points of the sub-matching target in S20 and represent moving objects 3 moving at a speed value Vo equal to or less than the second threshold value Vo2, thereby generating the excluded moving object data Dse. Furthermore, the display output block 120 in S40 generates static target data Dss by excluding each scanning point of the attention moving body data Dsw and the excluded moving body data Dse from the scanning point cloud data Ds and extracting all scanning points representing static targets whose speed value Vo exceeds the first threshold value Vo1 in a substantially stationary state and is therefore unrecognizable.

[0040] In S50, the display output block 120 (see FIG. 3) performs a coloring process on the data Dsw, Dse, and Dss separated from the scanning point cloud data Ds in S40, and reintegrates the data to display the display point cloud data Dd from the information presentation system 8. Specifically, in S50, the display output block 120 adjusts the set color for displaying the scanning points of the attention moving object data Dsw corresponding to the moving object 3 moving at a speed value Vo exceeding the second threshold value Vo2 as the display point cloud data Dd after reintegration to an attention color Cw that can attract attention. At this time, as shown for the scanning points surrounded by a two-dot chain line in the display point cloud data Dd after reintegration in FIG. 6, the scanning points of the attention moving object data Dsw are colored with a constant attention color Cw, for example, red (Cwf in FIG. 6), regardless of the magnitude of the speed value Vo recognized at the observation point to be the main matching target in S20. The colored cautionary moving object data Dsw is stored in the memory 10 for subsequent reintegration.

[0041] Meanwhile, in S50, the display output block 120 adjusts the set color for displaying scanning points of the excluded moving object data Dse corresponding to moving objects 3 moving at a speed value Vo equal to or less than the second threshold value Vo2 as display point cloud data Dd after reintegration to an exclusion display color Ce, which is different from the caution color Cw. In S50, the display output block 120 also adjusts the set color for displaying scanning points of the static target data Dss corresponding to static targets that are out of recognition and have a speed value Vo exceeding the first threshold value Vo1, indicating movement, as display point cloud data Dd after reintegration to the exclusion display color Ce. In either case, as shown for the scanning points outside the two-dot chain line in the display point cloud data Dd after reintegration in Figure 6, each data Dse, Dss is colored with the exclusion display color Ce, which is set to have a lower saturation and / or brightness than the caution color Cw, for example, achromatic. The colored excluded moving object data Dse and static target data Dss are stored in the memory 10 for subsequent reintegration.

[0042] 4, the display output block 120 in S50 further reintegrates the caution moving object data Dsw, in which the scanning points have been colored in the caution color Cw, with the data Dse and Dss, in which the scanning points have been colored in the exclusion display color Ce, thereby generating display point cloud data Dd as scanning point cloud data Ds that combines the scanning points displayed in either the caution color Cw or the exclusion display color Ce.

[0043] At this time, in addition to the static target data Dss from the current flow, static target data Dss from at least one previous flow (i.e., the previous flow) may also be read from memory 10 and integrated with other data Dsw and Dse from the current flow to form the display point cloud data Dd. However, because there is an execution time difference between the current flow and the previous flow, the data values ​​of the static target data Dss from the previous flow may be corrected to compensate for the amount of movement of the host vehicle 2 up to the current flow. As a result of the correction, the static target data Dss from the previous flow corresponds to the scanning point cloud data Ds from which scanning points corresponding to observation points at which the moving speed value Vo was recognized in the previous frame have been excluded and the amount of movement of the host vehicle 2 has been compensated. Therefore, the attention moving object data Dsw of the current flow, which is the scanning point cloud data Ds in which scanning points are colored corresponding to observation points at which the moving speed value Vo is recognized in the current frame, is integrated with the static target data Dss from the previous flow to form the display point cloud data Dd. As a result of the above, in S50, the display point cloud data Dd is output and displayed from the information presentation system 8, and the current flow ends.

[0044] (Action and effect) The effects of the first embodiment described above will be explained below.

[0045] In the first embodiment, scanning point cloud data Ds representing multiple scanning points acquired by an optical sensor 52s mounted on the host vehicle 2 and observation point cloud data Do representing multiple observation points acquired by a radar sensor 52o mounted on the host vehicle 2 are matched after acquisition. Therefore, according to the first embodiment, scanning points matched with observation points at which a moving speed value Vo is recognized are colored in a warning color Cw for a moving object 3 with the speed value Vo, and display point cloud data Dd is output and displayed. This allows the operator of the host vehicle 2 to visually recognize the display point cloud data Dd and pay attention to the presence of the moving object 3 moving at the speed value Vo corresponding to the scanning point with the warning color Cw. Therefore, it is possible to enhance the ability to attract attention to the moving object 3.

[0046] According to the first embodiment, the second threshold value Vo2, which increases in accordance with the distance value Lo of the observation point, is used as a reference, and an observation point that recognizes a speed value Vo exceeding the second threshold value Vo2 is subjected to matching with a scanning point. Accordingly, the closer a scanning point representing a moving object 3 is to the host vehicle 2, the lower the speed value Vo is subjected to matching with the observation point, and thus the point can be colored with the caution color Cw. Therefore, for a moving object 3 that is close to the host vehicle 2, it is possible to enhance attention-grabbing even when the object is moving at a low speed. Conversely, the farther a scanning point is from the host vehicle 2, the farther the scanning point is from the host vehicle 2, the higher the speed value Vo is subjected to matching with the observation point, and thus the point can be colored with the caution color Cw. Therefore, for a moving object 3 that is farther from the host vehicle 2, it is possible to enhance attention-grabbing by focusing on the object's high-speed movement.

[0047] According to the first embodiment, the display point cloud data Dd is displayed with a constant warning color Cw set regardless of the magnitude of the recognized speed value Vo. This allows the operator of the host vehicle 2 to easily pay attention to the moving object 3 with the speed value Vo by visually recognizing the scanning point that displays a constant color when the speed value Vo occurs. This makes it possible to quickly call attention to the moving object 3.

[0048] According to the first embodiment, a scanning point whose moving speed value Vo is matched to an unrecognized observation point is colored in the display point cloud data Dd with an exclusion display color Ce that is different from the attention color Cw of a scanning point whose moving speed value Vo is matched to a recognized observation point. This allows the operator of the host vehicle 2 to consciously exclude static targets whose speeds are unrecognized and that are represented by scanning points with an exclusion display color Ce that is different from the scanning points corresponding to the moving object 3, and pay attention to the moving object 3. This makes it possible to concentrate on the moving object 3 and increase attention-attracting properties.

[0049] According to the first embodiment, the scanning point cloud data Ds, in which scanning points corresponding to observation points whose speeds were recognized in previous frames have been excluded and the movement amount of the host vehicle 2 has been compensated for, is integrated with the scanning point cloud data Ds in which scanning points are colored corresponding to observation points whose speeds were recognized in the current frame. Therefore, in the display point cloud data Dd generated by the integration, scanning points representing a moving object 3 having a moving speed value Vo are colored in the caution color Cw only for the scanning points of the current frame to prevent the scanning points from being erroneously displayed as a trajectory due to the integration. This allows the operator of the host vehicle 2 to properly separate and pay attention to the moving object 3 represented by the scanning points colored in the caution color Cw from the static target represented by the scanning points colored in the exclusion display color Ce. This ensures high attention-grabbing focused on the moving object 3.

[0050] Second Embodiment The second embodiment is a modified example of the first embodiment. In S50 according to the second embodiment, as shown in Fig. 7, the scanning points of the attention moving object data Dsw are colored with a plurality of colors (Cw1, Cw2 in Fig. 7) such as red and yellow on a color wheel as an attention color Cw that changes in stages depending on the magnitude of the velocity value Vo recognized at the observation point of the main matching target in S20. In this case, it is preferable that the scanning points corresponding to the observation points with larger velocity values ​​Vo are colored with an attention color Cw (Cw1 in Fig. 7) such as red, which is more likely to attract attention in terms of human visual stimulation.

[0051] According to the second embodiment, the display point cloud data Dd is displayed and output, with a warning color Cw set that changes according to the magnitude of the recognized speed value Vo. This allows the operator of the host vehicle 2 to intuitively grasp the magnitude of the speed value Vo from the change in the warning color Cw, and to pay attention to the moving object 3 at a level that corresponds to the magnitude of the speed value Vo. Therefore, it is possible to provide a precise warning that is tailored to the speed value of the moving object 3.

[0052] (Other embodiments) Although multiple embodiments have been described above, the present disclosure should not be construed as being limited to those embodiments, and can be applied to various embodiments and combinations within the scope that does not deviate from the gist of the present disclosure.

[0053] In a modified example, the dedicated computer constituting the support system 1 may have at least one of a digital circuit and an analog circuit as a processor. Here, the digital circuit is at least one of the following: an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a system on a chip (SOC), a programmable gate array (PGA), and a complex programmable logic device (CPLD). Such a digital circuit may also have a memory that stores a program.

[0054] In a modified example, S20 may be skipped, and all of the observation points in the observation point cloud data Do may be subjected to the matching process with the scanning points in S30. In this case, the observation points to be matched as main and sub-matching targets may be selected in S40, and three types of data Dsw, Dse, and Dss may be further separated and generated from the scanning point cloud data Ds accordingly in S40. In the modified example, S40 may set the unique color of a static target recognized by image capture from a camera of the external sensor 52 as the exclusion display color Ce for the scanning points of the static target data Dss. In the modified example, S50 may skip integration of the static target data Dss using the past flow.

[0055] In addition to the forms described so far, the above-mentioned embodiments and variations may be implemented in the form of a processing circuit (e.g., a processing ECU, etc.) or a semiconductor device (e.g., a semiconductor chip, etc.) as an assistance device configured to be mountable on the host vehicle 2 and having at least one processor 12 and one memory 10. [Explanation of symbols]

[0056] 1: Support system, 2: Host vehicle, 3: Moving object, 10: Memory, 12: Processor, 52o: Radar sensor, 52s: Optical sensor, Ce: Exclusion display color, Cw: Caution color, Dd: Display point cloud data, Do: Observation point cloud data, Ds: Scanning point cloud data, Lo: Distance value, Vo: Speed ​​value, Vo1: First threshold, Vo2: Second threshold

Claims

1. An assistance system for assisting an operator in driving a host vehicle (2) equipped with a processor (12), an optical sensor (52s) that scans the outside world by irradiating light, and a radar sensor (52o) that observes the outside world by irradiating radio waves, Acquiring scanning point cloud data (Ds) representing a plurality of scanning points by the optical sensor and observation point cloud data (Do) representing a plurality of observation points by the radar sensor; matching the scanning points of the scanning point cloud data with the observation points of the observation point cloud data; The support system is configured to display and output display point cloud data (Dd) as scanning point cloud data in which the scanning points matched with the observation points at which the moving velocity value (Vo) is recognized are colored with an attention color (Cw) for the moving body (3) of the velocity value.

2. Matching the scanning points with the observation points includes: The assistance system of claim 1, further comprising matching the observation point at which the velocity value exceeding the threshold (Vo2) is recognized with the scanning point, based on a threshold (Vo2) that increases in accordance with the distance value (Lo) of the observation point.

3. The display output of the display point cloud data includes:

2. The assistance system according to claim 1, further comprising displaying and outputting the display point cloud data in which a constant attention color is set regardless of the magnitude of the velocity value to be recognized.

4. The display output of the display point cloud data includes: The assistance system according to claim 1 , further comprising displaying and outputting the display point cloud data in which the attention color is set and which changes depending on the magnitude of the velocity value recognized.

5. The display output of the display point cloud data includes: The assistance system according to claim 1, further comprising displaying and outputting the display point cloud data in which the scanning points that match the observation points outside of recognition are colored with a display color (Ce) that is different from the attention color, when the speed value during movement is matched with the observation points outside recognition.

6. The display output of the display point cloud data includes:

6. The assistance system according to claim 5, further comprising: displaying and outputting the display point cloud data generated by integrating the scanning point cloud data, in which the scanning points corresponding to the observation points for which the speed values ​​were recognized in past frames have been excluded and the amount of movement of the host vehicle has been compensated, with the scanning point cloud data, in which the scanning points are colored in correspondence with the observation points for which the speed values ​​were recognized in the current frame.

7. An assistance device for assisting an operator of a host vehicle in driving the host vehicle, the assistance device having a processor (12), an optical sensor (52s) for scanning the outside world by irradiating light, and a radar sensor (52o) for observing the outside world by irradiating radio waves, the assistance device being configured to be mountable on a host vehicle (2), Acquiring scanning point cloud data (Ds) representing a plurality of scanning points by the optical sensor and observation point cloud data (Do) representing a plurality of observation points by the radar sensor; matching the scanning points of the scanning point cloud data with the observation points of the observation point cloud data; The support device is configured to display and output display point cloud data (Dd) as scanning point cloud data in which the scanning points matched with the observation points at which the moving velocity value (Vo) is recognized are colored with an attention color (Cw) for the moving body (3) of the velocity value.

8. An assistance method executed by a processor (12) to assist an operator of a host vehicle (2) equipped with an optical sensor (52s) that scans the outside world by irradiating light and a radar sensor (52o) that observes the outside world by irradiating radio waves, the method comprising: Acquiring scanning point cloud data (Ds) representing a plurality of scanning points by the optical sensor and observation point cloud data (Do) representing a plurality of observation points by the radar sensor; matching the scanning points of the scanning point cloud data with the observation points of the observation point cloud data; and displaying and outputting display point cloud data (Dd) as scanning point cloud data in which the scanning points matched with the observation points at which the moving velocity value (Vo) is recognized are colored with an attention color (Cw) for the moving body (3) of the velocity value.

9. An assistance program is stored in a storage medium (10) to assist an operator of a host vehicle (2) equipped with an optical sensor (52s) that scans the outside world by irradiating light and a radar sensor (52o) that observes the outside world by irradiating radio waves, the assistance program including instructions for causing a processor (12) to execute the assistance, Acquiring scanning point cloud data (Ds) representing a plurality of scanning points by the optical sensor and observation point cloud data (Do) representing a plurality of observation points by the radar sensor; matching the scanning points of the scanning point cloud data with the observation points of the observation point cloud data; and displaying and outputting display point cloud data (Dd) as scanning point cloud data in which the scanning points matched with the observation points at which the moving velocity value (Vo) is recognized are colored with an attention color (Cw) for the moving body (3) of the velocity value.

Images