Object calculation method and calculation device

Abstract

To reduce a target recognition error.SOLUTION: A target calculation method is executed by an arithmetic unit including an acquisition section for acquiring sensor outputs being outputs of sensors which acquire information of a peripheral environment. The target calculation method comprises: detection processing for detecting a target so as to detect a target and including at least a position and a kind of the target by a plurality of methods with the use of the sensor outputs; same-target determination processing for determining the same target from the plurality of targets respectively detected by the plurality of methods in the detection processing; and fusion processing for fusing the target states so as to output the states as a fusion target concerning the targets determined to be the same target in the same-target determination processing.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a target calculation method and a calculation device. [Background technology]

[0002] Techniques using machine learning have been considered to achieve highly automated driving of vehicles. Patent Literature 1 discloses a method for improving the detection capability of a detection algorithm of a driving assistance system, the method comprising the steps of: providing a vehicle-based driving assistance system having a processing entity that executes a detection algorithm that produces a detection result, the driving assistance system having at least one sensor that detects static environmental features around the vehicle, receiving sensor information related to the static environmental features from the sensor at a processing entity of the vehicle, processing the received sensor information to thereby obtain processed sensor information, receiving at least one stored static environmental feature from an environmental data source, comparing the processed sensor information with the stored static environmental feature, determining whether an inconsistency exists between the processed sensor information and the stored static environmental feature, and if an inconsistency between the processed sensor information and the stored static environmental feature is determined, correcting the detection algorithm by providing training information derived from a comparison result between the processed sensor information and the stored static environmental feature to the machine training algorithm based on a machine learning algorithm. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2021-18823 Summary of the Invention [Problem to be solved by the invention]

[0004] In the invention described in Patent Document 1, recognition errors may occur. [Means for solving the problem]

[0005] A target calculation method according to a first aspect of the present invention is a target calculation method executed by a computing device having an acquisition unit that acquires sensor output, which is the output of a sensor that acquires information about a surrounding environment, and includes: a detection process that detects targets by a plurality of methods using the sensor output and detects target states of the targets, including at least a position and a type; an identical target determination process that determines the same target from the plurality of targets detected by each of the plurality of methods in the detection process; and a fusion process that fuses the target states of the targets determined to be the same in the identical target determination process and outputs a fused target. The detection process includes a rule-based detection process that uses the sensor output to detect a rule target that is a target based on a rule base, and an AI detection process that uses the sensor output to detect an AI target that is a target based on machine learning, and in the fusion process, for a rule target that is associated with one or more AI targets, the target state calculated based on the target state of the AI ​​target and the target state of the rule target is output as the fusion target, and for a rule target that does not have an associated AI target, the rule target is output as the fusion target. A target calculation method according to a second aspect of the present invention is a target calculation method executed by a computing device having an acquisition unit that acquires sensor output, which is the output of a sensor that acquires information about the surrounding environment, and includes: a detection process that detects targets by a plurality of methods using the sensor output and detects target states of the targets, including at least a position and a type; an identical target determination process that determines the same target from the plurality of targets detected by each of the plurality of methods in the detection process; and a fusion process that fuses the target states of the targets determined to be the same in the identical target determination process and outputs a fused target, wherein the detection process further includes: a rule-based detection process that detects rule targets that are targets based on a rule base using the sensor output; an AI detection process that detects AI targets that are targets based on machine learning using the sensor output; and a degradation detection process that detects degradation of the sensor output; and when degradation of the sensor output is detected by the degradation detection process, the fusion process fuses the AI ​​targets and the rule targets at a predetermined ratio to generate the fused target. A computing device according to a third aspect of the present invention includes an acquisition unit that acquires a sensor output that is an output from a sensor that acquires information about a surrounding environment; a detection unit that detects targets by a plurality of methods using the sensor output and detects target states of the targets including at least a position and a type; an identical target determination unit that determines the same target from the plurality of targets detected by each of the plurality of methods in the detection unit; and a fusion unit that fuses the target states of the targets that the identical target determination unit determines to be the same target and outputs a fused target. The detection unit executes a rule-based detection process that uses the sensor output to detect a rule target that is a target based on a rule base, and an AI detection process that uses the sensor output to detect an AI target that is a target based on machine learning, and the fusion unit outputs, for a rule target that is associated with one or more AI targets, the target state calculated based on the target state of the AI ​​target and the target state of the rule target as the fusion target, and for a rule target that does not have an associated AI target, outputs the rule target as the fusion target. A computing device according to a fourth aspect of the present invention includes an acquisition unit that acquires sensor output, which is the output of a sensor that acquires information about the surrounding environment; a detection unit that detects targets using a plurality of methods using the sensor output and detects target states of the targets, including at least their position and type; an identical target determination unit that determines that the same target is one of the multiple targets detected by each of the multiple methods in the detection unit; and a fusion unit that fuses the target states of the targets that the identical target determination unit determines to be the same target and outputs a fused target. The detection unit executes a rule-based detection process that uses the sensor output to detect rule targets that are targets based on a rule base, and an AI detection process that uses the sensor output to detect AI targets that are targets based on machine learning. The computing device further includes a degradation detection unit that detects deterioration of the sensor output. When the degradation detection unit detects deterioration of the sensor output, the fusion unit fuses the AI ​​targets and the rule targets at a predetermined ratio to generate the fused target. [Effects of the Invention]

[0006] According to the present invention, since a plurality of methods are used to detect targets, recognition errors can be reduced. [Brief explanation of the drawings]

[0007] [Figure 1] Functional configuration diagram of a computing device according to the first embodiment [Figure 2] FIG. 10 is a diagram showing an example of processing by a same target determination unit; [Figure 3]Hardware configuration diagram of the computing device [Figure 4] Flowchart showing the processing of the arithmetic unit [Figure 5] FIG. 10 shows a first operation example. [Figure 6] FIG. 2 shows a second operation example. [Figure 7] FIG. 3 shows a third example of operation. [Figure 8] Functional configuration diagram of a calculation device in Modification 2 [Figure 9] Functional configuration diagram of a computing device according to a second embodiment [Figure 10] FIG. 10 is a diagram showing an example of ratio information DETAILED DESCRIPTION OF THE INVENTION

[0008] -First embodiment- A first embodiment of a calculation device and a target calculation method will be described below with reference to FIGS.

[0009] (composition) 1 is a functional configuration diagram of a computing device 1. The computing device 1 is mounted on a vehicle 9 together with a first sensor 21, a second sensor 22, and a third sensor 23. The computing device 1 includes a first calculation unit 11, a second calculation unit 12, a same target determination unit 13, and a recognition fusion unit 14.

[0010] The first sensor 21, the second sensor 22, and the third sensor 23 are sensors that acquire information about the surrounding environment of the vehicle 9. The first sensor 21, the second sensor 22, and the third sensor 23 output the information obtained by sensing to the computing device 1 as sensor output. The specific configurations of the first sensor 21, the second sensor 22, and the third sensor 23 are not limited, and may be, for example, a camera, a laser range finder, or a LiDAR (Light Detection and Ranging). However, any of the first sensor 21, the second sensor 22, and the third sensor 23 may be the same type of sensor.

[0011] The first calculation unit 11 and the second calculation unit 12 calculate the target state based on the sensor outputs. In this embodiment, the target state refers to the position and type of the target. However, the target state may also include the target's speed. The target position is calculated as coordinates in a Cartesian coordinate system, for example, with the center of the vehicle 9 as the origin, the front of the vehicle 9 as the positive side of the X-axis, and the right side of the vehicle 9 as the positive side of the Y-axis. Examples of target types include automobiles, motorcycles, pedestrians, lane markings, stop lines, traffic lights, guardrails, and buildings. The first calculation unit 11 receives the sensor output of the first sensor 21, and the second calculation unit 12 receives the sensor outputs of the second sensor 22 and the third sensor 23. However, the sensor outputs of the same sensor may be input to the first calculation unit 11 and the second calculation unit 12.

[0012] The first calculation unit 11 and the second calculation unit 12 operate independently to calculate the target state, i.e., the position and type of the target. The first calculation unit 11 and the second calculation unit 12 calculate the target state at short time intervals, for example, every 10 ms, and output the target state to the same target determination unit 13 with an identifier, i.e., ID, attached.

[0013] The first calculation unit 11 detects targets based on rules. The first calculation unit 11 includes information on rules, such as predetermined calculation formulas. The first calculation unit 11 processes the sensor output according to these rules to obtain the target state, i.e., the position and type of the target. Hereinafter, the target calculated by the first calculation unit 11 will be referred to as a "rule target." The calculation of the target by the first calculation unit 11 will also be referred to as a "rule-based detection process."

[0014] The second calculation unit 12 detects targets based on machine learning. The second calculation unit 12 processes sensor outputs using parameters generated by a learning program using a large amount of learning data and an inference program to obtain target states. The processing by the second calculation unit 12 can also be said to be inference for unknown phenomena through an inductive approach using existing data. Hereinafter, targets calculated by the second calculation unit 12 will be referred to as "AI targets." The calculation of targets by the second calculation unit 12 will also be referred to as "AI detection processing."

[0015] The same target determination unit 13 simply determines whether the rule target calculated by the first calculation unit 11 is the same as the AI ​​target calculated by the second calculation unit 12. Specifically, the same target determination unit 13 associates the closest rule target that exists within a predetermined distance from each AI target. The same target determination unit 13 performs processing only based on the AI ​​target, and does not perform processing based on the rule target. The recognition fusion unit 14 uses the determination result of the same target determination unit 13 to fuse the rule target and the AI ​​target and output the result. Hereinafter, the target output by the recognition fusion unit 14 is referred to as a "fused target." In this embodiment, the rule target is used to determine the presence or absence of a target and its position, and the AI ​​target is used to determine the type of target. The target state output by the recognition fusion unit 14 is used by other devices installed in the vehicle 9 to realize, for example, autonomous driving or an advanced driving assistance system.

[0016] FIG. 2 is a diagram illustrating an example of the processing performed by the identical target determination unit 13. When the four rule targets and four AI targets shown in FIG. 2 are calculated, the identical target determination unit 13 associates the targets based on the identity determination results, as shown on the right side of the figure. In FIG. 2, a "#" indicates that no associated target exists. In the example shown in FIG. 2, the rule target A1 is the closest rule target within a predetermined distance from the AI ​​target B1. Also, the rule target A2 is the closest rule target within a predetermined distance from the AI ​​targets B2 and B3. Furthermore, it is shown that there was no rule target within a predetermined distance from the AI ​​target B4. The reason why the AI ​​target B4 is crossed out will be explained later.

[0017] 3 is a hardware configuration diagram of the arithmetic device 1. The arithmetic device 1 includes a CPU 41, which is a central processing unit, a ROM 42, which is a read-only storage device, a RAM 43, which is a readable and writable storage device, and a communication device 44. The CPU 41 loads a program stored in the ROM 42 into the RAM 43 and executes it to perform the various calculations described above.

[0018] The arithmetic device 1 may be realized by a field programmable gate array (FPGA), which is a rewritable logic circuit, or an application specific integrated circuit (ASIC), which is an application specific integrated circuit, instead of the combination of the CPU 41, ROM 42, and RAM 43. Furthermore, the arithmetic device 1 may be realized by a combination of different configurations, for example, a combination of the CPU 41, ROM 42, RAM 43, and FPGA, instead of the combination of the CPU 41, ROM 42, and RAM 43. The communication device 44 is, for example, a communication interface compatible with IEEE802.3, and transmits and receives information between the arithmetic device 1 and other devices mounted on the vehicle 9. The communication device 44 acquires sensor outputs from sensors mounted on the vehicle 9, and therefore may also be called an "acquisition unit."

[0019] (operation) Fig. 4 is a flowchart showing the processing of the calculation device 1. However, before the processing shown in Fig. 4 is started, the first calculation unit 11 and the second calculation unit have completed target detection. In Fig. 4, first, in step S301, the same target determination unit 13 selects one AI target. In the following step S302, the same target determination unit 13 identifies a rule target whose position is closest to the AI ​​target selected in step S301.

[0020] In the next step S303, the same target determination unit 13 determines whether the distance between the AI ​​target selected in step S301 and the rule target identified in step S302 is equal to or less than a predetermined threshold. If the same target determination unit 13 determines that the distance between them is equal to or less than the predetermined threshold, the process proceeds to step S304, where the same target determination unit 13 associates the AI ​​target with the rule target. Note that, as shown in the example of FIG. 2, one rule target may be associated with multiple AI targets.

[0021] If the same target determination unit 13 determines that the distance between the AI ​​target and the rule target is greater than a predetermined threshold, the process proceeds to step S305, where the AI ​​target selected in step S301 is deleted. This deletion process corresponds to, for example, displaying AI target B4 with a strikeout line in the example shown in Fig. 2. If the rule target is not detected, the distance between the AI ​​target and the rule target is considered to be infinite, and a negative determination is made in step S303.

[0022] In step S306, which is executed following step S304 or step S305, the same target determination unit 13 determines whether or not an unprocessed AI target exists. If the same target determination unit 13 determines that an unprocessed AI target exists, the process returns to step S301. If the same target determination unit 13 determines that an unprocessed AI target does not exist, the process proceeds to step S311. In step S311, the recognition fusion unit 14 selects one unselected rule target. In the following step S312, the recognition fusion unit 14 determines the number of AI targets associated with the rule target selected in step S311.

[0023] When the recognition fusion unit 14 determines that the associated rule object is "0," i.e., that the rule object is not associated with any rule object, it uses the position information and type information of the rule object. For example, rule objects A3 and A4 in the example shown in FIG. 2 correspond to this example. When the recognition fusion unit 14 determines that the associated rule object is "1," it uses a target that combines the position of the rule object and the type of the AI ​​object. For example, rule object A1 in the example shown in FIG. 2 corresponds to this example.

[0024] If the recognition fusion unit 14 determines that there are "two or more" associated rule objects, it regards the rule object as a plurality of objects that have the position of the rule object and combine the types of each AI object. For example, rule object A2 in the example shown in FIG. 2 corresponds to this example. In step S316, which is executed when the processing of any one of steps S313 to S315 is completed, the recognition fusion unit 14 determines whether there are any unprocessed rule objects, i.e., rule objects that have not been selected in step S311. If the recognition fusion unit 14 determines that there are any unprocessed rule objects, it returns to step S311, and if it determines that there are no unprocessed rule objects, it ends the processing shown in FIG. 4.

[0025] (Example of operation) Three operation examples will be described below with reference to Figures 5 to 7. In each operation example, a schematic diagram showing only each target is shown to explain the relationship between the rule target, the AI ​​target, and the fusion target.

[0026] FIG. 5 is a diagram illustrating a first operation example. The three diagrams in FIG. 5 show, from left to right, a rule target, an AI target, and a fusion target. In each diagram, the open square at the bottom represents the vehicle 9, and the hatched square at the top represents the detected target. This is also true for FIGS. 6 and 7, which will be described later. As indicated by reference numeral 1101, the rule calculation unit 11 detects one target A1 far from the vehicle 9. As indicated by reference numeral 1102, the AI ​​calculation unit 12 detects two targets B1 and B2 far from the vehicle 9. In this example, the distance between the AI ​​target B1 and the rule target A1 is equal to or less than a predetermined threshold, and the distance between the AI ​​target B2 and the rule target A1 is equal to or less than a predetermined threshold.

[0027] In this case, the process is as follows in the flowchart of Fig. 4. That is, for both AI targets B1 and B2, a positive determination is made in step S303 of Fig. 4, and they are associated with the rule target A1 in step S304. Then, in step S312, two AI targets are associated with the rule target A1, so the process proceeds to step S315, where two fusion targets having the position of the rule target A1 are output, as indicated by reference numeral 1103.

[0028] FIG. 6 is a diagram illustrating a second operation example. The rule calculation unit 11 did not detect any targets, as indicated by reference numeral 1201. The AI ​​calculation unit 12 detected two targets, B3 and B4, at a distance from the vehicle, as indicated by reference numeral 1202. In this case, the following processing is performed in the flowchart of FIG. 4. That is, regardless of whether targets B3 or B4 are selected in step S301, the distance to the non-existent rule target is set to infinity, and a negative determination is made in step S303. Therefore, targets B3 and B4 are deleted in step S305. In this example, since no rule target exists, the processing in steps S311 to S316 is not performed, and as a result, the recognition fusion unit 14 does not output a fusion target, as indicated by reference numeral S1203.

[0029] FIG. 7 is a diagram showing a third operation example. The rule calculation unit 11 detected one rule target A2, as indicated by reference numeral 1301. The AI ​​calculation unit 12 did not detect any targets, as indicated by reference numeral 1302. In this case, the following processing is performed in the flowchart of FIG. 4. That is, since no AI target has been detected, the processing of steps S301 to S305 is not performed, a negative judgment is made in step S306, and the process proceeds to step S311. In step S311, target A2 is selected, and in the following step S312, the recognition fusion unit 14 proceeds to step S313 because there is no related AI target. In step S313, the information of rule target A2 is used as the fusion target as is.

[0030] According to the first embodiment described above, the following advantageous effects can be obtained. (1) The communication device 44, which acquires sensor output, which is the output of a sensor that acquires information about the surrounding environment, executes the following target calculation method. The target calculation method includes a detection process executed by the first calculation unit 11 and the second calculation unit 12, which detects targets using multiple techniques using the sensor output and detects target states including at least the position and type of the targets; a same target determination process executed by the same target determination unit 13, which determines that the same target is detected from multiple targets detected by each of the multiple techniques in the detection process; and a fusion process executed by the recognition fusion unit 14, which fuses target states for targets determined to be the same in the same target determination process and outputs a fused target. Therefore, the target calculation method executed by the arithmetic device 1 detects targets using multiple techniques, thereby reducing recognition errors.

[0031] (2) The detection process executed by the calculation device 1 includes a rule-based detection process executed by the first calculation unit 11 for detecting a rule target, which is a target, based on a rule base using sensor output, and an AI detection process executed by the second calculation unit 12 for detecting an AI target, which is a target, based on machine learning using sensor output. Therefore, targets can be detected based on two techniques, rule-based detection and machine learning, which have different properties.

[0032] (3) In the same target determination process, as shown in steps S302 to S304 in Figure 4, rule targets that are within a predetermined distance from the AI ​​target are determined to be the same target and associated. In the fusion process, a fusion target is not generated based on an AI target that the same target determination unit determines does not have a rule target within the predetermined distance. Therefore, targets detected only by the AI ​​detection process, which has a tendency to overdetect, erroneously detecting non-existent targets, can be determined to be false detections and their output suppressed.

[0033] (4) In the fusion process, as shown in steps S312 to S315 in Fig. 4, for a rule object that has one or more associated AI objects, a target state calculated based on the target state of the AI ​​object and the target state of the rule object is output as a fused target, and for a rule object that has no associated AI object, the rule object is output as a fused target. Therefore, for mutually associated AI objects and rule objects, a fused target is output using information from both, and they are detected by rule-based detection that is less likely to cause overdetection, and for rule objects that have no associated AI objects, the information of the rule object is output as a fused target as is, thereby enabling highly accurate and complete target detection.

[0034] (5) In the fusion process, as shown in step S314 of FIG. 4, for a rule object that is associated with only one AI object, a fusion target is output that combines the type of the AI ​​object and the position of the rule object. As shown in step S315 of FIG. 4, for a rule object that is associated with two or more AI objects, a plurality of fusion targets is output that combines the position of the rule object and the type of each AI object. As shown in step S313 of FIG. 4, for a rule object that does not have an associated AI object, the rule object is output as a fusion target. Generally, it is not easy for rule-based detection to correctly identify two vehicles traveling close to each other at the same speed as two vehicles. In this embodiment, the accuracy of target detection can be improved in such cases by adopting the results of the AI ​​detection process.

[0035] (Variation 1) In steps S314 and S315 of Fig. 4, instead of directly using the position information of the rule object, a weighted average of the information of the rule object and the information of the AI ​​object may be used. In this case, however, a predetermined coefficient is set so that the information of the rule object is weighted higher than the information of the AI ​​object. In other words, the position of the fusion object in this case is closer to the position of the rule object than the position of the AI ​​object.

[0036] Furthermore, when the target state includes speed information, only the information of the rule target may be used, as in the position information in the first embodiment, or a weighted average of the information of the rule target and the information of the AI ​​target may be used. However, even in this case, a predetermined coefficient is set so that the information of the rule target is weighted higher than the information of the AI ​​target. In other words, the position of each fusion target in this case is closer to the position of the rule target than the position of the respective AI target.

[0037] (6) In the fusion process, for a rule object that is associated with only one AI object, the rule object is output as a fusion object having a position closer to the position of the rule object than the position of the AI ​​object; for a rule object that is associated with two or more AI objects, the rule object is output as a multiple fusion object having a position closer to the position of the rule object than the position of the AI ​​object; and for a rule object that does not have an associated AI object, the rule object is output as a fusion object.

[0038] (Variation 2) FIG. 7 is a functional configuration diagram of the arithmetic device 1 in Modification 2. The arithmetic device 1 shown in FIG. 7 further includes a degeneration determination unit 18 in addition to the configuration of the arithmetic device 1 in the first embodiment. The degeneration determination unit 18 outputs a degeneration operation command to the vehicle 9 when the rule targets output by the first calculation unit 11 and the AI ​​targets output by the second calculation unit 12 are significantly different. For example, the degeneration determination unit 18 determines that the rule targets and the AI ​​targets are significantly different when the distance between each rule target and the AI ​​target is equal to or greater than a predetermined distance, or when the difference between the number of rule targets and the number of AI targets is equal to or greater than a predetermined ratio. The degeneration operation command is a command to restrict the function of the vehicle 9. For example, if the vehicle 9 is equipped with an autonomous driving system, the degeneration operation command is a command to switch the autonomous driving system to manual driving or a command to stop the vehicle.

[0039] (Variation 3) The computing device 1 may include three or more target state detection units. Each target state calculation unit is classified into either a rule detection unit or an AI detection unit based on its operating principle. A target calculated by a target state calculation unit classified into a rule detection unit is considered a rule target, and a target calculated by a target state calculation unit classified into an AI detection unit is considered an AI target. The processing by the same target determination unit 13 and the recognition fusion unit 14 is the same as in the first embodiment.

[0040] (Variation 4) The recognition fusion unit 14 may further determine whether each calculated fusion target matches any previously calculated fusion target. For this determination, the position, speed, and type of the fusion target can be used, for example. The recognition fusion unit 14 preferably assigns an ID to each fusion target, and assigns the same ID to the same fusion target at different times.

[0041] --Second embodiment-- A second embodiment of the calculation device and target calculation method will be described with reference to Figures 9 and 10. In the following description, the same components as those in the first embodiment are denoted by the same reference numerals, and differences will be mainly described. Points that are not particularly described are the same as those in the first embodiment. This embodiment differs from the first embodiment mainly in that the ratio of AI targets to rule targets in the fusion target is changed depending on the situation.

[0042] FIG. 9 is a functional configuration diagram of a calculation device 1A according to the second embodiment. The calculation device 1A shown in FIG. 9 further includes a deterioration detection unit 15 and a ratio setting unit 16 in addition to the configuration of the calculation device 1 according to the first embodiment. The first calculation unit 11 and the second calculation unit 12 also output a numerical value indicating the likelihood of the detected target state. This numerical value indicating the likelihood is, for example, a value between 0 and 1, with a larger value indicating a higher likelihood. The deterioration detection unit 15 detects deterioration of the sensor output and outputs the type of deterioration to the ratio setting unit 16. However, if the deterioration detection unit 15 does not detect deterioration of the sensor output, it outputs to the ratio setting unit 16 a message indicating that there is no deterioration.

[0043] The sensor output degradation detected by degradation detection unit 15 includes output degradation caused by some factor in the sensor and output degradation caused by the surrounding environment. Output degradation caused by the sensor includes, for example, dirt adhering to the lens or image sensor when the sensor is a camera. Output degradation caused by the surrounding environment includes, for example, backlight, rain, dust, and nighttime when the sensor is a camera, and the presence of radio wave reflecting objects when the sensor is a radar. Degradation detection unit 15 may detect sensor output degradation using the sensor output, or may estimate sensor output degradation by acquiring information from an external device via communication.

[0044] Ratio setting unit 16 sets the ratio of rule targets to AI targets in the fusion process in the same target determination unit 13 and the recognition fusion unit 14 according to the type of deterioration of the sensor output. In this embodiment, arithmetic device 1A stores ratio information 17 in ROM 42, which is a storage unit. Ratio information 17 stores, for each type of deterioration of the sensor output, information on the probability of target presence in the fusion process, the target position, and the ratio of rule targets to AI targets when determining the target type.

[0045] FIG. 10 is a diagram showing an example of ratio information 17. In the example shown in FIG. 10, the ratio of rule targets and AI targets used to determine the probability of target presence, target position, and target type is listed for each of "normal" (no degradation in sensor output), lens dirt, radio wave reflecting object, rainy weather, and nighttime. For example, when the degradation detection unit 15 notifies the user of lens dirt, the ratio setting unit 16 outputs information on six numerical values ​​enclosed by dashed lines in FIG. 10 to the same target determination unit 13 and the recognition fusion unit 14.

[0046] The following describes differences between the processing of the identical target determination unit 13 and the recognition fusion unit 14 and that of the first embodiment. The processing of the identical target determination unit 13 and the recognition fusion unit 14 when the ratio information 17 outputs a "normal" value is the same as that of the first embodiment. The identical target determination unit 13 determines the presence of a target at each position based on the ratio of the presence probability in step S304 of FIG. 4. For example, when a rule target A9 is present within a predetermined distance from an AI target B9, the identical target determination unit 13 determines whether to associate the two targets as follows: That is, the identical target determination unit 13 associates the two targets when the sum of the product of the likelihood of the AI ​​target B9 calculated by the second calculation unit 12 and the value of the coefficient of the AI ​​target's presence probability in the ratio information 17 and the product of the likelihood of the rule target A9 calculated by the first calculation unit 11 and the value of the coefficient of the rule target's presence probability in the ratio information 17 exceeds a predetermined threshold, for example, "1.0."

[0047] 4 as follows. That is, the recognition fusion unit 14 calculates the position of the fusion target by taking a weighted average of the positions of the rule target and the AI ​​target, and uses the value of the ratio information 17 output by the ratio setting unit 16 as the coefficient of the weighted average. Furthermore, the recognition fusion unit 14 adopts, as the type of the fusion target, the type of the target that is larger than either the value obtained by multiplying the certainty of the rule target by the coefficient of the rule target type in the ratio information 17 or the value obtained by multiplying the certainty of the AI ​​target by the coefficient of the AI ​​target type in the ratio information 17.

[0048] According to the second embodiment described above, the following advantageous effects can be obtained. (7) The calculation device 1A includes a degradation detection process that detects degradation of the sensor output. When degradation of the sensor output is detected by the degradation detection process, the fusion process fuses the AI ​​target and the rule target at a predetermined ratio to generate a fused target. Therefore, the calculation device 1A can generate a fused target that fuses information from the rule target and the AI ​​target.

[0049] (8) The calculation device 1A includes a ROM 42 that stores ratio information 17 that defines the ratio of AI targets and rule targets for each type of sensor output degradation. The fusion process identifies the type of sensor output degradation and determines the ratio of AI targets and rule targets by referring to the ratio information. Therefore, the calculation device 1A can generate a fusion target that combines information on rule targets and AI targets with optimal weighting according to the situation. In particular, if the sensor output degradation state is included in the learning data used to generate the parameters used by the second calculation unit 12, the reliability of the AI ​​detection process is relatively high, so a high ratio can be set in the ratio information 17, thereby improving recognition accuracy.

[0050] (Modification of the second embodiment) The degradation of the sensor output may be applied to a portion of the sensor output. For example, if the sensor is a camera, the degradation detection unit 15 divides the image captured by the camera into multiple regions, determines the degradation of the output for each region, and sends the type of degradation for each region to the ratio setting unit 16. The ratio setting unit 16 determines the ratio of the AI ​​target and the rule target for each region of the sensor output based on the ratio information 17, and the same target determination unit 13 and the recognition fusion unit 14 fuse the AI ​​target and the rule target at the ratio specified by the ratio setting unit 16 for each region of the sensor output to generate a fused target.

[0051] In each of the above-described embodiments and modifications, the functional block configurations are merely examples. Some functional configurations shown as separate functional blocks may be configured as an integrated unit, or a configuration shown in a single functional block diagram may be divided into two or more functions. Furthermore, some of the functions of each functional block may be provided by other functional blocks.

[0052] In the above-described embodiments and modifications, the program is stored in ROM 42, but the program may be stored in a non-volatile storage device (not shown). Furthermore, the arithmetic device 1 may be provided with an input / output interface (not shown), and the program may be loaded from another device as needed via the input / output interface and a medium available to the arithmetic device 1. Here, the medium refers to, for example, a storage medium detachable from the input / output interface, or a communication medium, i.e., a wired, wireless, or optical network, or a carrier wave or digital signal propagating through the network. Furthermore, some or all of the functions realized by the program may be realized by a hardware circuit or FPGA.

[0053] The above-described embodiments and modifications may be combined with each other. Although various embodiments and modifications have been described above, the present invention is not limited to these. Other embodiments conceivable within the scope of the technical concept of the present invention are also included within the scope of the present invention. [Explanation of symbols]

[0054] 1, 1A…Arithmetic device 11...First calculation unit, rule calculation unit 12…Second calculation section, AI calculation section 13...Same target determination unit 14…Recognition fusion part 15...Deterioration detection unit 16...Ratio setting section 17...Percentage information 18...Degeneration judgment unit 44...Communication equipment

Claims

1. A target calculation method executed by a computing device including an acquisition unit that acquires a sensor output that is an output of a sensor that acquires information about a surrounding environment, a detection process for detecting a target by a plurality of methods using the sensor output and detecting a target state including at least a position and a type of the target; an identical target determination process for determining an identical target from the plurality of targets detected by each of the plurality of methods in the detection process; a fusion process for fusing the target states of the targets determined to be the same target in the same target determination process and outputting the result as a fused target, The detection process includes: a rule-based detection process for detecting a rule target, which is a target, by a rule base using the sensor output; and an AI detection process for detecting an AI target based on machine learning using the sensor output, In the fusion process, For the rule target having one or more associated AI targets, outputting the target state calculated based on the target state of the AI ​​target and the target state of the rule target as the fusion target; A target calculation method, wherein for a rule target that does not have an associated AI target, the rule target is output as the fusion target.

2. The target calculation method according to claim 1, In the fusion process, For the rule object that is associated with only one AI object, output the fusion object that combines the type of the AI ​​object and the position of the rule object; For the rule targets associated with two or more AI targets, output the rule targets as a plurality of fusion targets each combining the position of the rule target and the type of each AI target; A target calculation method, wherein for a rule target that does not have an associated AI target, the rule target is output as the fusion target.

3. The target calculation method according to claim 1, In the fusion process, For the rule object associated with only one AI object, output the rule object as the fusion object having a position closer to the position of the rule object than the position of the AI ​​object; For the rule target having two or more associated AI targets, output the rule target as a plurality of fusion targets having positions closer to the position of the rule target than the positions of the AI ​​targets; A target calculation method, wherein for a rule target that does not have an associated AI target, the rule target is output as the fusion target.

4. A target calculation method executed by a computing device including an acquisition unit that acquires a sensor output that is an output of a sensor that acquires information about a surrounding environment, a detection process for detecting a target by a plurality of methods using the sensor output and detecting a target state including at least a position and a type of the target; an identical target determination process for determining an identical target from the plurality of targets detected by each of the plurality of methods in the detection process; a fusion process for fusing the target states of the targets determined to be the same target in the same target determination process and outputting the result as a fused target, The detection process includes: a rule-based detection process for detecting a rule target, which is a target, by a rule base using the sensor output; an AI detection process for detecting an AI target based on machine learning using the sensor output; a deterioration detection process for detecting deterioration of the sensor output, The fusion processing is a target calculation method in which, when deterioration of the sensor output is detected by the deterioration detection processing, the AI ​​target and the rule target are fused at a predetermined ratio to generate the fused target.

5. 5. The target calculation method according to claim 4, The arithmetic device further includes a storage unit that stores ratio information that defines a ratio of the AI ​​target and the rule target for each type of deterioration of the sensor output, The fusion processing identifies a type of deterioration of the sensor output, and identifies a ratio of the AI ​​target and the rule target by referring to the ratio information.

6. an acquisition unit that acquires a sensor output that is an output of a sensor that acquires information about the surrounding environment; a detection unit that detects a target by a plurality of methods using the sensor output and detects a target state including at least a position and a type of the target; an identical target determination unit that determines an identical target from the plurality of targets detected by each of the plurality of methods in the detection unit; a fusion unit that fuses the target states of the targets that the same target determination unit has determined to be the same target and outputs a fused target state, The detection unit a rule-based detection process for detecting a rule target, which is a target, by a rule base using the sensor output; An AI detection process is performed to detect an AI target that is a target based on machine learning using the sensor output, The fusion portion is For the rule target having one or more associated AI targets, outputting the target state calculated based on the target state of the AI ​​target and the target state of the rule target as the fusion target; A computing device that outputs a rule target that does not have an associated AI target as the fusion target.

7. an acquisition unit that acquires a sensor output that is an output of a sensor that acquires information about the surrounding environment; a detection unit that detects a target by a plurality of methods using the sensor output and detects a target state including at least a position and a type of the target; an identical target determination unit that determines an identical target from the plurality of targets detected by each of the plurality of methods in the detection unit; a fusion unit that fuses the target states of the targets that the same target determination unit has determined to be the same target and outputs a fused target state, The detection unit a rule-based detection process for detecting a rule target, which is a target, by a rule base using the sensor output; An AI detection process is performed to detect an AI target that is a target based on machine learning using the sensor output, a deterioration detection unit that detects deterioration of the sensor output, The fusion unit is a computing device that, when the deterioration detection unit detects deterioration of the sensor output, fuses the AI ​​target and the rule target at a predetermined ratio to generate the fused target.

Images