Learning device, server, mobile object, learning method, learning program, and detection device

Abstract

To provide a technique for accurately detecting an object outside a mobile body.SOLUTION: A server (10) includes: a teacher data generation section (12) for generating teacher data based on first data imaged by an on-vehicle sensor (21) arranged in a vehicle (20) and second data indicating a position of an object on a road acquired by the server (10); and a learning section (13) for allowing machine-learning of a learning model with the use of the teacher data.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a learning device, a learning method, and a learning program for machine learning a learning model for detecting objects outside a moving body, a server and a moving body equipped with the learning model, and a detection device for detecting objects outside a moving body using the learning model. [Background technology]

[0002] Conventional technology is known that uses a camera or sensor placed on a moving object such as a vehicle to detect objects on the road located outside the moving object (e.g., pedestrians, and vehicles such as cars, motorcycles, and bicycles) and controls the moving object to prevent contact with or rear-end collision with the object.

[0003] For example, the technology described in Patent Document 1 detects a subject in a captured image as the object in question using a learning model that uses machine learning to learn image data from a speed range corresponding to the relative speed between the vehicle and the object, in order to improve the accuracy of detecting a subject in an image by a camera. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2019-215755 Summary of the Invention [Problem to be solved by the invention]

[0005] However, with conventional technologies such as the technology described in Patent Document 1, it may be difficult to detect objects on the road that are located outside the moving body in some situations, such as at night or when there are pedestrians or vehicles in the distance. [Means for solving the problem]

[0006] A learning device according to one embodiment of the present invention includes a training data generation unit that generates training data for machine learning a learning model that receives input of first data obtained by capturing or measuring the exterior of a mobile body using a camera or sensor disposed on the mobile body and second data indicating the position of a road object obtained by a server that manages information on road objects located on the road, and outputs third data indicating the detection results of an object external to the mobile body, and a learning unit that uses the training data to machine learn the learning model.

[0007] A learning method according to one aspect of the present invention includes a computer executing a training data generation process to generate training data for machine learning a learning model that receives input of first data obtained by capturing or measuring the exterior of a mobile body using a camera or sensor disposed on the mobile body and second data indicating the position of a road object obtained by a server that manages information on road objects located on the road, and outputs third data indicating the detection results of an object external to the mobile body; and a learning process to machine learn the learning model using the training data.

[0008] A detection device according to one embodiment of the present invention includes a detection unit that detects objects outside the moving body using a learning model that is machine-trained using training data based on first data obtained by capturing or measuring the outside of the moving body using a camera or sensor disposed on the moving body, and second data indicating the position of the road object obtained by a server that manages information on road objects located on the road, and that receives the first data as input and outputs third data indicating the detection result of the object outside the moving body. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a block diagram illustrating an example of a configuration of an information processing system according to a first embodiment. [Figure 2] 2 is a diagram illustrating an example of a relationship between a server and a mobile communication terminal according to the first embodiment. FIG. [Figure 3]4 is a flowchart showing an example of a learning method according to the first embodiment. [Figure 4] FIG. 10 is a diagram illustrating an example of first data. [Figure 5] 10A and 10B are diagrams illustrating an example of a detectable area and an imageable area. [Figure 6] FIG. 10 is a diagram showing an example of an imageable area in the vertical direction. [Figure 7] FIG. 2 is a diagram showing an example of an imageable area in the horizontal direction. [Figure 8] FIG. 2 is a diagram showing an example of an imageable area in which a captured image is captured. [Figure 9] FIG. 10 is a diagram illustrating an example of third data. [Figure 10] 3 is a flowchart showing an example of a detection method according to the first embodiment. [Figure 11] FIG. 10 is a block diagram showing an example of the configuration of an information processing system according to a second embodiment. [Figure 12] 10 is a flowchart showing an example of a learning method according to the second embodiment. [Figure 13] FIG. 10 is a block diagram showing an example of the configuration of an information processing system according to a third embodiment. [Figure 14] 11 is a flowchart showing an example of a learning method according to the third embodiment. [Figure 15] FIG. 10 is a diagram illustrating an example of a relationship between the orientation of a vehicle and an object. [Figure 16] FIG. 2 is a diagram illustrating an example of a relationship between an on-board sensor and an imageable area. [Figure 17] FIG. 2 is a block diagram illustrating the configuration of a computer that can be used as a server and an in-vehicle information processing device according to each embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] <Embodiment 1> An information processing system 1 according to the first embodiment will be described below with reference to FIGS.

[0011] [Information Processing System 1] 1 is a block diagram showing an example of the configuration of an information processing system according to embodiment 1. The information processing system 1 shown in FIG.

[0012] [Server 10] The server 10 may be a server that manages information about road objects located on the road. "On the road" means a route that the vehicle 20 can travel. The information about the road objects may include identification information of the road objects, their positions, types of the road objects, etc. The server 10 may include a memory unit 14, a communication unit 15, and a main control unit 17.

[0013] (Storage unit 14) The storage unit 14 may be a storage device that stores various data including the parameters (kernels) of the learning model M.

[0014] (Communications Department 15) The communication unit 15 may communicate with each of the vehicle 20, the vehicle terminal device (road object, vehicle with communication function) 30, and the pedestrian terminal device (road object, mobile communication terminal) 40. Examples of the communication unit 15 include, but are not limited to, those using mobile communication technologies such as cellular V2X and cellular V2P.

[0015] (Main control unit 17) The main control unit 17 may be a control unit that comprehensively controls the processing of the server 10. The main control unit 17 may include an information management unit 18 that manages various information in the main control unit 17, an external data acquisition unit 11, a teacher data generation unit 12, and a learning unit 13. The operation of each unit will be described later.

[0016] (Vehicle 20) The vehicle 20 may be, for example, a car, a motorcycle, a bicycle, etc. The vehicle 20 may include an on-board sensor 21, a position information acquisition unit 22, a communication unit 23, and an on-board information processing device (detection device) 200.

[0017] (In-vehicle sensor 21) The on-vehicle sensor 21 may be a camera (image sensor) or a sensor such as LiDAR or radar. In one example, the on-vehicle sensor 21 may be a camera that is disposed on the vehicle 20 and captures an image of the outside of the vehicle 20. In another example, the on-vehicle sensor 21 may be a sensor that is disposed on the vehicle 20 and measures the outside of the vehicle 20. In this specification, data captured by the on-vehicle sensor 21 and measured the outside of the vehicle 20 is also referred to as first data.

[0018] (Location information acquisition unit 22) The location information acquisition unit 22 may have, for example, a GNSS (Global Navigation Satellite System) function or a network positioning function, and may acquire location information of the vehicle 20. The location information of the vehicle 20 acquired by the location information acquisition unit 22 is transmitted to the communication unit 15 of the server 10 via the communication unit 23 of the vehicle 20.

[0019] (Communications Department 23) The communication unit 23 may communicate with the server 10. Examples of the communication unit 23 include, but are not limited to, those using mobile communication technologies such as cellular V2X and cellular V2P.

[0020] (In-vehicle information processing device 200) The in-vehicle information processing device 200 may be an information processing device that is disposed in the vehicle 20 and detects objects located on the road outside the vehicle 20. For example, the in-vehicle information processing device 200 may not only detect the presence or absence of an object, but also estimate the type of object and its distance from the vehicle 20. According to the information processing system 1 according to this embodiment, the accuracy of detection and estimation by the in-vehicle information processing device 200 can be improved. The in-vehicle information processing device 200 may include a main control unit 207, an output unit 203, and a memory unit 204.

[0021] (Main control unit 207) The main control unit 207 may be a control unit that comprehensively controls the processing of the in-vehicle information processing device 200. The main control unit 207 may include an external data acquisition unit (acquisition unit) 201, a detection unit 202, and an information management unit 208. The operation of each unit will be described later.

[0022] (output unit 203) The output unit 203 may include a display, a speaker, and the like, and may output information to a person operating or driving the vehicle 20. Furthermore, if the vehicle 20 is a fully autonomous vehicle, or if there is no person operating or driving the vehicle 20, the output unit 203 may output information to an autonomous driving control program stored in the memory unit 204.

[0023] (Storage unit 204) The storage unit 204 may be a storage device that stores various data such as data indicating the type (for example, "vehicle" or "automobile" which is identification information of the vehicle 20). The storage unit 204 may store various data such as an automatic driving control program when there is no one to operate or drive the vehicle 20, such as when the vehicle 20 is a fully automatic driving vehicle.

[0024] [Management of information on road objects] Next, a method for managing information on road objects by the server 10 will be described with reference to Fig. 2. Fig. 2 is a diagram showing the relationship between the server 10 and mobile communication terminals.

[0025] In one example, server 10 may communicate with mobile communication terminals located on the road (for example, vehicles with communication capabilities, mobile communication terminals carried by passersby, etc.) and acquire from these terminals the position information and type information of each terminal, thereby managing the position information and type of road objects that indicate their positional relationship with vehicle 20. The road objects managed by server 10 may be the mobile communication terminals themselves, or may be entities (for example, vehicles, passersby, etc.) that have mobile communication terminals.

[0026] In the following, as an example, a description will be given assuming that a vehicle (road object) 20' other than the vehicle 20 and a pedestrian (road object) H are on the road.

[0027] (Vehicle terminal device 30) The vehicle terminal device 30 is a mobile communication terminal including a communication unit 31, a location information acquisition unit 32, and a storage unit 33, and may be installed in a vehicle 20' separate from the vehicle 20.

[0028] The communication unit 31 may be capable of communicating with the server 10 using mobile communication technology. The location information acquisition unit 32 may have, for example, a GNSS function or a network positioning function, and may acquire location information of the vehicle terminal device 30. The storage unit 33 may store data indicating the type, which is identification information of the object equipped with the vehicle terminal device 30. The vehicle terminal device 30 is installed in a vehicle, and the storage unit 33 may store a fixed value indicating "vehicle" as identification information indicating the type.

[0029] The pedestrian terminal device 40 may be a mobile communication terminal including a communication unit 41, a location information acquisition unit 42, an object type determination unit 43, and a storage unit 44, and may be carried by the pedestrian H. The pedestrian terminal device 40 is a mobile communication terminal and may be carried not only by a walking pedestrian, but also by a driver of a vehicle such as a bicycle, a motorcycle, or an automobile.

[0030] The communication unit 41 may be capable of communicating with the server 10 using mobile communication technology. The location information acquisition unit 42 may have, for example, a GNSS function or a network positioning function, and may acquire location information of the passerby terminal device 40.

[0031] The object type determination unit 43 may determine the type of road object that indicates what type of road object the pedestrian terminal device 40 is attached to, in other words, what type of road object the pedestrian terminal device 40 moves with. As an example, the object type determination unit 43 may determine the type of the road object based on the moving speed of the pedestrian terminal device 40, etc. In this case, the object type determination unit 43 may determine whether the type of the object is a vehicle or a pedestrian. Furthermore, the object type determination unit 43 may further determine whether the vehicle is an automobile, bicycle, or motorcycle, and whether the pedestrian is a pedestrian or a running person. The object type determination unit 43 may store the determination result in the storage unit 44 as identification information indicating the type of object that is attached to the pedestrian terminal device 40.

[0032] The storage unit 44 may store identification information indicating the type of object that includes the passerby terminal device 40.

[0033] In one example, the external data acquisition unit 11 of the server 10 may acquire the second data by using the position information and identification information of the vehicle terminal device 30 and the pedestrian terminal device 40 as the position of the road object.

[0034] More specifically, the external data acquisition unit 11 may communicate with the vehicle terminal device 30 via the communication unit 15 and acquire location information of the vehicle terminal device 30 and identification information (e.g., "vehicle") of another vehicle 20' in which the vehicle terminal device 30 is located. The information management unit 18 may manage the location information and identification information as location information and identification information of the vehicle equipped with the vehicle terminal device 30.

[0035] Furthermore, the external data acquisition unit 11 may communicate with the pedestrian terminal device 40 via the communication unit 15 to acquire location information of the pedestrian terminal device 40 and identification information (e.g., the pedestrian) of the pedestrian carrying the pedestrian terminal device 40. The information management unit 18 may manage the location information and identification information of the pedestrian H carrying the pedestrian terminal device 40.

[0036] In addition, in one example, the information management unit 18 may manage the position information of each object by linking it with data indicating the type of the object, which is identification information of the object. For example, the information management unit 18 may manage the position information of the vehicle terminal device 30 by linking it with identification information indicating the type of "vehicle," and manage the position information of the pedestrian terminal device 40 by linking it with identification information indicating the type of "pedestrian." The information management unit 18 may manage the position information of the vehicle terminal device 30 by linking it with identification information indicating the type of "automobile" within the category of "vehicle," and manage the position information of the pedestrian terminal device 40 by linking it with identification information indicating the type of "pedestrian."

[0037] As described above, in one example, the server 10 can manage information about road objects by acquiring information from mobile communication terminals located on the road. Note that the method by which the server 10 manages information about road objects is not limited to this, and any known method may be used. For example, if pedestrians, vehicles, etc. have radio wave output devices that periodically or irregularly output radio waves, the server 10 may acquire and manage information about pedestrians, vehicles, etc. by detecting the radio waves using detectors located at various positions.

[0038] In the above example, a fixed value indicating a vehicle is stored in advance in the storage unit 33, and therefore the vehicle terminal device 30 does not include an object type determination unit that stores identification information of the object equipped with the vehicle terminal device 30. However, in the present embodiment, the vehicle terminal device 30 may determine the type of the object equipped with the vehicle terminal device 30 based on the moving speed of the object, etc., to obtain identification information indicating the type of the object, and store the identification information indicating the type of the object in the storage unit 33. This allows the storage unit 33 to store identification information indicating the type of vehicle, as in the above example. Therefore, the server 10 can manage object information in the same way as in the above example.

[0039] [Learning model learning method] A learning method for the learning model will be described with reference to Figures 3 to 5. Figure 3 is a flowchart showing an example of the learning method for the learning model. In one example, the information processing system (computer) 1 may execute a transmission process S11, a reception process S12, an external data acquisition process (acquisition process) S13, a teacher data generation process S14, and a learning process S15.

[0040] (Transmission process S11) The communication unit 23 of the vehicle 20 may transmit the first data detected by the on-board sensor 21 of the vehicle 20 and the location information of the vehicle 20 acquired by the location information acquisition unit 22 of the vehicle 20 to the server 10. The communication unit 23 may also include a timestamp in the information to be transmitted.

[0041] In addition, the communication unit 23 may transmit third data indicating the detection result of an object outside the vehicle 20 obtained by inputting the first data into the learning model M to the communication unit 15 of the server 10 instead of or in addition to the first data.

[0042] (Reception process S12) The communication unit 15 of the server 10 may receive the first data and the location information of the vehicle 20 from the communication unit 23 of the vehicle 20. If the first data is video data, the communication unit 15 may acquire only one frame of the video data. The number of pieces of first data received by the communication unit 15 is not particularly limited.

[0043] When the communication unit 15 receives the first data and the position information of the vehicle 20 from the vehicle 20, the information management unit 18 may manage the first data and the position information by linking them to a timestamp. Furthermore, when the communication unit 15 receives the position information and identification information (second data) of an object on the road from the vehicle terminal device 30 and the pedestrian terminal device 40, the information management unit 18 may manage the position information and the identification information by linking them to a timestamp. This allows the information management unit 18 to manage the first data and the second data received at multiple times in chronological order.

[0044] (External data acquisition process S13) The external data acquisition unit 11 may acquire the first data and the second data managed by the information management unit 18 from the information management unit 18. By linking the first data and the second data with time series data such as a timestamp, the external data acquisition unit 11 can acquire the first data and the second data acquired at the same timing from the information management unit 18.

[0045] (Teacher data generation process S14) The teacher data generating unit 12 may generate teacher data based on the first data and the second data acquired by the external data acquiring unit 11.

[0046] As described above, the first data may be data obtained by capturing or measuring an image of the outside of the vehicle 20 using a camera or a sensor disposed on the vehicle 20. The second data may be information indicating the position of an object on the road acquired by the server 10 that manages information about road objects located on the road, and may further include identification information of the object on the road.

[0047] The training data is training data for machine learning the learning model M, which receives the first data as input and outputs the third data indicating the detection result of an object outside the vehicle 20.

[0048] In one aspect, the third data includes information indicating a portion of the first data corresponding to the detected external object. For example, if the first data is a captured image, the information may be information indicating a range (e.g., a bounding box) in the captured image where the detected external object is captured, or if the first data is measurement data, the information may be information indicating a portion of the measurement data where the detected external object is measured.

[0049] As the correct answer data (label data) of such third data, the teacher data generating unit 12 refers to the second data and generates information indicating a portion of the first data corresponding to the road object. For example, if the first data is a captured image, the information may be information indicating a range in the captured image where the road object is likely to be captured, or if the first data is measurement data, the information may be information indicating a portion in the measurement data where the road object is likely to be measured.

[0050] That is, the training data generation unit 12 identifies external objects to be detected based on the first data based on the position information and identification information of road objects, and generates training data, thereby enabling the learning model M to learn to detect external objects that it was previously unable to detect.

[0051] (An example of the teacher data generation process S14) An example of the teacher data generation process S14 will be described with reference to Figures 4 to 9. In the following, an example will be described in which the teacher data generation unit 12 identifies positions where second data (position information and identification information of an object) are displayed in the first data (captured image) in the order of (i) to (v), and displays a bounding box and a label at the positions to generate third data.

[0052] (i) An example of the first data First, an example of the first data will be described with reference to Fig. 4. Fig. 4 is a diagram showing an example of the first data. The first data shown in Fig. 4 is a captured image I1. In the captured image I1, a vehicle 20' different from the vehicle 20 and a passerby H carrying a passerby terminal device 40 are captured in a state in which their type data can be identified.

[0053] (ii) Examples of detectable areas and imageable areas Next, an example of a detectable area in which the detection unit 202 can detect objects such as a vehicle 20' different from the vehicle 20 and a pedestrian H based only on the first data before learning the learning model M in this embodiment, and an imageable area in which the on-vehicle sensor 21 can capture an image of the object will be described with reference to Figures 5 to 8. Figures 5 to 8 will be described taking as an example a case in which the object is a pedestrian H and the on-vehicle sensor 21 is a camera.

[0054] 5 is a diagram showing an example of a detectable area and an imageable area. The detectable area r1 is an area in which the detection unit 202, before learning the learning model M in this embodiment, can detect a pedestrian H based only on the first data. The undetectable area r2 is an area in which the detection unit 202, before learning the learning model M in this embodiment, cannot detect a pedestrian H based only on the first data. The imageable area R is an area in which the on-board sensor 21 of the vehicle 20 can capture an image of the pedestrian H. As shown in FIG. 5, the imageable area R includes the detectable area r1 and the undetectable area r2.

[0055] Therefore, when pedestrian H is located in undetectable area r2 of imageable area R as shown in the upper diagram of Fig. 5, and detection unit 202 detects an object based only on the first data, there is a risk that the detection unit 202 will output the same detection result as when pedestrian H is not present as shown in the lower diagram of Fig. 5. In contrast, when training data generation unit 12 generates training data based on the first data and the second data and uses learning model M that has been machine-learned from the training data, detection unit 202 will be able to detect pedestrian H in undetectable area r2.

[0056] 6 is a diagram showing an example of an imageable area in the vertical direction. d shown in FIG. 6 indicates the distance of a pedestrian H from the on-board sensor 21. The imageable area R is a position where the distance from the on-board sensor 21 in the z-axis direction (horizontal direction in FIG. 6) is a position where the distance from the on-board sensor 21 in the y-axis direction (vertical direction in FIG. 6) is h a Hereinafter, at the position b, the distance in the y-axis direction from the on-board sensor 21 is h bHereinafter, at the position c, the distance in the y-axis direction from the on-board sensor 21 is h c The areas are as follows:

[0057] 6, a pedestrian H located at a position p in the z-axis direction from the on-board sensor 21 is included in the imageable area R and therefore appears in the captured image I1. On the other hand, a pedestrian H located at a position a in the z-axis direction from the on-board sensor 21 is not included in the imageable area R and therefore does not appear in the captured image I1.

[0058] 6 indicates a distance that satisfies the following formula (1), and the latitude and longitude of the pedestrian H and the latitude and longitude of the vehicle 20 are included in the position information (data indicating the position) of the pedestrian H and the vehicle 20, which are part of the second data. An example of the second data will be described later.

number

[0059] Furthermore, h indicates the height of the vehicle-mounted sensor 21, and satisfies the following formula (2): h ph h refers to the height of pedestrian H (for example, 165cm = 1.65m). p indicates the distance in the z-axis direction from the position p of the vehicle-mounted sensor 21 to the pedestrian H, and satisfies the following equation (3) in addition to the equation shown in FIG. 6. In the example shown in FIG. 6, θ pv indicates the angle formed by the pedestrian H at the position p with respect to the z-axis direction, which is the base line direction of the vehicle-mounted sensor 21. p satisfies the following equation (4). h and h ph is a predetermined value, and d is obtained by substituting the second data into the above-mentioned formula (1). Therefore, the teacher data generating unit 12 can obtain h, h ph By substituting the values ​​of θ and d into the following equation (3), pv Furthermore, the teacher data generating unit 12 can calculate the calculated θ pv By substituting into equation (4), p can be calculated. h=h ph +h p ···(2) hp =hh ph =dsinθ pv ···(3) p=dcosθ pv ···(4)

[0060] 7 is a diagram showing an example of an imageable area in the horizontal direction. The imageable area R is a position where the distance in the z-axis direction from the on-board sensor 21 is z1 and the distance in the x-axis direction from the on-board sensor 21 is x h Below, at the position where the distance in the z-axis direction is z2, the distance in the x-axis direction is x f The areas are as follows:

[0061] 7, a pedestrian H present at a position (coordinates (x1, z1)) at a distance z1 in the z-axis direction from the on-board sensor 21 is included in the imageable area R and therefore appears in the captured image I1. On the other hand, a pedestrian H present at a position (coordinates (x2, z2)) at a distance z2 in the z-axis direction from the on-board sensor 21 is not included in the imageable area R and therefore does not appear in the captured image I1.

[0062] From Figure 7, x h , z l , x1 satisfy the following formulas (5) to (7). In the example shown in FIG. h indicates the angle of the boundary of the imageable area R with respect to the z-axis, which is the base line direction of the vehicle-mounted sensor 21. p1 indicates the angle of the pedestrian H at the coordinate (x1, z1) relative to the x-axis, which is a direction perpendicular to the base line direction of the vehicle-mounted sensor 21. x h =z1tan1 / 2θ h ···(5) z1=dsinθ p1 ···(6) x1=dcosθ p1 ···(7)

[0063] Also, by substituting equation (6) into equation (5), x h satisfies the following equation (8), and by dividing equation (7) by equation (8), the following equation (9) is satisfied. x h =dsinθp1 tan1 / 2θ h ···(8) x1 / x h =1 / tanθ p1 tan1 / 2θ h ···(9)

[0064] Fig. 8 is a diagram showing an example of an imageable area shown in a captured image. In the example shown in Fig. 8, the imageable area R is represented as a captured image I1. As shown in Fig. 8, the length of the captured image I1 in the x-axis direction (the horizontal direction in Fig. 8) is the horizontal range shown in the captured image I1, and the length of the captured image I1 in the y-axis direction (the vertical direction in Fig. 8) is the vertical range shown in the captured image I1, and the entire range of these is the imageable area R. Therefore, Fig. 8 is also a diagram showing an example of the relationship between the imageable area R and the captured image I1.

[0065] Furthermore, the imageable area R is also an area where the length of the pedestrian H in the x-axis direction on the captured image I1 is less than half the length of the captured image I1 in the x-axis direction, and the length of the pedestrian H in the y-axis direction on the captured image I1 is less than half the length of the captured image I1 in the y-axis direction. From Fig. 8, the imageable area R is also an area that satisfies the following expressions (10) and (11). αdcosθ pv ≦αdsinθ pv tan1 / 2θ h ···(10) αdsinθ pv ≦αdsin1 / 2θ v ···(11)

[0066] In the example shown in FIG. 8, the angle of the pedestrian H from the x-axis direction (horizontal direction in FIG. 8), which is perpendicular to the base line direction of the vehicle-mounted sensor 21, is θ pv and θ in Fig. 7 p1 is θ pv This explains the case where dsinθ in Equation (10) pv tan1 / 2θ h is the coordinate of the passerby H in equation (8) from x1 to x h and the angle θ p1 θ pvIt is also replaced by dcosθ pv satisfies the following equation (12): d sin θ in equation (11) pv is the h in Eq. (3) p =hh ph =dsinθ pv Corresponds to. dcosθ pv = latitude of pedestrian H - latitude of vehicle 20 (12)

[0067] In addition, α in equations (10) and (11) is 2d sinθ pv tan1 / 2θ h and 2dsin1 / 2θ v This is the coefficient to fit within 300 pixels. pv tan1 / 2θ h and 2dsin1 / 2θ v is multiplied by the coefficient, the distance d of the pedestrian H from the vehicle-mounted sensor 21 is also multiplied by the coefficient. Therefore, dcosθ in equations (10) and (11), which respectively indicate the coordinate of the pedestrian H in the x-axis direction and the coordinate of the pedestrian H in the y-axis direction, pv and dsinθ pv are also multiplied by the coefficients.

[0068] (iv) Example of calculating the position of an object in a captured image Next, an example of calculating the position of an object in a captured image will be described. Hereinafter, an example of calculating the position of a passerby H in a captured image I1 will be described with reference to FIGS.

[0069] As shown in FIG. 8, the coordinates of the passerby H in the captured image I1 are (αdcosθ pv , αdsinθ pv )

[0070] For example, the teacher data generating unit 12 substitutes the distance between the vehicle 20 and the pedestrian H (for example, 25 m) and the difference between the latitude of the pedestrian H and the latitude of the vehicle 20 (for example, 0.001°) into d and the latitude of the pedestrian H - the latitude of the vehicle 20 in the equation (12), respectively, to obtain cosθ pvCalculate the calculated cosθ pv Based on θ pv Calculate.

[0071] Also, assume that the height of the vehicle-mounted sensor 21 from the road is 2.0 m, the height of the pedestrian H is 1.7 m, and the distance between the vehicle 20 and the pedestrian H is 25 m. In this case, the training data generating unit 12 uses these values ​​as h and h in equation (3). ph , and d respectively, and then sinθ pv is calculated to be 0.3 (= 2.0 - 1.7) / 25 = 0.012. pv Based on θ pv is calculated to be 0.68756585641086°.

[0072] The teacher data generating unit 12 calculates the θ pv αdcosθ pv and αdsinθ pv and specify the position of the passerby H in the captured image I1.

[0073] (v) Example of generating training data An example of generating training data will be described with reference to Figures 8 and 9. Figure 9 is a diagram showing an example of training data.

[0074] The teacher data generation unit 12 may generate teacher data by displaying the position of an object in the captured image based on the second data. For example, the teacher data generation unit 12 may generate, based on the second data, information indicating a bounding box (a portion of the first data corresponding to a road object) B2 surrounding the pedestrian H and identification information (type of road object) C2 associated with the bounding box at the position of the pedestrian H in the captured image I1 calculated as in the example of (iv) calculating the position of an object in a captured image, and add the information to the first data, thereby generating teacher data. Similarly, the teacher data generation unit 12 may generate information indicating a bounding box B1 surrounding another vehicle 20' and identification information C1 at the position of another vehicle 20' in the captured image I1 identified in the same manner as the pedestrian H, and add the information to the first data, thereby generating teacher data.

[0075] As a result, the training data generation unit 12 generates, as training data, (i) a bounding box B1 indicating the position in the image I3 (third data) of the vehicle 20' detected as an object, (ii) identification information C1 indicating the type of object indicated by the bounding box B1, (iii) a bounding box B2 indicating the position in the image I3 of the pedestrian H detected as an object, and (iv) an image I3 to which identification information C2 indicating the type of object indicated by the bounding box B2 has been added. In this way, the training data may include information indicating the type of the road object in addition to information indicating the portion of the first data corresponding to the road object. The training data may be managed by the information management unit 18, similar to the first data and the second data.

[0076] In the above example, the teacher data generation unit 12 includes the identification information C1 of "vehicle" and the identification information C2 of "pedestrian" in the teacher data. However, for example, if the data on the types of another vehicle 20' and pedestrian H in the second data includes "automobile," which is a subconcept of "vehicle," and "pedestrian," which is a subconcept of "pedestrian," the teacher data generation unit 12 may replace the identification information C1 and C2 with these.

[0077] (Learning process S15) The learning unit 13 of the server 10 can train the learning model M by machine learning using the training data generated in the training data generation process S14, so that the learning model M receives first data as input and outputs third data indicating the object detection result.

[0078] For example, the learning unit 13 may construct a learning model M by having a neural network such as an SSD (Single Shot Multibox Detector) learn the teacher data generated in the teacher data generation process S14 through machine learning. By having such a neural network learn the teacher data through machine learning, the learning unit 13 can construct a learning model M that can detect multiple objects from one image data and recognize (label) identification information of the detected objects.

[0079] [Detection method] Next, the detection method according to the first embodiment will be described with reference to Fig. 10. Fig. 10 is a flowchart showing an example of the detection method according to the first embodiment.

[0080] In the detection method according to the first embodiment, the information processing system 1 may execute a transmission process S21, a reception process S22, an external data acquisition process (acquisition process) S23, a detection process S24, and an output process S25.

[0081] (Transmission process S21) The communication unit 15 of the server 10 may transmit the learning model M learned in the learning process S15 to the communication unit 23 of the vehicle 20.

[0082] (Reception process S22) The communication unit 23 of the vehicle 20 may receive the learning model M. The information management unit 208 may store the learning model M received by the communication unit 23 in the memory unit 204.

[0083] (External data acquisition process S23) The external data acquisition unit 201 may acquire first data detected by the on-board sensor 21 of the vehicle 20.

[0084] (Detection process S24) The detection unit 202 in the vehicle information processing device 200 of the vehicle 20 may input the first data acquired in the external data acquisition process S23 into the learning model M received in the receiving process S22 and stored in the memory unit 204, and detect an object outside the vehicle 20.

[0085] (Output process S25) The output unit 203 may output third data indicating the detection result of the object detected in the detection process S24. For example, the output unit 203 may output third data in which a bounding box indicating the position of the external object and information indicating the type of the object, which is identification information of the object, are added to the first data.

[0086] If the detection unit 202 attempts to detect an object before learning the learning model M, the pedestrian H in the first data, such as an image captured by the on-board sensor 21, may be too small for the detection unit 202 to identify the pedestrian H as a pedestrian and not detect it as an object. As a result, the output unit 203 may output third data with low detection accuracy, such as one that does not include a bounding box or identification information indicating the type of object, or one that includes a bounding box in an incorrect position.

[0087] In response to this, the training data generation unit 12 generates training data by adding position information and identification information of real-world objects to the first data, even if the objects cannot be detected as objects from the first data. The learning unit 13 generates a learning model M by machine learning the training data. This allows the detection unit 202 to use the trained learning model M to detect the object of the vehicle 20 regardless of the situation, even if the object cannot be seen or identified in the first data from the first data alone, thereby improving object detection performance. As a result, it is possible to output third data to which highly accurate bounding boxes and identification information have been added.

[0088] The output unit 203 may display an image in which the bounding box indicated by the third data is superimposed on the image indicated by the first data as the object detection result on a car navigation system, etc. This allows the passengers of the vehicle 20 to be notified of another vehicle 20' and pedestrian H, which are objects of the vehicle 20, thereby reducing the possibility of the vehicle 20 coming into contact with or crashing into the other vehicle 20' or pedestrian H.

[0089] The output unit 203 may output at least one of a warning display and a warning sound when the distance of each object to the vehicle 20 is less than a predetermined value and / or the approaching speed of each object to the vehicle 20 is equal to or greater than a predetermined value. In this way, the output unit 203 can prevent a collision between the vehicle 20 and each object.

[0090] <Modification> In the above example, the vehicle 20 is the target, but the present embodiment is not limited to this. Instead of a vehicle, a general moving object such as a drone can be similarly targeted. In this way, when the vehicle 20 is replaced with a drone, the second data may further include altitude information of each object from the ground.

[0091] Furthermore, in the above example, the one or more objects to be detected include one other vehicle 20' and one pedestrian H, but the number of other vehicles 20' and pedestrians H is not particularly limited, and for example, the number of one may be zero or more, or the number of both may be more than one.

[0092] <Embodiment 2> In the information processing system according to the present invention, like the information processing system 1α according to the second embodiment, the on-board information processing device 200α of the vehicle 20α may perform learning and detection.

[0093] An information processing system 1α according to the second embodiment will be described below with reference to Figures 11 and 12. For ease of explanation, components having the same functions as those described in the above embodiments will be denoted by the same reference numerals, and their description will be omitted.

[0094] [Information Processing System 1α] 11 is a block diagram showing an example of the configuration of an information processing system according to embodiment 2. The information processing system 1α may include a server 10α and a vehicle 20α instead of the server 10 and the vehicle 20 according to embodiment 1. Except for this, the information processing system 1α according to embodiment 2 may have the same configuration as the information processing system 1 according to embodiment 1.

[0095] (Server 10α) The server 10α may include a main control unit 17α and a storage unit 14α instead of the main control unit 17 and the storage unit 14 in the first embodiment. The main control unit 17α does not include the teacher data generation unit 12 and the learning unit 13 in the first embodiment, and the storage unit 14α does not include the learning model M in the first embodiment. Except for this, the server 10α may have the same configuration as the server 10 according to the first embodiment.

[0096] (Vehicle 20α) The vehicle 20α may be equipped with an in-vehicle information processing device 200α instead of the in-vehicle information processing device 200 according to the first embodiment. Except for this, the vehicle 20α may have the same configuration as the vehicle 20 according to the first embodiment.

[0097] (In-vehicle information processing device 200α) The in-vehicle information processing device 200α may include a main control unit 207α and a storage unit 204α instead of the main control unit 207 and the storage unit 204 in the first embodiment, and the main control unit 207α may further include a teacher data generation unit 206 and a learning unit 205. Except for this, the in-vehicle information processing device 200α may have the same configuration as the in-vehicle information processing device 200 according to the first embodiment.

[0098] The teacher data generation unit 206 and the learning unit 205 may have the same configuration as the teacher data generation unit 12 and the learning unit 13 in the first embodiment, respectively. The storage unit 204α may further include a learning model M. Except for this, the storage unit 204α may have the same configuration as the storage unit 204 in the first embodiment.

[0099] [Learning Method] Next, a learning method according to the second embodiment will be described with reference to Fig. 12. Fig. 12 is a flowchart showing an example of the learning method according to the second embodiment.

[0100] In the learning method according to the second embodiment, the information processing system 1α may execute a transmission process S31, a reception process S32, an external data acquisition process S33, a teacher data generation process S34, and a learning process S35. The transmission process S31, the reception process S32, the external data acquisition process S33, and the teacher data generation process S34 are similar to the transmission process S11, the reception process S12, the external data acquisition process S13, and the teacher data generation process S14 in the first embodiment, respectively.

[0101] <Embodiment 3> The information processing system according to the present invention, like the information processing system 1β according to embodiment 3, may estimate the orientation (posture) of the vehicle 20 based on data indicating the position of a static object (specific object) among the objects contained in the first data, the position of which has been specified in advance.

[0102] An information processing system 1β according to the third embodiment will be described below with reference to Figures 13 to 16. For ease of explanation, the same reference numerals are used to designate components having the same functions as those described in the above embodiments, and the description thereof will be omitted.

[0103] [Information Processing System 1β] 13 is a block diagram showing an example of the configuration of an information processing system according to embodiment 3. The information processing system 1β may include a server 10β instead of the vehicle 20 and the server 10 according to embodiment 1. Except for this, the information processing system 1β according to embodiment 3 may have the same configuration as the information processing system 1 according to embodiment 1.

[0104] (Server 10β) The server 10β may include a main control unit 17β instead of the main control unit 17 in the first embodiment, and the main control unit 17β may further include an estimation unit 19, and may include a teacher data generation unit 12β instead of the teacher data generation unit 12 in the first embodiment. Except for this, the server 10β may have the same configuration as the server 10 according to the first embodiment.

[0105] (Estimation part 19) The estimation unit 19 may estimate the orientation of the vehicle 20 based on data indicating the positions of static objects among the objects included in the first data.

[0106] (Teacher Data Generation Unit 12β) The teacher data generating unit 12β may generate the third data based on the first data and the second data including the orientation of the vehicle 20 estimated by the estimating unit 19.

[0107] [Generation method] Next, a generation method according to the third embodiment will be described with reference to Fig. 14. Fig. 14 is a flowchart showing an example of the generation method according to the third embodiment.

[0108] In the generation method according to the third embodiment, the information processing system 1β may execute a transmission process S41, a reception process S42, an estimation process S43, an external data acquisition process S44, a teacher data generation process S45, and a learning process S46.

[0109] The transmission process S41, the reception process S42, the external data acquisition process S44, and the learning process S46 are similar to the transmission process S11, the reception process S12, the external data acquisition process S13, and the learning process S15 in the first embodiment, respectively.

[0110] (Estimation process S43) The estimation unit 19 of the server 10β may estimate the orientation of the vehicle 20 based on data indicating the positions of static objects among the objects included in the first data.

[0111] (An example of the estimation process S43) An example of the estimation process S43 will be described using Figures 15 and 16. Figure 15 is a diagram showing an example of the relationship between the orientation of a vehicle and an object. Figure 16 is a diagram showing an example of the relationship between an on-board sensor and an imageable area. In the example shown in Figure 15, a case will be described in which the on-board sensor 21 of the vehicle 20 is a camera.

[0112] 15, when the vehicle 20 is facing upward, the area Rα that can be captured by the on-board sensor 21 of the vehicle 20 includes the entire image of the traffic light T and road W, which are static objects, and the entire image of the pedestrian H, which is a dynamic object. In this case, the captured image I1α captured by the on-board sensor 21 of the vehicle 20 includes the entire image of the traffic light T and road W, which are static objects, and the entire image of the pedestrian H, which is a dynamic object.

[0113] On the other hand, when the vehicle 20 is facing downward, the measurable area Rβ of the on-board sensor 21 of the vehicle 20 does not include the traffic light T in front of the on-board sensor 21, and does not include a part of the traffic light T at the back. In this case, the captured image I1β captured by the on-board sensor 21 of the vehicle 20 does not include the traffic light T in front of the on-board sensor 21, and does not include a part of the traffic light T at the back.

[0114] In this way, the orientation of the on-board sensor 21 changes depending on the orientation of the vehicle 20, and the objects captured in the image captured by the on-board sensor 21 also change. Therefore, if the orientation of the vehicle 20 differs from that expected by the person operating or driving the vehicle 20, for example, the angle of the pedestrian H relative to the on-board sensor 21 shown in FIGS. 6 to 8 included in the second data will differ, and robustness will be impaired.

[0115] In response to this, the estimation unit 19 estimates the orientation of the vehicle 20 based on data indicating the positions of static objects among the objects included in the first data. In the example shown in Fig. 15, when the entire image of a traffic light T on the far side of the vehicle 20 is captured near the center of the captured image I1α, the estimation unit 19 estimates that the orientation of the vehicle 20 is upward based on the data indicating the position of the traffic light T, which is a static object. Similarly, when only the bottom edge of the traffic light T on the far side is captured at the top edge of the captured image I1β, the estimation unit 19 estimates that the orientation of the vehicle 20 is downward.

[0116] Since the absolute position of a static object seen from a vehicle 20 moving in the same direction and in the same place is the same, the estimation unit 19 can estimate the orientation of the vehicle 20 with high accuracy by performing the estimation as described above. The data indicating the orientation of the vehicle 20 estimated by the estimation unit 19 is stored in the storage unit 204 as second data and managed by the information management unit 208.

[0117] The estimation unit 19 may, for example, generate data reproduced by a database or digital twin technology based on the first data, and estimate the orientation of the vehicle 20 based on the data generated based on the first data. This allows the estimation unit 19 to estimate the orientation of the vehicle 20 with higher accuracy.

[0118] Here, the estimation unit 19 can acquire data indicating the position of a stationary object with particularly high reliability when the stationary object is located within a LiDER measurable area Rβ, which is within the angle of view of both the camera and the LiDER, as shown in Fig. 16. On the other hand, the estimation unit 19 can acquire data indicating the position of a stationary object (such as a traffic light T at the back of Fig. 16) that is not included in the LiDER measurable area Rβ but is included in the camera's imageable area Rγ.

[0119] (Teacher data generation process S45) The teacher data generation unit 12β may generate third data based on the first data and second data including the position information, identification information, and data indicating the orientation of the vehicle 20 estimated by the estimation unit 19 in the estimation process S43 of the vehicle 20.

[0120] (An example of the teacher data generation process S45) An example of the teacher data generation process S45 will be described with reference to Figures 6 to 8. The teacher data generation unit 12β calculates the angle of the pedestrian H with respect to the on-board sensor 21 in Figures 6 to 8 (θ pv , θ in Fig. 7 p1 and θ in Fig. 8 pv ) is adjusted to the correct value.

[0121] This allows the teacher data generation unit 12β to adjust the second data, including the angle of the pedestrian H relative to the on-board sensor 21 in Figures 6 to 8, so that it matches the expectations of the person operating or driving the vehicle 20. As a result, the teacher data generation unit 12β can generate high-quality third data based on the highly reliable second data, thereby ensuring robustness.

[0122] [Hardware configuration and software implementation example] The servers 10, 10α, and 10β and the control blocks of the on-board information processing devices 200, 200α, and 200β in the vehicles 20 and 20α may be realized by logic circuits (hardware) formed on integrated circuits (IC chips) or the like, or may be realized by software using a CPU (Central Processing Unit). In the latter case, the servers 10, 10α, and 10β and the on-board information processing devices 200, 200α, and 200β can be configured using a computer (electronic calculator) such as that shown in FIG.

[0123] FIG. 17 is a block diagram illustrating the configuration of a computer 910 that can be used as the servers 10, 10α, and 10β, and the in-vehicle information processing devices 200, 200α, and 200β. The computer 910 includes an arithmetic unit 912, a main memory device 913, an auxiliary memory device 914, and an input / output interface 915, all of which are connected to one another via a bus 911. The arithmetic unit 912, the main memory device 913, and the auxiliary memory device 914 may each be, for example, a CPU, a RAM (random access memory), or a hard disk drive. The input / output interface 915 is connected to an input device 920 through which a user inputs various information to the computer 910, and an output device 930 through which the computer 910 outputs various information to the user. The input device 920 and the output device 930 may be built into the computer 910 or may be connected (externally) to the computer 910. For example, the input device 920 may be a keyboard, a mouse, a touch sensor, etc., and the output device 930 may be a display, a printer, a speaker, etc. Also, a device having the functions of both the input device 920 and the output device 930, such as a touch panel in which a touch sensor and a display are integrated, may be applied. The communication interface 916 is an interface that allows the computer 910 to communicate with external devices.

[0124] The auxiliary storage device 914 stores a learning program and a detection program for operating the computer 910 as the servers 10, 10α, and 10β, and the in-vehicle information processing devices 200, 200α, and 200β. The arithmetic device 912 then loads the learning program and detection program stored in the auxiliary storage device 914 onto the main storage device 913 and executes instructions contained in the learning program and detection program, thereby causing the computer 910 to function as each unit of the servers 10, 10α, and 10β, and the in-vehicle information processing devices 200, 200α, and 200β. Note that the recording medium used by the auxiliary storage device 914 to record information such as the learning program and detection program may be any computer-readable "non-transitory tangible medium," such as a tape, disk, card, semiconductor memory, or programmable logic circuit.

[0125] Alternatively, the computer 910 may be configured to function using a learning program and a detection program stored on a recording medium external to the computer 910, or a program supplied to the computer 910 via any transmission medium (such as a communication network or broadcast waves).The present invention may also be realized in the form of a data signal embedded in a carrier wave, in which the program is embodied by electronic transmission.

[0126] [Additional Notes] A part of the invention described in this specification can be described as, but is not limited to, the following supplementary notes.

[0127] (Appendix 1) a training data generation unit that generates training data for machine learning a learning model that receives input of first data obtained by capturing or measuring the outside of the mobile body using a camera or sensor disposed on the mobile body, and second data indicating the positions of road objects acquired by a server that manages information on road objects located on roads, and outputs third data indicating the detection results of objects outside the mobile body; and A learning device comprising: a learning unit that uses the teacher data to machine-train the learning model.

[0128] (Appendix 2) The computer a training data generation process for generating training data for machine learning a learning model that receives input of first data obtained by capturing or measuring the outside of a mobile body using a camera or sensor disposed on the mobile body, and second data indicating the positions of road objects acquired by a server that manages information on road objects located on roads, and outputs third data indicating the detection results of objects outside the mobile body; A learning method that executes a learning process in which the learning model is machine-learned using the training data.

[0129] (Appendix 3) A detection device comprising a detection unit that detects objects outside a moving body using a learning model that is machine-trained using training data based on first data obtained by capturing or measuring the outside of the moving body using a camera or sensor disposed on the moving body, and second data indicating the position of the road object obtained by a server that manages information on road objects located on the road, and that receives the first data as input and outputs third data indicating the detection result of the object outside the moving body.

[0130] The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope of the claims. Embodiments obtained by appropriately combining the technical means disclosed in different embodiments are also included in the technical scope of the present invention.

[0131] Furthermore, the present invention can detect objects around the vehicle 20 regardless of the situation, thereby contributing to the achievement of Goal 9 of the Sustainable Development Goals (SDGs), "Build resilient infrastructure, promote inclusive and sustainable industrialization, and promote innovation and infrastructure." [Explanation of symbols]

[0132] 1, 1α, 1β Information Processing System 10, 10α, 10β servers 11, 201 External data acquisition unit 12, 12β, 206 Training data generation section 13, 205 Learning Department 12, 206 Teacher data generation unit 19 Estimation part 20, 20α vehicles 20' Another vehicle 21 In-vehicle sensors 30 Vehicle terminal equipment 40 Passenger terminal device 43 Object type determination unit 200, 200α, 200β In-vehicle information processing device 202 Detection unit B1, B2 bounding box C1, C2 labels H Passerby I1, I1α, I1β, I1α, I1β images I3 Images S13, S23, S33, S44 External data acquisition processing S14, S34, S45 Training data generation process S15, S35, S46 Learning process S24 detection process S43 Estimation process

Claims

1. a training data generation unit that generates training data for machine learning a learning model that receives input of first data obtained by capturing or measuring the outside of the mobile body using a camera or sensor disposed on the mobile body and second data indicating the positions of road objects acquired by a server that manages information on road objects located on roads, and outputs third data indicating the detection results of objects outside the mobile body; and a learning unit that performs machine learning on the learning model using the training data; Equipped with the third data includes information indicating a portion of the first data corresponding to the external object; the training data includes information indicating a portion of the first data corresponding to the road object, In the third data, information indicating a type of the external object is associated with information indicating a portion of the first data; the second data includes information indicating a type of the road object; A learning device, wherein in the teacher data, information indicating the type of the road object is associated with information indicating a portion of the first data.

2. 2. The learning device according to claim 1, wherein the teacher data generation unit refers to the second data and identifies a portion of the first data that corresponds to the road object based on a positional relationship between the moving body and the road object.

3. 3. The learning device according to claim 2, wherein the teacher data generation unit identifies the orientation of the moving body based on the fact that a specific object whose position has been identified in advance is imaged or measured in the first data, and further identifies the portion of the first data that corresponds to the road object based on the identified orientation.

4. the first data is a captured image of the outside of the moving object captured by the camera, 4. The learning device according to claim 1, wherein the road object is a mobile communication terminal or a vehicle having a communication function.

5. A server comprising the learning device according to any one of claims 1 to 4.

6. A moving body comprising the learning device according to any one of claims 1 to 4.

7. A learning program for causing a computer to function as the learning device according to any one of claims 1 to 4, the learning program causing a computer to function as the teacher data generation unit and the learning unit.

8. The computer a training data generation process for generating training data for machine learning a learning model that receives input of first data obtained by capturing or measuring the outside of the mobile body using a camera or sensor disposed on the mobile body and second data indicating the positions of road objects acquired by a server that manages information on road objects located on the road, and outputs third data indicating the detection results of objects outside the mobile body; a learning process for machine learning the learning model using the training data; the third data includes information indicating a portion of the first data corresponding to the external object; the training data includes information indicating a portion of the first data corresponding to the road object, In the third data, information indicating a type of the external object is associated with information indicating a portion of the first data; the second data includes information indicating a type of the road object; In the teacher data, information indicating the type of the road object is associated with information indicating the portion of the first data. How to learn.

9. a detection unit that detects an object outside the mobile body using a learning model that is machine-trained using teacher data based on first data obtained by capturing or measuring the outside of the mobile body using a camera or sensor disposed on the mobile body and second data indicating the positions of the road objects acquired by a server that manages information on road objects located on roads, the learning model receiving the first data and outputting third data indicating the detection result of the object outside the mobile body; the third data includes information indicating a portion of the first data corresponding to the external object; the training data includes information indicating a portion of the first data corresponding to the road object, In the third data, information indicating a type of the external object is associated with information indicating a portion of the first data; the second data includes information indicating a type of the road object; In the teacher data, information indicating the type of the road object is associated with information indicating the portion of the first data. Detection device.

10. A moving object comprising the detection device according to claim 9.

Images