In-vehicle device, information processing method and program

Abstract

To provide a technique for accurately and efficiently determining a lane on which a front vehicle traveling in front of an own vehicle travels.SOLUTION: An on-vehicle device acquires a photographed image in which a front vehicle traveling in front of an own vehicle appears, and determines a lane on which the front vehicle travels by analysis of the acquired photographed image. When it is impossible to determine the lane on which the front vehicle travels by the analysis of the photographed image, the on-vehicle device acquires trajectory data indicating a trajectory of a relative position of the front vehicle with respect to the own vehicle, and determines the lane on which the front vehicle travels based on the acquired trajectory data.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to an in-vehicle device, an information processing method, and a program. [Background technology]

[0002] Patent Document 1 proposes an object recognition device for suppressing malfunction of a driving assistance device. Specifically, the object recognition device proposed in Patent Document 1 includes a distance detection unit, a distance change determination unit, and a suppression unit. The distance detection unit repeatedly detects the instantaneous distance from the vehicle to an object. The distance change determination unit determines whether the instantaneous distance currently detected by the distance detection unit for the target object is closer than the instantaneous distance previously detected and whether the amount of change in the instantaneous distance exceeds a threshold. If the determination result is affirmative, the suppression unit outputs a suppression signal that suppresses operation of the driving assistance device. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2020-199993 Summary of the Invention [Problem to be solved by the invention]

[0004] An object of the present disclosure is to provide a technology for accurately and efficiently determining the lane in which a vehicle traveling ahead of the vehicle is traveling. [Means for solving the problem]

[0005] An in-vehicle device according to a first aspect of the present disclosure includes a control unit configured to perform the following: acquire an image showing a vehicle traveling ahead of the vehicle; determine the lane in which the vehicle is traveling by analyzing the acquired image; if the lane in which the vehicle is traveling cannot be determined by analyzing the image, acquire trajectory data indicating the trajectory of the relative position of the vehicle relative to the vehicle; and determine the lane in which the vehicle is traveling based on the acquired trajectory data.

[0006] An information processing method according to a second aspect of the present disclosure is an information processing method executed by a computer, and includes acquiring an image showing a forward vehicle traveling in front of one's own vehicle, determining the lane in which the forward vehicle is traveling by analyzing the acquired image, and if the lane in which the forward vehicle is traveling cannot be determined by analyzing the image, acquiring trajectory data indicating the trajectory of the relative position of the forward vehicle with respect to the one's own vehicle, and determining the lane in which the forward vehicle is traveling based on the acquired trajectory data.

[0007] A program according to a third aspect of the present disclosure is a program for causing a computer to execute an information processing method, the information processing method including: acquiring a captured image showing a forward vehicle traveling ahead of a host vehicle; determining a lane in which the forward vehicle is traveling by analyzing the acquired captured image; if the lane in which the forward vehicle is traveling cannot be determined by analyzing the captured image, acquiring trajectory data indicating a trajectory of a position of the forward vehicle relative to the host vehicle; and determining the lane in which the forward vehicle is traveling based on the acquired trajectory data.

[0008] In each of the above aspects, acquiring a captured image showing a forward vehicle may include acquiring a captured image and detecting the forward vehicle in the acquired captured image. A trained machine learning model generated by machine learning may be used to detect the forward vehicle. The learning model includes one or more computational parameters for executing the computation of the inference process. The values ​​of the computational parameters of the machine learning model may be adjusted as appropriate to acquire the ability to perform the inference process of detecting an object, such as a vehicle, appearing in a given image through machine learning using the training data. The type of machine learning model may be selected as appropriate depending on the embodiment. The machine learning model may be, for example, a neural network. [Effects of the Invention]

[0009] According to the present disclosure, it is possible to accurately and efficiently determine the lane in which a vehicle traveling ahead of one's own vehicle is traveling. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 schematically illustrates an example of a situation to which the present disclosure is applied. [Figure 2] FIG. 2 is a diagram showing an example of a method for determining the lane of a preceding vehicle based on trajectory data (a method using road information) by an in-vehicle device according to an embodiment. [Figure 3] FIG. 3 is a diagram illustrating an example of a method for determining the lane of a leading vehicle based on trajectory data by an on-board device according to an embodiment (a method using trajectory data of surrounding vehicles). [Figure 4] FIG. 4 is a diagram illustrating an example of a hardware configuration of the in-vehicle device according to the embodiment. [Figure 5] FIG. 5 is a diagram illustrating an example of a software configuration of the in-vehicle device according to the embodiment. [Figure 6] FIG. 6 is a flowchart illustrating an example of a processing procedure of the in-vehicle device according to the embodiment. [Figure 7] FIG. 7 is a diagram showing an example of a method for acquiring trajectory data of an in-vehicle device according to a modified example. DETAILED DESCRIPTION OF THE INVENTION

[0011] In conventional methods, a vehicle traveling ahead is detected from an image captured by an imaging device mounted on a vehicle. When this detection result is used for vehicle control (for example, to activate the brakes when a vehicle ahead approaches), it is necessary to determine whether the vehicle ahead is traveling in the same lane as the vehicle. For example, if the captured image shows images of markings for identifying lanes, such as dividing lines or medians, around the vehicle ahead, it is possible to determine the lane in which the vehicle ahead is traveling by analyzing the captured image.

[0012] However, it is not always possible to detect markings around a vehicle ahead in a captured image. For example, this may occur when the vehicle ahead is far away from the vehicle in question, or when the lane markings are unclear (for example, when the lane markings have disappeared or become faint). In these cases, it may be impossible to detect lane markings around the vehicle ahead in a captured image to the extent that the positional relationship between the vehicle ahead and the markings can be identified.

[0013] In such cases, it has been difficult with conventional methods to accurately and efficiently determine the lane in which the vehicle ahead of the vehicle is traveling. For this reason, for example, when there is a curve in the distance, the vehicle ahead may be mistakenly recognized as traveling in the same lane as the vehicle ahead of the vehicle (hereinafter also referred to as the "vehicle ahead") because the vehicle ahead is located ahead of the vehicle in the traveling direction.

[0014] In contrast, an in-vehicle device according to a first aspect of the present disclosure acquires a captured image showing a vehicle ahead of the vehicle, determines the lane in which the vehicle ahead is traveling by analyzing the acquired captured image, and, if the lane in which the vehicle ahead is traveling cannot be determined by analyzing the captured image, calculates a trajectory of the relative position of the vehicle ahead with respect to the vehicle ahead. and determining the lane in which the forward vehicle is traveling based on the acquired trajectory data.

[0015] In a first aspect of the present disclosure, the lane in which a forward vehicle is traveling is determined by analyzing captured images. If lane determination using this image analysis is impossible, the lane in which the forward vehicle is traveling is determined using trajectory data of the forward vehicle. The trajectory data of the forward vehicle allows the relative position of the forward vehicle relative to the vehicle in question to be determined over time. That is, the lane in which the forward vehicle is traveling can be estimated by tracing back to the forward vehicle's past states. Therefore, even if the lane of the forward vehicle cannot be determined by analyzing captured images that show instantaneous states, it may be possible to determine the lane in which the forward vehicle is traveling. For example, as described above, even if there is a curve in the distance, if the trajectory data indicates that the forward vehicle is continuously traveling in an adjacent lane, it is possible to prevent the forward vehicle from being mistakenly recognized as a leading vehicle. Therefore, additional determination processing using trajectory data can improve the accuracy of determining the lane in which the forward vehicle is traveling. In addition, the additional determination processing using trajectory data is performed when lane determination using image analysis is impossible and is omitted if lane determination using image analysis is successful. This allows the target vehicles for which the additional determination process is to be executed to be limited, thereby improving the efficiency of information processing. Therefore, according to the first aspect of the present disclosure, it is possible to accurately and efficiently determine the lane in which the forward vehicle traveling ahead of the own vehicle is traveling.

[0016] An embodiment according to one aspect of the present disclosure (hereinafter also referred to as "the present embodiment") will be described below with reference to the drawings. However, the present embodiment described below is merely an example of the present disclosure in all respects. Various improvements or modifications may be made without departing from the scope of the present disclosure. In implementing the present disclosure, specific configurations according to the embodiment may be adopted as appropriate. Note that while data appearing in the present embodiment is described in natural language, more specifically, it is specified using computer-recognizable pseudo-language, commands, parameters, machine language, etc.

[0017] [1 Application example] 1 is a schematic diagram illustrating an example of a situation in which the present disclosure is applied. An in-vehicle device 1 according to this embodiment is mounted on a vehicle VA. The in-vehicle device 1 according to this embodiment is one or more computers configured to determine the lane in which a forward vehicle VF observed ahead of the vehicle VA is traveling.

[0018] Specifically, the in-vehicle device 1 acquires a captured image 20 showing a forward vehicle VF traveling ahead of the vehicle VA. The in-vehicle device 1 determines the lane in which the forward vehicle VF is traveling by analyzing the acquired captured image 20. If the lane in which the forward vehicle VF is traveling cannot be determined by analyzing the captured image 20, the in-vehicle device 1 acquires trajectory data 30 that indicates the trajectory of the relative position of the forward vehicle VF with respect to the vehicle VA. The in-vehicle device 1 determines the lane in which the forward vehicle VF is traveling based on the acquired trajectory data 30. On the other hand, if the lane determination based on the analysis of the captured image 20 is successful, the in-vehicle device 1 adopts the determination result and omits the process of determining the lane based on the trajectory data 30. The in-vehicle device 1 then outputs information related to the result of determining the lane in which the forward vehicle VF is traveling.

[0019] As described above, in this embodiment, the in-vehicle device 1 attempts to determine the lane in which the forward vehicle VF is traveling by analyzing the captured image 20. If it is not possible to determine the lane by analyzing the captured image 20, the in-vehicle device 1 additionally attempts to determine the lane in which the forward vehicle VF is traveling by using the trajectory data 30 of the forward vehicle VF. The trajectory data 30 of the forward vehicle VF makes it possible to analyze the lane in which the forward vehicle VF is traveling by going back to the past state of the forward vehicle VF, so even if it is not possible to determine the lane in which the forward vehicle VF is traveling by analyzing the captured image 20, the lane in which the forward vehicle VF is traveling can be determined. Therefore, it is possible to improve the accuracy of determining the lane in which the forward vehicle VF is traveling. In addition, the additional determination process using the trajectory data 30 is performed when lane determination by analysis of the captured image 20 is impossible, and is omitted when lane determination by analysis of the captured image 20 is successful. This makes it possible to limit the target vehicles for which the additional determination process is performed, thereby improving the efficiency of information processing. Therefore, according to this embodiment, it is possible to accurately and efficiently determine the lane in which the forward vehicle VF traveling ahead of the host vehicle VA is traveling.

[0020] (Acquisition of captured images) The captured image 20 showing the forward vehicle VF may be acquired as appropriate depending on the embodiment. In one example, acquiring the captured image 20 showing the forward vehicle VF may include acquiring a captured image and detecting the forward vehicle VF in the acquired captured image. That is, the in-vehicle device 1 may acquire, as the captured image 20, an image in which the forward vehicle VF is detected from among the acquired captured images. However, the method of acquiring the captured image 20 is not limited to this example. In another example, the detection of the forward vehicle VF may be performed by another computer. The number of vehicles shown in the captured image may be arbitrary. Multiple vehicles may be detected in the captured image. In this case, the in-vehicle device 1 may treat each vehicle detected in the captured image as a forward vehicle VF and determine the lane in which each vehicle is traveling.

[0021] (Detection of a vehicle ahead) The method for detecting a forward vehicle VF in a captured image is not particularly limited and may be selected appropriately depending on the embodiment. In one example, the forward vehicle VF may be detected by a predetermined analysis process such as pattern matching. In another example, a trained machine learning model generated by machine learning may be used for image processing to detect the forward vehicle VF. The machine learning model has one or more calculation parameters for performing calculations in an inference process. The inference process may be detecting an object such as a vehicle in a given image. The type of machine learning model is not particularly limited and may be selected appropriately depending on the embodiment. The machine learning model may be, for example, a neural network. When a neural network is used, examples of calculation parameters include the weight of connections between each node and the threshold value of each node. The machine learning model may be trained appropriately by machine learning to acquire the ability to detect an object such as a vehicle in a given image. That is, in the machine learning process, the values ​​of the calculation parameters of the machine learning model may be adjusted (optimized) appropriately to acquire the ability to perform an inference process to detect the object. Learning data used in the machine learning may be collected appropriately so that the machine learning can acquire the ability to perform the inference process for object detection.

[0022] (imaging device) In this embodiment, the vehicle VA is equipped with an imaging device C, and the captured image (captured image 20) is acquired (generated) by the imaging device C. The imaging device C may be appropriately positioned so as to capture an image of the situation ahead of the vehicle VA. Furthermore, the type of imaging device C is not particularly limited and may be appropriately selected depending on the embodiment. In one example, the imaging device C may be a monocular camera (for example, a general RGB camera). In another example, the imaging device C may be a stereo camera. Note that traveling ahead of the vehicle VA may mean being within a range that can be captured by the imaging device C.

[0023] (Lane determination method using image analysis) The method of determining the lane by analyzing the captured image 20 may be determined as appropriate depending on the embodiment. In one example, the in-vehicle device 1 may detect, in the captured image 20, the forward vehicle VF as well as markings that exist around the forward vehicle VF and are used to identify the lane. Then, the in-vehicle device 1 may detect the forward vehicle VF as seen from its own vehicle VA (i.e., as shown in the captured image 20). The lane in which the forward vehicle VF is traveling may be determined based on the positional relationship between the VF and the markings. The criteria for determining the lane based on the positional relationship may be set appropriately depending on the embodiment.

[0024] The type of marker is not particularly limited as long as it can serve as an indicator for identifying lanes, and may be selected appropriately depending on the embodiment. The marker may be, for example, a dividing line, a median strip, a side strip, etc. The dividing line may be, for example, a lane boundary line, a center line of the road, an outer edge line of the road, etc. The surroundings of the forward vehicle VF (i.e., the range in which the marker is referenced) are not particularly limited as long as it is within a range in which the lane of the forward vehicle VF can be identified by the marker, and may be selected appropriately depending on the embodiment.

[0025] Similar to the vehicle detection method, the sign detection method is not particularly limited and may be appropriately selected depending on the embodiment. In one example, the sign may be detected by a predetermined analysis process such as pattern matching. In another example, a trained machine learning model generated by machine learning may be used in the image processing for detecting the sign. When vehicles and signs are detected using a trained machine learning model, the trained machine learning model for detecting vehicles and the trained machine learning model for detecting signs may be the same or different.

[0026] The method of determining a lane by analyzing the captured image 20 need not be limited to the above example. As another example, when the image capture device C is disposed approximately at the center of the vehicle VA in the vehicle width direction and the forward vehicle VF is traveling in the same lane as the vehicle VA and is relatively close to the vehicle VA, the forward vehicle VF may appear relatively large in the captured image 20, obscuring the sign. Therefore, the in-vehicle device 1 may determine the lane in which the forward vehicle VF is traveling by evaluating whether the forward vehicle VF is in the same lane as the vehicle VA and in the vicinity thereof, depending on the size and position of the forward vehicle VF captured in the captured image 20. The criteria for determining whether the forward vehicle VF is in the same lane as the vehicle VA and in the vicinity thereof may be set as appropriate depending on the embodiment. In yet another example, a trained machine learning model generated by machine learning may be used in the determination process based on the series of image analyses.

[0027] (Cases where lane determination is not possible using image analysis) The inability to determine the lane by analyzing the captured image 20 may include various cases in which the lane in which the forward vehicle VF is traveling cannot be determined with a certain degree of accuracy using any indicator or a trained machine learning model.

[0028] In one example, the inability to determine the lane in which the forward vehicle VF is traveling through the analysis of the captured image 20 may include the inability to determine the lane in which the forward vehicle VF is traveling through because the forward vehicle VF is located far away from the vehicle's own vehicle VA and lane markings around the forward vehicle VF cannot be detected. Being located far away may mean that the forward vehicle VF is located a distance from the markings in the captured image 20 that makes it impossible to determine the lane in which the forward vehicle VF is traveling through the markings. This far distance may vary depending on, for example, the performance (resolution, etc.) of the image capture device C, the state of the markings, the image capture environment, etc. In one example, the in-vehicle device 1 may measure the distance from the vehicle's own vehicle VA to the forward vehicle VF, and determine that the forward vehicle VF is located far away based on whether the measured distance exceeds a threshold. Based on this, the in-vehicle device 1 may recognize that the lane in which the forward vehicle VF is traveling through the analysis of the captured image 20 is impossible to determine. The threshold may be determined appropriately depending on the embodiment.

[0029] Furthermore, the inability to determine the lane in which the forward vehicle VF is traveling in the analysis of the captured image 20 may include the inability to determine the lane in which the forward vehicle VF is traveling due to the lane markings around the forward vehicle VF being unclear and the markings being indistinguishable. The image of the sign may not be clearly visible within the image area 0. The degree of blurring may vary depending on, for example, the image analysis method, the performance of the imaging device C, the imaging environment, etc.

[0030] In each example, the fact that a marking object is undetectable may include not only the fact that the marking object cannot be detected at all, but also the fact that the marking object is detectable but is only detectable to the extent that the positional relationship between the forward vehicle VF and the marking object cannot be identified. In other words, the fact that a marking object is undetectable may include the fact that the marking object can be detected in the captured image 20 but is not detectable to an extent that is useful for determining the lane in which the forward vehicle VF is traveling.

[0031] (Trajectory data) The trajectory data 30 may be generated as needed to indicate the trajectory of the relative position of the forward vehicle VF with respect to the vehicle's own vehicle VA. The position trajectory indicates a time series of position changes. The relative position of the forward vehicle VF with respect to the vehicle's own vehicle VA corresponds to the distance from the vehicle's own vehicle VA to the forward vehicle VF.

[0032] In this embodiment, the trajectory data 30 may be generated from captured images obtained by the imaging device C. As an example, the trajectory data 30 of the forward vehicle VF may be derived from the results of detecting the forward vehicle VF in each of the captured image 20 and one or more captured images acquired earlier than the captured image 20, and calculating the distance from the subject vehicle VA to the detected forward vehicle VF in each of the captured images.

[0033] As a specific example, the in-vehicle device 1 may continuously acquire captured images using an imaging device C. The in-vehicle device 1 may detect a vehicle (forward vehicle VF) in each acquired captured image and calculate the distance to the detected vehicle. The distance calculation method may be appropriately selected depending on, for example, the type of imaging device C and the vehicle detection method. In one example, the imaging device C may be a monocular camera. In this case, the shorter the distance from the host vehicle VA to the forward vehicle VF, the larger the forward vehicle VF appears in the captured image 20, and the longer the distance from the host vehicle VA to the forward vehicle VF, the smaller the forward vehicle VF appears in the captured image 20. Therefore, the distance to the vehicle can be estimated based on the size of the vehicle appearing in the captured image. When a vehicle is detected using a bounding box using a trained machine learning model, the distance to the vehicle can be estimated based on the size of the bounding box of the detected vehicle. The in-vehicle device 1 may calculate the distance to the vehicle using these estimation methods. In another example, the imaging device C may be a stereo camera. In this case, the in-vehicle device 1 may calculate the distance to the vehicle based on the parallax of the captured images obtained by the stereo camera.

[0034] The in-vehicle device 1 may then generate trajectory data 30 of the forward vehicle VF by recording the distance calculation results in chronological order. The time length of the trajectory indicated by the trajectory data 30 may be set arbitrarily (for example, about several seconds). This method allows the trajectory data 30 to be obtained at low cost. The timing of generating the trajectory data 30 may be determined appropriately depending on the embodiment. In one example, the in-vehicle device 1 may calculate the distance to the vehicle from continuously acquired captured images as needed and store the distance calculation results as history, thereby generating the trajectory data 30 of the forward vehicle VF regardless of whether or not lane determination processing using the trajectory data 30 is performed. In another example, the in-vehicle device 1 may store past captured images, and when performing lane determination using the trajectory data 30, may generate the trajectory data 30 of the forward vehicle VF by tracing back from the captured image 20 to the past captured images and calculating the distance to the vehicle.

[0035] (Lane determination method using trajectory data) The trajectory data 30 allows the user to grasp the transition (time series) of the relative position of the forward vehicle VF with respect to the user's own vehicle VA. As long as it is a method for estimating the lane, the method for determining the lane of the forward vehicle VF from the trajectory data 30 is not particularly limited, and may be determined appropriately depending on the embodiment.

[0036] (1) How to use road information In one example, determining the lane in which the forward vehicle VF is traveling based on the trajectory data 30 may include: specifying the range of the lane in which the subject vehicle VA is traveling by referring to road information; and determining whether the lane in which the forward vehicle VF is traveling is the same as the lane in which the subject vehicle VA is traveling, depending on whether the trajectory of the forward vehicle VF indicated by the trajectory data 30 is within the range of the specified lane. That is, the in-vehicle device 1 may specify the range of lanes on the road around the subject vehicle VA (at least the lane in which the subject vehicle VA is traveling, if multiple lanes are provided) by referring to road information. Then, the in-vehicle device 1 may estimate the lane in which the forward vehicle VF is traveling based on the relationship between the range of the specified lane, the position of the subject vehicle VA, and the position of the forward vehicle VF.

[0037] 2 is a schematic diagram illustrating an example of a situation in which the on-board device 1 according to this embodiment employs a method of utilizing road information to determine the lane in which the forward vehicle VF is traveling. For ease of explanation, the example in FIG. 2 assumes the following conditions: The road on which the vehicle VA is traveling has two lanes (R0, R1) in the same traveling direction. The vehicle VA is traveling in lane R0, and there is a curve ahead of the vehicle VA. The forward vehicle VF is traveling in lane R1 adjacent to lane R0 (i.e., a lane different from the vehicle VA), but is approaching the curve and is therefore located in the straight-ahead direction of the vehicle VA.

[0038] First, the in-vehicle device 1 may identify the geographical position of its own vehicle VA (hereinafter also referred to as "own vehicle position"). The method for identifying the own vehicle position is not particularly limited and may be appropriately selected depending on the embodiment. As will be described later, in this embodiment, the in-vehicle device 1 may identify the position of its own vehicle VA using a GPS (Global Positioning System) measuring device. Next, the on-vehicle device 1 may refer to the road information 55 based on the vehicle position to identify the range of at least the lane in which the vehicle VA is traveling among the lanes on the road around the vehicle position. In the example of FIG. 2, the on-vehicle device 1 may identify the range of at least the lane R0 of the two lanes (R0, R1). The range around the vehicle position may be determined appropriately to an extent that the lane of the forward vehicle VF can be determined. The road information 55 may be configured appropriately to indicate attributes of the road, including the geographic range of the lane. The geographic range of the lane may be expressed in any format. The geographic range of the lane may be identified according to, for example, the width of the lane, the position of the lane center, etc. A road includes one or more lanes. The number of lanes may be arbitrary.

[0039] The in-vehicle device 1 may then estimate the lane in which the forward vehicle VF is traveling based on the relationship between the range of the identified lane, the position of the vehicle's own vehicle VA, and the position of the forward vehicle VF. That is, the in-vehicle device 1 may determine whether the lane in which the forward vehicle VF is traveling is the same as the lane in which the vehicle's own vehicle VA is traveling, depending on whether the trajectory of the forward vehicle VF indicated by the trajectory data 30 is within the range of the identified lane. In the example of Fig. 2, the in-vehicle device 1 may determine whether the lane in which the forward vehicle VF is traveling is the same as the lane in which the vehicle's own vehicle VA is traveling, depending on whether the trajectory of the forward vehicle VF indicated by the trajectory data 30 is within the range of lane R0.

[0040] When the trajectory of the forward vehicle VF is within the range of lane R0, the in-vehicle device 1 may determine that the lane in which the forward vehicle VF is traveling is the same as the lane in which the vehicle VA is traveling (i.e., the lane in which the forward vehicle VF is traveling is lane R0). On the other hand, when the trajectory of the forward vehicle VF is not within the range of lane R0, the in-vehicle device 1 may determine that the lane in which the forward vehicle VF is traveling is not the same as the lane in which the vehicle VA is traveling (i.e., the lane in which the forward vehicle VF is traveling is a lane other than lane R0 (lane R1 in the example of FIG. 2)). In the scene illustrated in FIG. 2, this determination process allows the in-vehicle device 1 to recognize that the lane in which the forward vehicle VF is traveling is lane R1. It is possible.

[0041] According to this determination method, the lane in which the forward vehicle VF is traveling can be appropriately determined. Note that, when the range of lanes other than the lane in which the host vehicle VA is traveling is also identified by referring to road information, determining that the trajectory of the forward vehicle VF is not within the range of the lane in which the host vehicle VA is traveling may be configured by determining that the trajectory of the forward vehicle VF is within the range of a lane other than the lane in which the host vehicle VA is traveling. In the example of FIG. 2, when the range of lane R1 is also identified by referring to road information, determining that the trajectory of the forward vehicle VF is not within the range of lane R0 may be configured by determining that the trajectory of the forward vehicle VF is within the range of lane R1.

[0042] (2) Using the trajectory data of surrounding vehicles Furthermore, when the lane in which the forward vehicle VF is traveling cannot be determined by analyzing the captured image 20 and there is a peripheral vehicle other than the forward vehicle VF traveling in front of the vehicle VA and the lane in which the peripheral vehicle is traveling is identified, the in-vehicle device 1 may be configured to further acquire trajectory data indicating the trajectory of the relative position of the peripheral vehicle with respect to the vehicle VA. In this case, determining the lane in which the forward vehicle VF is traveling based on the trajectory data 30 may include determining the lane in which the forward vehicle VF is traveling based on the result of comparing the trajectory data 30 of the forward vehicle VF with the trajectory data of the peripheral vehicles.

[0043] 3 shows an example of a situation in which the on-board device 1 according to this embodiment determines the lane in which the forward vehicle VF is traveling by employing a method that uses the trajectory data of the peripheral vehicles. For ease of explanation, the example shown in FIG. 3 assumes the following conditions in addition to those shown in FIG. 2: A peripheral vehicle VZ is present in front of the vehicle VA and behind the forward vehicle VF. The peripheral vehicle VZ, like the forward vehicle VF, is traveling in a lane R1 adjacent to the lane R0.

[0044] The forward vehicle VF is a vehicle other than the subject vehicle VA, and is a vehicle for which lane determination is to be performed. On the other hand, the peripheral vehicle VZ is a vehicle other than the subject vehicle VA, and is a vehicle for which the lane in which it is traveling has already been determined (specified). In one example, the lane in which the peripheral vehicle VZ is traveling may be determined by the method disclosed herein before the determination of the forward vehicle VF. That is, the lane in which the peripheral vehicle VZ is traveling may also be determined as a forward vehicle. In another example, the on-board device 1 may be configured to be able to determine the lane in which a vehicle is traveling by other methods in addition to the method disclosed herein. In this case, the lane in which the peripheral vehicle VZ is traveling may be determined by other methods. Note that typically, the peripheral vehicle VZ may be a vehicle traveling in a different lane (i.e., a lane other than the lane in which the subject vehicle VA is traveling among multiple lanes in the same traveling direction provided on a road) in the same traveling direction as the lane in which the subject vehicle VA is traveling.

[0045] When this method is adopted, the in-vehicle device 1 may acquire trajectory data 35 of the surrounding vehicle VZ in addition to the trajectory data 30 of the forward vehicle VF. The trajectory data 30 of the forward vehicle VF may be referred to as "first trajectory data," and the trajectory data 35 of the surrounding vehicle VZ may be referred to as "second trajectory data." The trajectory data 35 is configured to indicate the trajectory of the relative position of the surrounding vehicle VZ with respect to the vehicle VA. The trajectory data 35 of the surrounding vehicle VZ may be generated in the same manner as the trajectory data 30 of the forward vehicle VF. Next, the in-vehicle device 1 may compare the trajectory data 30 of the forward vehicle VF with the trajectory data 35 of the surrounding vehicle VZ. Then, the in-vehicle device 1 may determine the lane in which the forward vehicle VF is traveling based on the result of the comparison.

[0046] In comparing the trajectory data (30, 35), the in-vehicle device 1 may determine whether the trajectory of the forward vehicle VF indicated by the trajectory data 30 and the trajectory of the peripheral vehicle VZ indicated by the trajectory data 35 are similar to each other to the extent that it can be evaluated that the forward vehicle VF and the peripheral vehicle VZ are traveling in the same lane. In other words, in comparing the trajectory data (30, 35), the in-vehicle device 1 may determine whether the forward vehicle VF is also traveling (or has traveled) on the trajectory of the peripheral vehicle VZ. The evaluation index may be set as appropriate. Whether the trajectories are similar or not may be evaluated based on, for example, an index such as whether the trajectories at least partially match or are close within the range of the lane width.

[0047] When the trajectory of the forward vehicle VF and the trajectory of the peripheral vehicle VZ are similar, the in-vehicle device 1 can determine that the forward vehicle VF is traveling in the same lane as the peripheral vehicle VZ. On the other hand, when the trajectory of the forward vehicle VF and the trajectory of the peripheral vehicle VZ are not similar, the in-vehicle device 1 can determine that the forward vehicle VF is traveling in a different lane from the peripheral vehicle VZ. In the example of Fig. 3, the in-vehicle device 1 can determine that the forward vehicle VF is traveling in the same lane R1 as the peripheral vehicle VZ because the trajectory of the forward vehicle VF and the trajectory of the peripheral vehicle VZ are similar as a result of comparing the trajectory data (30, 35).

[0048] According to this determination method, similar to the method of FIG. 2, the lane in which the forward vehicle VF is traveling can be appropriately determined. Note that the number of peripheral vehicles VZ is not limited to the example of FIG. 3, and there may be two or more peripheral vehicles VZ. Furthermore, the position of the peripheral vehicles VZ is not limited to behind the forward vehicle VF. The peripheral vehicles VZ may be located ahead of the forward vehicle VF. The peripheral vehicles VZ may be traveling in a different lane from the forward vehicle VF.

[0049] (3) Other In this embodiment, the in-vehicle device 1 may be configured to determine the lane in which the forward vehicle VF is traveling by employing at least one of the method illustrated in FIG. 2 and the method illustrated in FIG. 3. Note that the method of determining the lane using the trajectory data 30 is not limited to these examples. As another example, a determination model for determining the lane in which the forward vehicle VF is traveling may be prepared as appropriate. The determination model may be configured using a trained machine learning model generated by machine learning. The in-vehicle device 1 may use this determination model to determine the lane in which the forward vehicle VF is traveling from the trajectory data 30.

[0050] 2 and 3, if the lane of the forward vehicle VF is determined using a conventional method, the forward vehicle VF located in the straight direction of the forward vehicle VA may be erroneously recognized as a preceding vehicle because the forward vehicle VF is located far away from the vehicle VA and the lane boundary between the two lanes (R0, R1) cannot be detected. This may result in the forward vehicle VF being erroneously recognized as traveling in lane R0. In contrast, according to the lane determination method of this embodiment, as shown in FIGS. 2 and 3, the forward vehicle VF is estimated to be continuously traveling in lane R1 from the trajectory data 30, thereby preventing the forward vehicle VF from being erroneously recognized as a preceding vehicle. Therefore, the additional determination process using the trajectory data 30 can improve the accuracy of determining the lane in which the forward vehicle VF is traveling.

[0051] (output of information) The content of the process for outputting information relating to the lane determination result of the forward vehicle VF may be determined appropriately depending on the embodiment. In one example, the in-vehicle device 1 may simply output the result of determining the lane in which the forward vehicle VF is traveling. In another example, if the lane determination result indicates that the forward vehicle VF is a preceding vehicle traveling ahead in the same lane as the vehicle VA, the in-vehicle device 1 may turn on a flag indicating the presence of a preceding vehicle as part of the information output process. This flag may be used for operational control of the vehicle VA (for example, speed limiting, braking, etc.).

[0052] [2 Configuration Examples] [Hardware configuration example] FIG. 4 is a diagram illustrating an example of the hardware configuration of the in-vehicle device 1 according to this embodiment. As shown in FIG. 1, the in-vehicle device 1 according to this embodiment is a computer to which a control unit 11, a memory unit 12, a communication interface 13, an external interface 14, a GPS measuring device 15, an input device 16, an output device 17, and a drive 18 are electrically connected.

[0053] The control unit 11 includes a CPU (Central Processing Unit) which is a hardware processor, The device includes RAM (Random Access Memory), ROM (Read Only Memory), etc., and is configured to execute information processing based on programs and various data. The control unit 11 (CPU) is an example of a processor resource. The storage unit 12 is configured, for example, with a hard disk drive, a solid state drive, etc. The storage unit 12 is an example of a memory resource. In this embodiment, the storage unit 12 stores various information such as a program 81 and map information 50.

[0054] The program 81 is a program for causing the in-vehicle device 1 to execute information processing (see FIG. 6 described below) for determining the lane in which the forward vehicle VF is traveling. The program 81 includes a series of instructions for this information processing. The map information 50 includes road information 55 and is appropriately configured to indicate the locations of roads that the vehicle can travel on. As long as information about lanes around the vehicle position can be extracted by a search using the vehicle position as a query, the data formats of the map information 50 and the road information 55 are not particularly limited and may be appropriately determined depending on the embodiment. The road information 55 may be stored separately from the map information 50.

[0055] The communication interface 13 is, for example, a wired LAN (Local Area Network) module, a wireless LAN module, etc., and is configured to perform wired or wireless communication via a network. The in-vehicle device 1 may use the communication interface 13 to perform data communication with other computers.

[0056] The external interface 14 is, for example, a USB (Universal Serial Bus) port, a dedicated port, or the like, and is an interface for connecting to an external device. The type and number of the external interfaces 14 may be determined appropriately depending on the type and number of the external devices to be connected. In this embodiment, the in-car device 1 may be connected to the imaging device C via the external interface 14.

[0057] The GPS measurement device 15 is configured to measure its own position by receiving signals from GPS satellites. In this embodiment, the in-vehicle device 1 may use the GPS measurement device 15 to identify the position of its own vehicle VA.

[0058] The input device 16 is a device for inputting, for example, an operation button, etc. The output device 17 is a device for outputting, for example, a display, a speaker, etc. A user can operate the in-vehicle device 1 by using the input device 16 and the output device 17. The input device 16 and the output device 17 may be integrally configured, for example, by a touch panel display, etc.

[0059] The drive 18 is a device for reading various information such as programs stored in a storage medium 91. At least one of the program 81 and the map information 50 may be stored in the storage medium 91. Accordingly, the in-vehicle device 1 may acquire at least one of the program 81 and the map information 50 from the storage medium 91. The storage medium 91 is a medium that stores information such as programs by electrical, magnetic, optical, mechanical, or chemical action so that a computer or other device, machine, or the like can read the various information such as the stored programs.

[0060] In FIG. 4, an example of the storage medium 91 is a disk-type storage medium such as a CD or a DVD. However, the type of storage medium 91 is not limited to a disk type, and may be other than a disk type. Examples of storage media other than a disk type include semiconductor memories such as flash memories. The type of drive 18 may be selected appropriately depending on the type of storage medium 91.

[0061] Regarding the specific hardware configuration of the in-vehicle device 1, components can be omitted, replaced, or added as appropriate depending on the embodiment. For example, the control unit 11 may include multiple hardware processors. The hardware processors may be configured with a microprocessor, an FPGA (field-programmable gate array), a GPU (Graphics Processing Unit), etc. At least one of the communication interface 13, the external interface 14, the GPS measuring device 15, the input device 16, the output device 17, and the drive 18 may be omitted. The in-vehicle device 1 may be configured with multiple computers. In this case, the hardware configurations of the computers may or may not be the same. The in-vehicle device 1 may be any computer that is at least temporarily installed in the vehicle VA and performs information processing. The in-vehicle device 1 may be a computer designed specifically for the service provided, a general-purpose computer, a mobile phone including a smartphone, a tablet PC (Personal Computer), etc.

[0062] [Software configuration example] 5 schematically shows an example of the software configuration of the in-vehicle device 1 according to this embodiment. The control unit 11 of the in-vehicle device 1 loads the program 81 stored in the storage unit 12 into the RAM, and executes the instructions included in the program 81 loaded into the RAM using the CPU. As a result, the in-vehicle device 1 according to this embodiment operates as a computer including an image acquisition unit 111, a detection unit 112, a first determination unit 113, a trajectory acquisition unit 114, a second determination unit 115, and an output unit 116 as software modules. That is, in this embodiment, each software module of the in-vehicle device 1 is realized by the control unit 11 (CPU).

[0063] The image acquisition unit 111 is configured to acquire a captured image. The detection unit 112 is configured to detect a vehicle in the acquired captured image. In this embodiment, a captured image 20 showing the forward vehicle VF is acquired by the operations of the image acquisition unit 111 and the detection unit 112. The first determination unit 113 is configured to determine the lane in which the forward vehicle VF is traveling by analyzing the acquired captured image 20. The trajectory acquisition unit 114 is configured to acquire trajectory data 30 indicating the trajectory of the relative position of the forward vehicle VF with respect to the vehicle VA when the lane in which the forward vehicle VF is traveling cannot be determined by analyzing the captured image 20. The second determination unit 115 is configured to determine the lane in which the forward vehicle VF is traveling based on the acquired trajectory data 30. The output unit 116 is configured to output information related to the result of determining the lane in which the forward vehicle VF is traveling.

[0064] In this embodiment, an example is described in which each software module of the in-vehicle device 1 is implemented by a general-purpose CPU. However, some or all of the software modules may be implemented by one or more dedicated processors. Each module may be implemented as a hardware module. Furthermore, with regard to the software configuration of the in-vehicle device 1, modules may be omitted, replaced, or added as appropriate depending on the embodiment.

[0065] [3 Example of operation] FIG. 6 is a flowchart showing an example of the processing procedure of the in-vehicle device 1 according to this embodiment. The processing procedure described below is an example of an information processing method according to one aspect of the present disclosure. However, the processing procedure described below is merely an example, and each step may be changed as much as possible. Furthermore, steps may be omitted, replaced, or added as appropriate in the following processing procedure depending on the embodiment. Addition may be made.

[0066] (Step S101) In step S101, the control unit 11 operates as the image acquisition unit 111 and acquires the captured image generated by the imaging device C.

[0067] In this embodiment, the control unit 11 acquires a captured image from an imaging device C mounted on the vehicle VA. In one example, the control unit 11 may acquire the captured image directly from the imaging device C. In another example, the control unit 11 may acquire the captured image from the imaging device C via, for example, another computer. After acquiring the captured image, the control unit 11 proceeds to the next step S102.

[0068] (Steps S102 and S103) In step S102, the control unit 11 operates as the detection unit 112 and detects a vehicle (forward vehicle VF) in the acquired captured image.

[0069] The method for detecting a vehicle may be selected as appropriate depending on the embodiment. In one example, the control unit 11 may detect the forward vehicle VF in the captured image by performing a predetermined analysis process such as pattern matching. In another example, the control unit 11 may detect the forward vehicle VF in the captured image by using a trained machine learning model. In this case, the in-vehicle device 1 may include the trained machine learning model by storing learning result data. The learning result data may be configured as appropriate to enable reproduction of the trained machine learning model. The learning result data may be stored in any storage area, such as the storage unit 12, the storage medium 91, an external storage device, or another computer, and the control unit 11 may appropriately acquire the learning result data from any storage area.

[0070] In step S103, the control unit 11 determines where the process should branch depending on the detection result of step S102. If the forward vehicle VF is not detected in the captured image by the process of step S102, the control unit 11 returns the process to step S101 and repeats the processes of steps S101 and S102. On the other hand, if the forward vehicle VF is detected in the captured image by the process of step S102, the control unit 11 proceeds to the next step S104. In this embodiment, the forward vehicle VF is detected by the processes of steps S101 and S102, and a captured image 20 showing the forward vehicle VF is obtained. Note that if multiple vehicles are detected in the process of step S102, the processes from step S104 onwards may be executed for each detected vehicle.

[0071] (Steps S104 and S105) In step S104, the control unit 11 operates as the first determination unit 113, and determines the lane in which the forward vehicle VF is traveling by analyzing the acquired captured image 20.

[0072] The method of determining the lane using image analysis may be determined appropriately depending on the embodiment. In one example, the control unit 11 may detect the forward vehicle VF as well as signs present around the forward vehicle VF in the captured image 20. The control unit 11 may then determine the lane in which the forward vehicle VF is traveling based on the positional relationship between the forward vehicle VF and the signs as seen from the subject vehicle VA. In another example, the control unit 11 may determine the lane in which the forward vehicle VF is traveling by evaluating whether the forward vehicle VF is in the same lane as the subject vehicle VA and in the vicinity thereof based on the size and position of the forward vehicle VF captured in the captured image 20.

[0073] In step S105, the control unit 11 determines where the process should branch depending on whether or not the lane in which the forward vehicle VF is traveling has been determined by the image analysis in step S104. In the process of step S4, if the lane of the forward vehicle VF cannot be determined, the control unit 11 proceeds to the next step S106. On the other hand, if the lane of the forward vehicle VF can be determined by the process of step S104, the control unit 11 skips the processes of steps S106 and S107 and proceeds to step S108.

[0074] In this embodiment, the inability to determine the lane of the forward vehicle VF may include the inability to determine the lane in which the forward vehicle VF is traveling because the forward vehicle VF is located far away from the vehicle VA and lane markings around the forward vehicle VF cannot be detected. Also, the inability to determine the lane in which the forward vehicle VF is traveling may include the inability to determine the lane in which the forward vehicle VF is traveling because lane markings around the forward vehicle VF are unclear and cannot be detected.

[0075] (Step S106) In step S106, the control unit 11 operates as the trajectory acquisition unit 114 and acquires the trajectory data 30 indicating the trajectory of the relative position of the forward vehicle VF with respect to the vehicle VA.

[0076] In this embodiment, the control unit 11 may detect the forward vehicle VF in each of the captured image 20 and one or more captured images acquired earlier than the captured image 20. The control unit 11 may then calculate the distance to the detected forward vehicle VF in each of the captured image 20 and the previous captured image, and record the distance calculation results in chronological order to generate trajectory data 30 of the forward vehicle VF. In one example, the process of generating the trajectory data 30 may be executed in parallel with the processes of steps S101 and S102. In another example, the process of generating the trajectory data 30 may be executed in step S106.

[0077] If the amount of trajectory data 30 acquired is not sufficient to determine the lane of the forward vehicle VF, the control unit 11 may continue observing the forward vehicle VF without proceeding to the next step S107 (for example, the processing of steps S101 and S102 may be repeated to acquire a time series of multiple captured images showing the forward vehicle VF). In this way, the control unit 11 may acquire the trajectory data 30. Once the trajectory data 30 has been acquired, the control unit 11 proceeds to the next step S107.

[0078] (Step S107) In step S107, the control unit 11 operates as the second determination unit 115 and determines the lane in which the forward vehicle VF is traveling based on the acquired trajectory data 30.

[0079] The method for determining the lane of the forward vehicle VF based on the trajectory data 30 is not particularly limited and may be determined appropriately depending on the embodiment. In the present embodiment, the control unit 11 may determine the lane in which the forward vehicle VF is traveling based on the trajectory data 30 by employing at least one of the methods exemplified in Figs. 2 and 3 above.

[0080] 2 is employed, the control unit 11 may identify the vehicle position using the GPS measuring device 15. Next, the control unit 11 may identify at least the range of lanes in which the vehicle VA is traveling among lanes on roads around the vehicle position by referring to road information 55 included in map information 50 using the identified vehicle position as a reference. Then, the control unit 11 may determine whether the lane in which the vehicle VF is traveling is the same as the lane in which the vehicle VA is traveling, depending on whether the trajectory of the vehicle VF indicated by the trajectory data 30 is within the range of the identified lanes.

[0081] When the method illustrated in FIG. 3 is adopted, the control unit 11 may acquire the trajectory data 35 of the surrounding vehicle VZ in addition to the trajectory data 30 of the forward vehicle VF. The lane in which the surrounding vehicle VZ is traveling may be identified by already performing the processing of steps S101 to S107 (the processing of step S104 or step S107) on the surrounding vehicle VZ. Alternatively, the lane in which the surrounding vehicle VZ is traveling may be identified by another method. Next, the control unit 11 may compare the trajectory data 30 of the forward vehicle VF with the trajectory data 35 of the surrounding vehicle VZ. Then, the control unit 11 may determine the lane in which the forward vehicle VF is traveling based on the result of the comparison. After determining the lane in which the forward vehicle VF is traveling, the control unit 11 proceeds to the next step S108.

[0082] (Step S108) In step S108, the control unit 11 operates as the output unit 116 and outputs information relating to the result of determining the lane in which the forward vehicle VF is traveling.

[0083] The output destination and the content of the output process may be determined appropriately depending on the embodiment. The output destination may be selected from, for example, RAM, the memory unit 12, the output device 17, the storage medium 91, an external storage device, another computer, etc. In one example, the control unit 11 may output the result of determining the lane in which the forward vehicle VF is traveling to the output device 17 (e.g., a display) together with the captured image 20. In another example, when the determination result in step S104 or step S107 determines that the forward vehicle VF is a preceding vehicle traveling ahead in the same lane as the host vehicle VA, the control unit 11 may turn on a flag indicating the presence of a preceding vehicle as part of the information output process. If the in-vehicle device 1 is configured to be able to control the operation of the vehicle VA, the control unit 11 may execute operational control of the vehicle VA (e.g., speed limit, brake application, etc.) in response to the flag being turned on. Alternatively, if the operation of the vehicle VA is controlled by another computer, the control unit 11 may notify the other computer that the flag has been turned on and cause the other computer to execute operational control of the vehicle VA in accordance with the flag.

[0084] When the output of the information is completed, the control unit 11 ends the processing procedure of the in-vehicle device 1 according to this operation example. The control unit 11 may repeatedly execute the processing of steps S101 to S108. The timing at which the processing is repeated may be determined appropriately depending on the embodiment. In this way, the in-vehicle device 1 may be configured to continuously monitor the lane in which the forward vehicle VF is traveling.

[0085] (Features) In this embodiment, in step S104, the in-vehicle device 1 attempts to determine the lane in which the forward vehicle VF is traveling by analyzing the captured image 20. If lane determination is not possible through the analysis of the captured image 20, in step S107, the in-vehicle device 1 additionally attempts to determine the lane in which the forward vehicle VF is traveling using the trajectory data 30 of the forward vehicle VF. Even if the lane of the forward vehicle VF cannot be determined through the processing of step S104, the trajectory data 30 of the forward vehicle VF may be able to determine the lane in which the forward vehicle VF is traveling. This improves the accuracy of determining the lane in which the forward vehicle VF is traveling. In addition, the additional determination processing in step S107 is performed when lane determination is not possible through the processing of step S104, and is omitted by branching to step S105 if lane determination is successful through the processing of step S104. This limits the target vehicles for which the additional determination processing in S107 is performed, thereby improving the efficiency of information processing. Therefore, according to this embodiment, it is possible to accurately and efficiently determine the lane in which the forward vehicle VF traveling ahead of the host vehicle VA is traveling.

[0086] [4 Variations] Although the embodiments of the present disclosure have been described in detail above, the above description is merely an example of the present disclosure in every respect. Various improvements or modifications may be made without departing from the scope of the present disclosure. For example, the following modifications are possible: The following modifications can be combined as appropriate.

[0087] <4.1> In the above embodiment, the trajectory data (30, 35) may be generated from captured images obtained by the imaging device C. However, the method for generating the trajectory data (30, 35) is not limited to this example and may be determined appropriately depending on the embodiment. In another example, a distance sensor may be used to generate the trajectory data (30, 35).

[0088] FIG. 7 schematically shows an example of a method for acquiring trajectory data (30, 35) of the in-vehicle device 1 according to this modification. As shown in FIG. 7, the in-vehicle device 1 may further be equipped with a distance sensor S in addition to the imaging device C. The distance sensor S may be appropriately disposed so as to be able to measure the distance to vehicles (forward vehicle VF, surrounding vehicles VZ) present ahead of the vehicle VA. The type of the distance sensor S is not particularly limited as long as it is able to measure the distance, and may be appropriately selected depending on the embodiment. The distance sensor S may be, for example, a LiDAR (Light Detection and Ranging, Laser Imaging Detection and Ranging), a millimeter-wave radar, an infrared sensor, an ultrasonic sensor, or the like. The in-vehicle device 1 may be connected to the distance sensor S via an external interface 14.

[0089] In this modification, the control unit 11 may generate trajectory data 30 of the forward vehicle VF by continuously measuring the distance from the host vehicle VA to the forward vehicle VF using the distance sensor S. The control unit 11 may also generate trajectory data 35 of the peripheral vehicle VZ by continuously measuring the distance from the host vehicle VA to the peripheral vehicle VZ using the distance sensor S. The generation of the trajectory data (30, 35) may be executed in parallel with the above-described information processing. In another example, the generation of the trajectory data (30, 35) by the distance sensor S may be executed by another computer. In this case, the control unit 11 may acquire the trajectory data (30, 35) from the other computer. This method makes it possible to obtain trajectory data (30, 35) that relatively accurately indicate the relative positions of the vehicles (VF, VZ). As a result, improved accuracy in determining the lane in which the forward vehicle VF is traveling can be expected.

[0090] <4.2> In the above embodiment, the in-vehicle device 1 is configured to be able to identify the geographical position of its own vehicle VA by including the GPS measuring device 15. However, the method for identifying the position of the vehicle VA is not limited to this example. In another example, the in-vehicle device 1 may be configured to identify its own vehicle position by a method other than the method using the GPS measuring device 15. In this case, the GPS measuring device 15 may be omitted from the hardware configuration of the in-vehicle device 1.

[0091] In the above embodiment, the map information 50 and the road information 55 are stored in the storage unit 12. However, the storage location of the map information 50 and the road information 55 is not limited to this example. The map information 50 and the road information 55 may be stored in an external storage device (for example, a network server). In this case, the in-vehicle device 1 may access the map information 50 and the road information 55 via a network, for example, to acquire information about lanes around the vehicle position.

[0092] In the above embodiment, the type of each vehicle (VA, VF, VZ) is not particularly limited and may be selected arbitrarily. Typically, each vehicle (VA, VF, VZ) may be an automobile.

[0093] [5 Supplementary Information] The processes and means described in this disclosure may be freely used as long as no technical contradiction occurs. It can be implemented in combination with

[0094] Furthermore, a process described as being performed by one device may be shared and executed by multiple devices. Alternatively, a process described as being performed by different devices may be executed by a single device. In a computer system, the hardware configuration that realizes each function can be flexibly changed.

[0095] The present disclosure can also be realized by providing a computer program implementing the functions described in the above embodiments to a computer, and having one or more processors in the computer read and execute the program. Such a computer program may be provided to the computer via a non-transitory computer-readable storage medium connectable to the computer's system bus or via a network. Non-transitory computer-readable storage media include, for example, any type of disk, such as a magnetic disk (e.g., a floppy disk, a hard disk drive (HDD), etc.), an optical disk (e.g., a CD-ROM, a DVD disk, a Blu-ray disk), a read-only memory (ROM), a random-access memory (RAM), an EPROM, an EEPROM, a magnetic card, a flash memory, an optical card, or any type of medium suitable for storing electronic instructions. [Explanation of symbols]

[0096] 1...In-vehicle device, VA...(own) vehicle, 11...control unit, 12...storage unit, 13...communication interface, 14...External interface, 15...GPS measuring device, 16...input device, 17...output device, 18...drive, 81...program, 91...storage medium, 111...image acquisition unit, 112...detection unit, 113...first determination unit, 114...trajectory acquisition unit, 115... second determination unit, 116... output unit, 20...Captured image, 30...(forward vehicle) trajectory data, 35...Trajectory data (of surrounding vehicles) 50...Map information, 55...Road information, VF...front vehicle, VZ...surrounding vehicles, R0·R1...lanes

Claims

1. Acquiring a captured image showing a vehicle ahead of the vehicle; Determining a lane in which the forward vehicle is traveling by analyzing the acquired captured image; When the lane in which the forward vehicle is traveling cannot be determined by analyzing the captured image, acquiring trajectory data indicating a trajectory of a relative position of the forward vehicle with respect to the vehicle; and and determining a lane in which the forward vehicle is traveling based on the acquired trajectory data. An in-vehicle device, The control unit is further configured to, when the lane in which the forward vehicle is traveling cannot be determined by analysis of the captured image and there is a peripheral vehicle other than the forward vehicle traveling in front of the vehicle, the peripheral vehicle having a specified lane in which the forward vehicle is traveling, acquire trajectory data indicating a trajectory of a relative position of the peripheral vehicle with respect to the vehicle; determining a lane in which the forward vehicle is traveling based on the trajectory data includes determining a lane in which the forward vehicle is traveling based on a result of a comparison between the trajectory data of the forward vehicle and the trajectory data of the surrounding vehicles. In-vehicle device.

2. Determining a lane in which the forward vehicle is traveling based on the trajectory data By referring to road information, the range of the lane in which the vehicle is traveling is specified; and determining whether the lane in which the forward vehicle is traveling is the same as the lane in which the vehicle itself is traveling, depending on whether the trajectory of the forward vehicle indicated by the trajectory data is within the range of the identified lane; Including, The in-vehicle device according to claim 1 .

3. The trajectory data of the forward vehicle is derived from a result of detecting the forward vehicle in each of the captured image and one or more captured images acquired before the captured image, and calculating a distance to the detected forward vehicle. The in-vehicle device according to claim 1 or 2.

4. The trajectory data of the forward vehicle is generated by continuously measuring the distance to the forward vehicle using a distance sensor. The in-vehicle device according to claim 1 or 2.

5. The inability to determine the lane in which the forward vehicle is traveling by analyzing the captured image includes the inability to determine the lane in which the forward vehicle is traveling because the forward vehicle is far away from the vehicle itself and lane markings existing around the forward vehicle cannot be detected. The vehicle-mounted device according to claim 1 .

6. The inability to determine the lane in which the forward vehicle is traveling by analyzing the captured image includes the inability to determine the lane in which the forward vehicle is traveling because lane markings present around the forward vehicle are unclear and the markings cannot be detected, The vehicle-mounted device according to claim 1 .

7. 1. A computer-implemented information processing method, comprising: Acquiring a captured image showing a vehicle ahead of the vehicle; Determining a lane in which the forward vehicle is traveling by analyzing the acquired captured image; When the lane in which the forward vehicle is traveling cannot be determined by analyzing the captured image, acquiring trajectory data indicating a trajectory of a relative position of the forward vehicle with respect to the vehicle; and determining a lane in which the forward vehicle is traveling based on the acquired trajectory data; The computer further executes, when the lane in which the forward vehicle is traveling cannot be determined by the analysis of the captured image and there is a surrounding vehicle other than the forward vehicle traveling in front of the host vehicle and the lane in which the surrounding vehicle is traveling is identified, acquiring trajectory data indicating a trajectory of a relative position of the surrounding vehicle with respect to the host vehicle; determining a lane in which the forward vehicle is traveling based on the trajectory data includes determining a lane in which the forward vehicle is traveling based on a result of a comparison between the trajectory data of the forward vehicle and the trajectory data of the surrounding vehicles. Information processing methods.

8. A program for causing a computer to execute an information processing method, The information processing method includes: Acquiring a captured image showing a vehicle ahead of the vehicle; Determining a lane in which the forward vehicle is traveling by analyzing the acquired captured image; When the lane in which the forward vehicle is traveling cannot be determined by analyzing the captured image, acquiring trajectory data indicating a trajectory of a relative position of the forward vehicle with respect to the vehicle; and determining a lane in which the forward vehicle is traveling based on the acquired trajectory data; Including, The computer is further caused to acquire trajectory data indicating a trajectory of a relative position of the surrounding vehicle with respect to the vehicle when the lane in which the preceding vehicle is traveling cannot be determined by the analysis of the captured image and there is a surrounding vehicle other than the preceding vehicle traveling in front of the vehicle and the lane in which the surrounding vehicle is traveling is identified, Determining the lane in which the forward vehicle is traveling based on the trajectory data includes determining the lane in which the forward vehicle is traveling based on a result of comparing the trajectory data of the forward vehicle and the trajectory data of the surrounding vehicles. determining which lane to travel in; program.

Images