Information processing apparatus, information processing method, and information processing system

Abstract

To prevent a vehicle which is not a stolen vehicle from being erroneously recognized as a stolen vehicle.SOLUTION: An acquisition unit 232 that acquires, from a plurality of imaging vehicles, feature information indicating a feature of a target vehicle included in a captured image obtained by imaging surroundings of each of the plurality of imaging vehicles, reliability information indicating reliability of the feature information, and position information indicating a position at which the captured image is captured, a storage unit 22 that stores a feature of a registered vehicle, a determination unit 233 that determines whether the target vehicle is a similar vehicle based on the feature of the registered vehicle and the feature of the target vehicle indicated by the feature information, and an output unit 235 that outputs a route along which the similar vehicle has moved based on a plurality of positions at which a plurality of captured images including the similar vehicle are captured and reliability of the feature of the similar vehicle.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to an information processing device, an information processing method, and an information processing system. [Background technology]

[0002] A technology has been proposed for tracking a stolen vehicle using images captured by an imaging device mounted on a moving vehicle. For example, Patent Document 1 describes tracking a stolen vehicle by receiving images of the stolen vehicle captured at different locations by multiple vehicles and location information indicating the locations corresponding to these images, using a server. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2019-79330 Summary of the Invention [Problem to be solved by the invention]

[0004] If the imaging environment of the imaging device mounted on the vehicle is not good, it is not possible to correctly determine whether the vehicle shown in the image is a stolen vehicle. For this reason, the invention described in Patent Document 1 has the problem of falsely identifying a vehicle that is not stolen as a stolen vehicle.

[0005] The present invention has been made in consideration of this point, and aims to provide an information processing device, an information processing method, and an information processing system that can prevent a vehicle that is not stolen from being mistakenly recognized as a stolen vehicle. [Means for solving the problem]

[0006] An information processing device of a first aspect of the present invention includes an acquisition unit that acquires, from a plurality of image-capturing vehicles, feature information indicating the features of a target vehicle included in captured images of the surroundings of each of the plurality of image-capturing vehicles, accuracy information indicating the accuracy of the feature information, and location information indicating the location where the captured images were captured; a memory unit that stores the features of registered vehicles; a determination unit that determines whether the target vehicle is a similar vehicle based on the features of the registered vehicle and the features of the target vehicle indicated by the feature information; and an output unit that outputs the route traveled by the similar vehicle based on a plurality of locations where a plurality of image-capturing images including the similar vehicle were captured and the accuracy of the features of the similar vehicle.

[0007] The output unit may output a trajectory of the similar vehicle along the route based on times at which the plurality of captured images were captured.

[0008] The output unit may output the route that does not include the position where the second captured image was captured when the distance from the position where the first captured image including the similar vehicle was captured to the position where the second captured image that was captured after the first captured image and includes the similar vehicle was captured is greater than a predetermined value.

[0009] The output unit may output the route that passes through positions where one or more captured images having the accuracy equal to or greater than a predetermined threshold value, with priority over the route that passes through positions where one or more captured images having the accuracy lower than the threshold value.When the output unit is unable to output a first accuracy route along which the similar vehicle has traveled as the route based on the positions of the similar vehicles acquired by the acquisition unit in association with the plurality of pieces of accuracy information indicating an accuracy equal to or greater than a predetermined threshold value, the output unit may output a second accuracy route along which the similar vehicle has traveled as the route, further based on the positions of the similar vehicles, including the position of the similar vehicle, acquired by the acquisition unit in association with the accuracy information indicating an accuracy lower than the threshold value.

[0010] The output unit may stop outputting the second accurate route when transitioning from a state in which the first accurate route cannot be output to a state in which the first accurate route can be output. The information processing device may further include a storage control unit that, when the determination unit determines that a similarity between a feature of the registered vehicle stored in the storage unit and a feature of the target vehicle indicated by the feature information is equal to or greater than a threshold, causes the storage unit to store the location information, which indicates a location where the captured image containing the target vehicle is captured, in association with the registered vehicle for a predetermined period of time, and, when the determination unit determines that the similarity between the feature of the registered vehicle stored in the storage unit and the feature of the target vehicle indicated by the feature information is less than the threshold, deletes the location information, which indicates a location where the captured image containing the target vehicle is captured, from the storage unit without storing the location information in the storage unit for the predetermined period of time, and a reception unit that receives request information requesting information on the location of the registered vehicle from a user terminal, and when the reception unit receives the request information, the output unit may transmit the location information, which is associated with the registered vehicle and stored in the storage unit, to the user terminal.

[0011] An information processing method of a second aspect of the present invention is executed by a computer and includes the steps of acquiring, from a plurality of image-capturing vehicles, feature information indicating the features of a target vehicle contained in captured images of the surroundings of each of the plurality of image-capturing vehicles, accuracy information indicating the accuracy of the feature information, and location information indicating the location at which the captured images were captured; referring to a memory unit that stores the features of registered vehicles, determining whether the target vehicle is a similar vehicle based on the features of the registered vehicle and the features of the target vehicle indicated by the feature information; and outputting the route traveled by the similar vehicle based on a plurality of locations at which a plurality of image-capturing images including the similar vehicle were captured and the accuracy of the features of the similar vehicle.

[0012] An information processing system according to a third aspect of the present invention is an information processing system comprising a plurality of image capturing vehicles that capture images of their surroundings, and an information processing device that communicates with the plurality of image capturing vehicles, wherein the image capturing vehicles have an image capturing device that generates an image of the surroundings of the image capturing vehicle, an extraction unit that extracts features of a target vehicle included in the image capturing image, and a transmission control unit that transmits to the information processing device feature information indicating the features of the target vehicle included in the image capturing image, accuracy information that indicates the accuracy of the feature information, and location information that indicates the location where the image capturing image was taken, and the information processing device has an acquisition unit that acquires from the plurality of image capturing vehicles the feature information indicating the features of the target vehicle included in the image capturing image of the surroundings of each of the plurality of image capturing vehicles, accuracy information that indicates the accuracy of the feature information, and location information that indicates the location where the image capturing image was taken, a memory unit that stores the features of registered vehicles, a determination unit that determines whether the target vehicle is a similar vehicle based on the features of the registered vehicle and the features of the target vehicle indicated by the feature information, and an output unit that outputs the route traveled by the similar vehicle based on the locations where a plurality of image capturing images including the similar vehicle were captured and the accuracy of the features of the similar vehicle. [Effects of the Invention]

[0013] According to the present invention, it is possible to suppress the false recognition of a vehicle that is not stolen as a stolen vehicle. [Brief explanation of the drawings]

[0014] [Figure 1] 1 shows a configuration of an information processing system S according to an embodiment. [Figure 2] FIG. 2 is a diagram illustrating a configuration of an imaging vehicle. [Figure 3] 10 shows an example of characteristic information transmitted by a transmission control unit to an information processing device. [Figure 4] 1 shows the configuration of an information processing device. [Figure 5] An example of a method for improving the accuracy of route estimation will be described. [Figure 6] 10 shows another example of route estimation by the estimation unit. [Figure 7]10 is a flowchart showing a processing procedure for estimating a route traveled by a registered vehicle by an information processing device. [Figure 8] 10 is a flowchart showing a processing procedure for estimating a route of a registered vehicle by an information processing device. DETAILED DESCRIPTION OF THE INVENTION

[0015] 1 shows the configuration of an information processing system S of this embodiment. The information processing system S includes a plurality of image capturing vehicles 101 and an information processing device 200. The image capturing vehicle 101 is a vehicle that captures images of its surroundings. The image capturing vehicle 101 transmits captured images generated by capturing images of its surroundings to the information processing device 200 via a network. The information processing device 200 analyzes whether the captured images received from the image capturing vehicle 101 include a vehicle (hereinafter referred to as a "similar vehicle") similar to a pre-registered vehicle (e.g., a stolen vehicle), thereby identifying the route traveled by the similar vehicle and outputting the identified route.

[0016] 1 shows two image capturing vehicles 101, it is assumed that the information processing device 200 receives captured images from three or more image capturing vehicles 101. The image capturing vehicles 101 are, for example, commercial vehicles capable of autonomous driving. The image capturing vehicles 101 may be equipped with an ADAS (Advanced Driver Assistance System) for assisting the driver in driving operations.

[0017] While the imaging vehicle 101 is traveling, it captures images of the surroundings of the imaging vehicle 101 using an imaging device. The imaging vehicle 101 recognizes vehicles (hereinafter also referred to as target vehicles) surrounding the imaging vehicle 101 by performing image recognition on the captured images generated by the imaging device. In Fig. 1, multiple target vehicles recognized by the imaging vehicle 101 are shown inside the dashed lines.

[0018] The image capturing vehicle 101 extracts features of the recognized target vehicle, such as the vehicle model and vehicle identification number. The image capturing vehicle 101 determines the accuracy of the extracted features. The image capturing vehicle 101 transmits feature information indicating the extracted features of the target vehicle, accuracy information indicating the accuracy of the features indicated by the feature information, and location information indicating the location of the image capturing vehicle 101 to the information processing device 200 ((1) in FIG. 1 ).

[0019] The information processing device 200 communicates with a plurality of image capturing vehicles 101. The information processing device 200 acquires feature information, accuracy information, and location information from the plurality of image capturing vehicles 101. The information processing device 200 stores features of registered vehicles. A registered vehicle is a vehicle whose features are registered in advance by a user in order to search for the location of the registered vehicle.

[0020] The information processing device 200 identifies the degree of similarity between the stored features of the registered vehicle and the features of the target vehicle indicated by the feature information acquired from the captured vehicle 101 ((2) in FIG. 1). The information processing device 200 determines whether the identified similarity is equal to or greater than a predetermined threshold. The information processing device 200 repeats the same determination on the feature information acquired from other captured vehicles 101, thereby identifying multiple locations where multiple captured images showing similar vehicles, which are target vehicles determined to be similar to the registered vehicle, were captured. The information processing device 200 estimates at least one or more routes traveled by the similar vehicles based on the identified multiple locations ((3) in FIG. 1).

[0021] When the information processing device 200 receives request information requesting information indicating the route traveled by a registered vehicle from an information terminal (not shown) owned by a person using the information processing device 200 to search for a stolen vehicle or a user of a registered vehicle, the information processing device 200 transmits information indicating the route traveled by a similar vehicle similar to the registered vehicle to the information terminal. At this time, the information processing device 200 outputs information indicating the route traveled by the similar vehicle using a method described below, based on accuracy information indicating the accuracy of the characteristics of the similar vehicle. In this way, when a vehicle is stolen, the information processing device 200 can identify the route traveled by the stolen vehicle. At this time, the information processing device 200 can prevent the misidentification of the stolen vehicle from occurring due to excessive trust in information indicating the characteristics of the target vehicle with low accuracy.

[0022] [Configuration of imaging vehicle] 2 is a diagram showing the configuration of the imaging vehicle 101. The imaging vehicle 101 includes a control device 1, an imaging device 2, a LIDAR (Light Detection And Ranging) 3, a position sensor 4, and a communication unit 5.

[0023] The control device 1 is, for example, an ECU (Electronic Control Unit). The control device 1 includes a storage unit 11 and a control unit 12. The control unit 12 includes a generation unit 121, an extraction unit 122, an identification unit 123, and a transmission control unit .

[0024] The imaging device 2 generates captured images by capturing images of the surroundings of the traveling imaging vehicle 101 at a predetermined frame rate. The imaging device 2 inputs the generated captured images to the generation unit 121. The LIDAR 3 measures the distance to an object around the imaging vehicle 101 and the shape of the object by emitting laser light and measuring the reflected wave of the laser light. The LIDAR 3 inputs information indicating the measured distance to the object and the shape of the object to the generation unit 121.

[0025] The position sensor 4 measures the position of the image capturing vehicle 101. The position sensor 4 has, for example, a GPS (Global Positioning System) receiver. The position sensor 4 identifies the latitude and longitude of the position of the image capturing vehicle 101 based on position information contained in radio waves received from a GPS satellite. The position sensor 4 inputs the position information indicating the position of the image capturing vehicle 101 to the identification unit 123.

[0026] The communication unit 5 is a wireless communication module for communicating with the information processing device 200. The communication unit 5 transmits various pieces of information input from the transmission control unit 124 to the information processing device 200.

[0027] The storage unit 11 is configured with, for example, a read-only memory (ROM) and a random access memory (RAM). The storage unit 11 stores a program to be executed by the control unit 12. The control unit 12 is, for example, a processor mounted on an ECU. The control unit 12 executes the program stored in the storage unit 11 to function as a generation unit 121, an extraction unit 122, an identification unit 123, and a transmission control unit 124.

[0028] The generation unit 121 generates shape information indicating the shapes of objects around the imaging vehicle 101 based on at least one of the captured image input from the imaging device 2 and the measurement results of the LIDAR. The generation unit 121 may generate the shape information using millimeter wave radar.

[0029] The extraction unit 122 extracts features of the target vehicle included in the captured image generated by the imaging device 2. For example, the features of the target vehicle are at least one of the vehicle type, color, and vehicle identification number. For example, the extraction unit 122 determines a degree of certainty indicating the likelihood of the extracted features based on the shape information generated by the generation unit 121. The extraction unit 122 may determine the degree of certainty based on the captured image. For example, when the shape indicated by the shape information is the shape of the target vehicle when viewed from a position where the front of the target vehicle is visible, the extraction unit 122 increases the degree of certainty of the extracted vehicle identification number compared to when the shape indicated by the shape information is the shape of the target vehicle when viewed from a position where the front of the target vehicle is not visible. When the shape indicated by the shape information is the shape of the target vehicle when viewed from an oblique angle, the extraction unit 122 increases the degree of certainty of the extracted vehicle type compared to when the shape indicated by the shape information is the shape of the target vehicle when viewed from the front.

[0030] The extraction unit 122 may determine the accuracy using a machine learning model. In this case, the extraction unit 122 reads out from the storage unit 11 a trained machine learning model that uses shape information as input data and outputs a vehicle model and an accuracy that indicates the accuracy of the vehicle model. The extraction unit 122 inputs the shape information into the read machine learning model and obtains the vehicle model and the accuracy that are output by the machine learning model, thereby extracting the vehicle model and the accuracy of the vehicle model of the target vehicle. The extraction unit 122 outputs the extracted features and the accuracy that indicates the accuracy of the features to the identification unit 123 and the transmission control unit 124.

[0031] The identification unit 123 identifies the position of the imaging vehicle 101 based on the position information input from the position sensor 4. The identification unit 123 outputs information indicating the identified position of the imaging vehicle 101 to the transmission control unit .

[0032] The transmission control unit 124 transmits various types of information to the information processing device 200 via the communication unit 5. The transmission control unit 124 transmits feature information indicating the features of the target vehicle appearing in the captured image generated by the generation unit 121. The transmission control unit 124 transmits accuracy information indicating the accuracy of this feature information. The transmission control unit 124 transmits location information indicating the location at which the captured image was captured to the information processing device 200. The transmission control unit 124 transmits time information indicating the time at which the captured image was captured to the information processing device 200.

[0033] 3 shows an example of feature information transmitted by the transmission control unit 124 to the information processing device 200. As shown in the first and second rows from the top of Fig. 3, the transmission control unit 124 transmits time information indicating the time when the captured image was captured, "July 3, 2024, 10:10:05," and location information indicating the location where the captured image was captured, "XXX degrees north latitude, YYY degrees east longitude." As shown in the third row from the top of Fig. 3, the transmission control unit 124 transmits feature information indicating the model of the target vehicle shown in the captured image, "XXX company, YY model," and accuracy information indicating an accuracy of "85%" that is the likelihood of this model.

[0034] 3, the transmission control unit 124 transmits feature information indicating the color "red" of the target vehicle shown in the captured image and accuracy information indicating a 95% accuracy of this color. As shown in the fifth row from the top of Fig. 3, the transmission control unit 124 transmits feature information indicating the vehicle identification number "ZZ-ZZ at Shinagawa YYY" of the target vehicle shown in the captured image and accuracy information indicating a 90% accuracy of this vehicle identification number.

[0035] [Configuration of information processing device 200] 4 shows the configuration of the information processing device 200. The information processing device 200 includes a communication unit 21, a storage unit 22, and a control unit 23. The control unit 23 includes a reception unit 231, an acquisition unit 232, a determination unit 233, an estimation unit 234, an output unit 235, and a storage control unit 236.

[0036] The communication unit 21 is an interface for communicating with information terminals owned by users of the multiple image capture vehicles 101 and registered vehicles. The communication unit 21 inputs the received various information to the control unit 23.

[0037] The storage unit 22 is configured by, for example, a ROM, a RAM, etc. The storage unit 22 stores a program to be executed by the control unit 23. The storage unit 22 stores characteristics of registered vehicles. The characteristics of registered vehicles are at least one of the vehicle type, color, and vehicle identification number. In the example of FIG. 4, the storage unit 22 stores the characteristics of registered vehicles in association with the identification information of the registered vehicles.

[0038] The control unit 23 is, for example, a CPU (Central Processing Unit). The control unit 23 executes a program stored in the storage unit 22, thereby functioning as a reception unit 231, an acquisition unit 232, a determination unit 233, an estimation unit 234, an output unit 235, and a storage control unit 236.

[0039] The reception unit 231 communicates with an information terminal owned by a user of the registered vehicle via the communication unit 21. The reception unit 231 receives first request information requesting information on the location of the registered vehicle from the user's information terminal. The first request information includes identification information of the registered vehicle (e.g., vehicle identification number). The reception unit 231 may also receive second request information requesting information on the route traveled by the registered vehicle from the information terminal of the user of the registered vehicle. The second request information includes identification information of the registered vehicle. The second request information may include information provided by the user indicating where the registered vehicle is parked and the time the user last confirmed the registered vehicle. The reception unit 231 outputs at least one of the received first request information and second request information to the output unit 235.

[0040] The acquisition unit 232 acquires various types of information from the multiple image capturing vehicles 101 via the communication unit 21. The acquisition unit 232 acquires, from the multiple image capturing vehicles 101, feature information indicating features of the target vehicle included in captured images of the surroundings of each of the multiple image capturing vehicles 101, accuracy information indicating the accuracy of this feature information, and location information indicating the location at which the captured image was captured. The acquisition unit 232 acquires time information indicating the time at which the captured image was captured.

[0041] The acquisition unit 232 outputs the acquired feature information to the determination unit 233. The acquisition unit 232 outputs the acquired position information and time information to the estimation unit 234. The acquisition unit 232 outputs the acquired accuracy information to the estimation unit 234 and the output unit 235.

[0042] The determination unit 233 determines whether the target vehicle is a similar vehicle based on the features of the registered vehicle stored in the storage unit 22 and the features of the target vehicle indicated by the feature information acquired by the acquisition unit 232. A similar vehicle is a target vehicle that is similar to any of the registered vehicles stored in the storage unit 22 by a predetermined threshold or more. The predetermined threshold is, for example, a similarity level at which the probability that the registered vehicle and the similar vehicle do not match is 1%. Note that the determination unit 233 may use other methods to determine whether the target vehicle and the registered vehicle are similar. For example, a partial image of the target vehicle included in the captured image may be used as input data to determine whether the target vehicle is similar to the registered vehicle using a learning model that determines whether the target vehicle is similar to the registered vehicle. Alternatively, the similarity of the target vehicle included in the captured image and the registered vehicle may be determined using other methods such as so-called image matching.

[0043] First, the determination unit 233 determines whether the accuracy indicating the accuracy of the vehicle model of the target vehicle acquired by the acquisition unit 232 is equal to or greater than a first accuracy threshold α and whether the accuracy indicating the accuracy of the color of the target vehicle is equal to or greater than a second accuracy threshold β. The first accuracy threshold α and the second accuracy threshold β are, for example, the minimum accuracy values ​​expected when a captured image is generated in a standard environment. When the accuracy indicating the accuracy of the vehicle model of the target vehicle acquired by the acquisition unit 232 is equal to or greater than the first accuracy threshold α and the accuracy indicating the accuracy of the color of the target vehicle is equal to or greater than the second accuracy threshold β, the determination unit 233 determines a first similarity between the vehicle model of the registered vehicle and the vehicle model indicated by the feature information acquired by the acquisition unit 232. The determination unit 233 determines a second similarity between the color of the registered vehicle and the color indicated by the feature information acquired by the acquisition unit 232. The determination unit 233 determines a third similarity between the vehicle identification number of the registered vehicle and the vehicle identification number indicated by the feature information acquired by the acquisition unit 232.

[0044] The determination unit 233 determines whether the identified first similarity is equal to or greater than a first threshold and whether the identified second similarity is equal to or greater than a second threshold. The first threshold is, for example, the minimum value of the first similarity expected between a registered vehicle and a target vehicle that have the same color when the captured images are generated in a standard environment. The second threshold is, for example, the minimum value of the second similarity expected between a registered vehicle and a target vehicle that have the same model when the captured images are generated in a standard environment.

[0045] When the first similarity is equal to or greater than the first threshold and the second similarity is equal to or greater than the second threshold, the determination unit 233 determines whether the accuracy indicating the accuracy of the vehicle identification number of the target vehicle acquired by the acquisition unit 232 is equal to or greater than a third accuracy threshold γ. The third accuracy threshold γ is, for example, the minimum value of accuracy expected when the captured image is generated in a standard environment.

[0046] When the accuracy indicating the accuracy of the vehicle identification number of the target vehicle is equal to or greater than a third accuracy threshold γ, the determination unit 233 identifies a third similarity between the vehicle identification number of the registered vehicle and the vehicle identification number indicated by the feature information acquired by the acquisition unit 232. The determination unit 233 determines whether the identified third similarity is equal to or greater than the third threshold. The third threshold is, for example, the minimum value of the third similarity expected between a registered vehicle and a target vehicle having the same vehicle identification number when a captured image is generated in a standard environment. When the determination unit 233 determines that the third similarity is equal to or greater than the third threshold, the determination unit 233 associates the location information, time information, and registered vehicle corresponding to the feature information acquired by the acquisition unit 232 with each other, labels them as main information, and stores them in the storage unit 22.

[0047] On the other hand, when the first similarity is equal to or greater than the first threshold and the second similarity is equal to or greater than the second threshold, but the accuracy indicating the reliability of the registered vehicle's vehicle identification number is less than the third accuracy threshold γ, the judgment unit 233 associates the location information corresponding to the feature information acquired by the acquisition unit 232 with the time information and the identification information of the registered vehicle, labels them as reference information, and stores them in the memory unit 22.

[0048] If the accuracy indicating the accuracy of the vehicle type of the target vehicle acquired by the acquisition unit 232 is less than a first accuracy threshold α, or if the accuracy indicating the accuracy of the color of the target vehicle is less than a second accuracy threshold β, the determination unit 233 discards the position information and time information corresponding to the feature information acquired by the acquisition unit 232 without storing them in the storage unit 22. If the specified first similarity is less than the first threshold, if the second similarity is less than the second threshold, or if the third similarity is less than the third threshold, the determination unit 233 discards the position information and time information corresponding to the feature information acquired by the acquisition unit 232 without storing them in the storage unit 22. The determination unit 233 repeats the same determination between the features of all registered vehicles stored in the storage unit 22 and the features of the target vehicle indicated by the feature information acquired by the acquisition unit 232.

[0049] The determination unit 233 may calculate an overall similarity based on the first similarity, the second similarity, and the third similarity. For example, the determination unit 233 may calculate the overall similarity by taking a weighted average of the first similarity, the second similarity, and the third similarity.

[0050] The determination unit 233 determines whether the calculated overall similarity is equal to or greater than a predetermined overall threshold. If the determination unit 233 determines that the calculated overall similarity is equal to or greater than the overall threshold, the determination unit 233 may associate the location information corresponding to the feature information acquired by the acquisition unit 232, the time information, and the identification information of the registered vehicle, label them as main information, and store them in the storage unit 22. The overall threshold is, for example, a similarity at which the probability that the registered vehicle and the similar vehicle do not match is 1 percent.

[0051] On the other hand, if the judgment unit 233 determines that the calculated overall similarity is less than the overall threshold, it may discard the location information and time information corresponding to this similarity without storing them in the memory unit 22.

[0052] [Route Estimation] The estimation unit 234 estimates the route taken by the similar vehicle. For example, the estimation unit 234 estimates at least one or more routes taken by the similar vehicle, based on multiple positions at which multiple captured images showing a similar vehicle determined by the determination unit 233 to be similar to a registered vehicle were taken. The estimation unit 234 estimates a trajectory of the similar vehicle along one or more routes, based on the relationship between the multiple positions and the multiple times at which the multiple captured images were taken. The trajectory is represented by the route taken by the similar vehicle and the times at which the similar vehicle arrived at each of multiple positions on this route.

[0053] More specifically, the estimation unit 234 reads out from the storage unit 22 the position information and time information that are associated with the identification information of the registered vehicle and labeled as main information and stored in the storage unit 22. The estimation unit 234 plots the positions indicated by the read position information on the map data in the time order indicated by the time information corresponding to the positions, thereby estimating the trajectory of movement of the similar vehicle.

[0054] FIG. 5 shows an example of a movement trajectory of a similar vehicle estimated by the estimation unit 234. The double circles in FIG. 5 indicate positions indicated by multiple pieces of position information acquired by the acquisition unit 232 from multiple image-captured vehicles 101. The estimation unit 234 estimates a trajectory such that the vehicle passes through the positions indicated by these pieces of position information in the order of time indicated by the time information corresponding to the position information. In the example of FIG. 5, the estimation unit 234 estimates a trajectory that starts from position A corresponding to the time "13:02:10", passes through position B corresponding to the time "13:03:05", position C corresponding to the time "13:03:45", and position D corresponding to the time "13:04:15".

[0055] To prevent a target vehicle different from a registered vehicle from being mistakenly recognized as a similar vehicle, the estimation unit 234 estimates a route (hereinafter also referred to as a first accuracy route) traveled by the similar vehicle based on the positions of the multiple similar vehicles acquired by the acquisition unit 232 in association with multiple pieces of accuracy information indicating accuracy levels equal to or higher than a predetermined threshold. For example, the estimation unit 234 estimates a first route traveled by the similar vehicle based on the positions of the multiple similar vehicles acquired by the acquisition unit 232 in association with a color corresponding to an accuracy level equal to or higher than a first accuracy threshold α, a vehicle model corresponding to an accuracy level equal to or higher than a second accuracy threshold β, and a vehicle identification number corresponding to an accuracy level equal to or higher than a third accuracy threshold γ. The estimation unit 234 may estimate multiple first accuracy routes by repeating the same process. In this way, the estimation unit 234 estimates the route of the similar vehicle using relatively reliable feature information, thereby reducing the risk of mistakenly recognizing a target vehicle different from a registered vehicle as a similar vehicle.

[0056] On the other hand, the estimation unit 234 may not be able to estimate the first accuracy route. In this case, the estimation unit estimates a route traveled by the similar vehicle (hereinafter also referred to as a second accuracy route) based further on the positions of a plurality of similar vehicles, including the position of the similar vehicle acquired by the acquisition unit 232 in association with accuracy information indicating an accuracy lower than the threshold.

[0057] For example, when the estimation unit 234 determines that location information or the like labeled as primary information is not stored in association with the identification information of the registered vehicle included in the second request information received by the reception unit 231, the estimation unit 234 identifies the parking location of the registered vehicle indicated in the user-provided information received by the reception unit 231 and the time the user last confirmed the registered vehicle. The estimation unit 234 identifies location information labeled as reference information that indicates a location that is estimated to be inaccessible to the registered vehicle from the parking location. The estimation unit 234 deletes the identified location information and the time information corresponding to this location information from the storage unit 22. The estimation unit 234 estimates a second accuracy route traveled by the similar vehicle by plotting the positions indicated by the remaining location information labeled as reference information stored in the storage unit 22.

[0058] The estimation unit 234 may estimate a second accurate route when the number of positions of the multiple similar vehicles labeled as primary information is less than the number of waypoints required to estimate the route of the similar vehicle. The estimation unit 234 may estimate a second accurate route when the distance between the positions of the multiple similar vehicles labeled as primary information is equal to or greater than a reference value. The reference value is, for example, several kilometers or tens of kilometers.

[0059] The estimation unit 234 may transition from a state in which it is unable to estimate a first accurate route to a state in which it is able to estimate a first accurate route. When the estimation unit 234 transitions from a state in which it is unable to estimate a first accurate route to a state in which it is able to estimate a first accurate route, the estimation unit 234 newly estimates a first accurate route.

[0060] However, even if the characteristics of the target vehicle corresponding to a certainty equal to or higher than the threshold are compared with the characteristics of similar vehicles, the determination unit 233 may erroneously determine that a target vehicle other than the registered vehicle is a similar vehicle. In such a case, multiple similar vehicles exist simultaneously at different locations. Therefore, if the distance between the multiple locations at which similar vehicles are identified is equal to or greater than the distance that makes it impossible for the multiple similar vehicles to travel between the multiple times at which the captured images corresponding to the multiple locations were taken, the estimation unit 234 estimates that the multiple similar vehicles identified at the multiple locations are different vehicles. In this case, the estimation unit 234 estimates the route that one similar vehicle will take so as not to pass through the location at which the other similar vehicle was captured.

[0061] Specifically, the estimation unit 234 estimates the distance traveled by the similar vehicle from the location where the first captured image, which depicts the similar vehicle, was captured to the location where the second captured image, which was captured after the time the first captured image was captured and depicts the similar vehicle, and the time difference between the time the first captured image was captured and the time the second captured image was captured. The estimation unit 234 estimates the travel speed of the similar vehicle based on the estimated travel distance and the estimated time difference. If the estimated travel speed of the similar vehicle is greater than a predetermined value, the estimation unit 234 does not include the location where the second captured image was captured in the route traveled by the registered vehicle. The predetermined value is, for example, the maximum speed at which the similar vehicle can travel or three times the speed limit.

[0062] Fig. 6 shows an example of a method for improving the route estimation accuracy in this way. Assuming that the similar vehicle passes through positions A to E indicated by the position information acquired by the acquisition unit 232 in the example of Fig. 6 in the time order indicated by the time information corresponding to the position information, the estimation unit 234 estimates a trajectory that starts from position A corresponding to the time "9:10:10", passes through position B corresponding to the time "9:13:15", position C corresponding to the time "9:14:10", position D corresponding to the time "9:14:30", and position E corresponding to the time "9:15:05".

[0063] The estimation unit 234 estimates, with respect to the estimated trajectory, the movement speed of the similar vehicle from position A to position B, the movement speed from position B to position C, the movement speed from position C to position D, and the movement speed from position D to position E. In the example of FIG. 6 , the movement speed of the similar vehicle from position B to position C estimated by the estimation unit 234 is greater than a predetermined value. Similarly, the movement speed from position C to position D estimated by the estimation unit 234 is also greater than a predetermined value. On the other hand, the movement speed of the similar vehicle from position A to position B and the movement speed of the similar vehicle from position D to position E estimated by the estimation unit 234 are both equal to or less than a predetermined value. In this case, the estimation unit 234 estimates a new trajectory of the similar vehicle that does not pass through position C. As indicated by the solid arrow in FIG. 6 , the estimation unit 234 estimates a new trajectory that starts from position A and passes through positions B, D, and E in that order.

[0064] [Output of various information] The output unit 235 communicates with an information terminal owned by a person searching for a stolen vehicle using the information processing device 200 or a user of a registered vehicle via the communication unit 21. When the reception unit 231 receives first request information requesting information on the location of the registered vehicle, the output unit 235 identifies the location information stored in the memory unit 22 in association with the identification information of the registered vehicle. The output unit 235 transmits the identified location information to the information terminal.

[0065] When the receiving unit 231 receives second request information requesting information on routes traveled by a registered vehicle, the output unit 235 outputs at least one or more routes estimated by the estimation unit 234 for a similar vehicle determined by the determination unit 233 to have a similarity to the registered vehicle equal to or greater than a predetermined threshold. When the estimation unit 234 estimates a trajectory of travel of the similar vehicle along one or more routes, the output unit 235 outputs the trajectory estimated by the estimation unit 234 as a route.

[0066] The output unit 235 outputs the route traveled by the similar vehicle based on multiple locations where multiple captured images including the similar vehicle were captured and the accuracy of the characteristics of the similar vehicle. The output unit 235 outputs at least one or more routes estimated by the estimation unit 234. For example, the output unit 235 outputs the route in a different manner depending on the accuracy. Specifically, the output unit 235 outputs in red a route that passes only through locations where captured images of the similar vehicle have an accuracy of the vehicle identification number equal to or greater than a threshold. The output unit 235 outputs in blue a route that includes locations where captured images of the similar vehicle have an accuracy of less than a threshold. By outputting the route in different manners depending on the accuracy in this way, it becomes easier for a person viewing the output route to determine whether to head to the current location of the similar vehicle.

[0067] The output unit 235 may output, among at least one or more routes estimated by the estimation unit 234, a route (hereinafter also referred to as a main route) that passes through positions where one or more captured images with a certainty equal to or higher than a predetermined threshold were captured, in preference to a route (hereinafter also referred to as a sub-route) that passes through positions where one or more captured images with a certainty lower than the threshold were captured. For example, the output unit 235 displays the main route on the first page of a map that is displayed on an information terminal of a user of the registered vehicle to display the route traveled by the registered vehicle. The output unit 235 displays the sub-route on the second page of this map.

[0068] Furthermore, the output unit 235 may output at least one or more routes estimated by the estimation unit 234 together with accuracy information corresponding to each route. For example, the output unit 235 may display the accuracy of the vehicle identification number extracted from the captured image at each position where an image showing a similar vehicle was captured on the route estimated by the estimation unit 234.

[0069] When the estimation unit 234 estimates a first accurate route, the output unit 235 outputs the estimated first accurate route. When the estimation unit 234 estimates a second accurate route in a state in which the estimation unit 234 cannot estimate the first accurate route, the output unit 235 outputs the estimated second accurate route. When the state transitions from a state in which the estimation unit 234 cannot estimate the first accurate route to a state in which the estimation unit 234 can estimate the first accurate route, the output unit 235 stops outputting the second accurate route estimated by the estimation unit 234. In this case, the output unit 235 outputs the first accurate route newly estimated by the estimation unit 234.

[0070] [Delete information about vehicles other than registered vehicles] The storage control unit 236 stores the location information acquired by the acquisition unit 232 in the storage unit 22. When the determination unit 233 determines that the similarity between the features of the registered vehicle stored in the storage unit 22 and the features of the target vehicle indicated by the feature information acquired by the acquisition unit 232 is equal to or greater than a predetermined threshold, the storage control unit 236 associates the location information indicating the location where the captured image containing the target vehicle was taken with the registered vehicle and stores the location information in the storage unit 22 for a predetermined period of time. For example, when the determination unit 233 determines that the overall similarity between the features of the registered vehicle stored in the storage unit 22 and the features of the target vehicle indicated by the feature information acquired by the acquisition unit 232 is equal to or greater than the overall threshold, the storage control unit 236 associates the location information indicating the location where the captured image containing the target vehicle was taken with the registered vehicle and stores the location information in the storage unit 22 for a predetermined period of time. The predetermined period is, for example, a period designated by the user of the registered vehicle.

[0071] When the determination unit 233 determines that the degree of similarity between the features of the registered vehicle stored in the storage unit 22 and the features of the target vehicle indicated by the feature information acquired by the acquisition unit 232 is less than a predetermined threshold, the storage control unit 236 deletes the location information indicating the location where the captured image containing the target vehicle was taken from the storage unit 22 without storing the location information for a predetermined period in the storage unit 22. In this way, the storage control unit 236 immediately deletes the captured image containing the target vehicle that is unlikely to be a registered vehicle, thereby preventing the storage unit 22 from running out of capacity.

[0072] [Variation for obtaining feature information and accuracy information from a machine learning model] In the present embodiment, an example has been described in which the acquisition unit 232 acquires, from the image capturing vehicle 101, feature information indicating the features of a target vehicle appearing in a captured image in which the surroundings of each of the multiple image capturing vehicles 101 are captured, and accuracy information indicating the accuracy of this feature information. The acquisition unit 232 may acquire feature information and accuracy information output by a trained machine learning model stored in the storage unit 22. First, the acquisition unit 232 acquires shape information generated by the generation unit 121 from the image capturing vehicle 101. The acquisition unit 232 may input the shape information to a trained machine learning model that uses the shape information as input data and outputs a vehicle type and an accuracy indicating the likelihood of the vehicle type as output data, and acquire the vehicle type and the accuracy of the vehicle type output by the machine learning model.

[0073] [Processing procedure for estimating the route traveled by a registered vehicle by an information processing device] 7 is a flowchart showing a processing procedure for managing location information corresponding to feature information acquired by the acquisition unit 232. This processing procedure starts, for example, when the acceptance unit 231 accepts a user operation to register the feature of a registered vehicle.

[0074] First, the determination unit 233 determines whether or not the accuracy indicating the accuracy of the vehicle model of the target vehicle acquired by the acquisition unit 232 is equal to or greater than a first accuracy threshold α (S101). If the accuracy indicating the accuracy of the vehicle model of the target vehicle acquired by the acquisition unit 232 is equal to or greater than the first accuracy threshold α (YES in S101), the determination unit 233 determines whether or not the accuracy indicating the accuracy of the color of the target vehicle is equal to or greater than a second accuracy threshold β (S102). If the accuracy indicating the accuracy of the color of the target vehicle acquired by the acquisition unit 232 is equal to or greater than the second accuracy threshold β (YES in S102), the determination unit 233 identifies a first similarity between the vehicle model of the registered vehicle and the vehicle model indicated by the feature information acquired by the acquisition unit 232. The determination unit 233 identifies a second similarity between the color of the registered vehicle and the color indicated by the feature information acquired by the acquisition unit 232. The determination unit 233 determines whether the first similarity is equal to or greater than a first threshold and whether the second similarity is equal to or greater than a second threshold (S103).

[0075] If the first similarity is greater than or equal to the first threshold and the second similarity is greater than or equal to the second threshold (YES in S103), the judgment unit 233 judges whether the accuracy indicating the reliability of the vehicle identification number of the target vehicle acquired by the acquisition unit 232 is greater than or equal to the third accuracy threshold γ (S104).

[0076] If the accuracy indicating the likelihood of the vehicle identification number of the target vehicle is equal to or greater than the third accuracy threshold γ (YES in S104), the determination unit 233 identifies a third similarity between the vehicle identification number of the registered vehicle and the vehicle identification number indicated by the feature information acquired by the acquisition unit 232. The determination unit 233 determines whether the third similarity is equal to or greater than the third threshold (S105). If the determination unit 233 determines that the third similarity is equal to or greater than the third threshold (YES in S105), the determination unit 233 associates the location information, time information, and registered vehicle corresponding to the feature information acquired by the acquisition unit 232, labels them as main information, and stores them in the storage unit 22 (S106).

[0077] When the accuracy indicating the accuracy of the vehicle identification number of the registered vehicle is less than the third accuracy threshold γ (NO in S104), the determination unit 233 associates the location information corresponding to the feature information acquired by the acquisition unit 232 with the time information and the identification information of the registered vehicle, labels the information as reference information, and stores the information in the storage unit 22 (S107). When the accuracy of the vehicle type is less than the first accuracy threshold α (NO in S101) or when the accuracy of the color is less than the second accuracy threshold β (NO in S102), the location information and time information acquired by the acquisition unit 232 are discarded (S108). On the other hand, when the accuracy of the vehicle identification number indicated by the accuracy information acquired by the acquisition unit 232 is less than the third accuracy threshold γ (NO in S104), the location information is not discarded because the similarity between the vehicle identification number and the registered vehicle is high for features other than the vehicle identification number, and therefore the location information can be used as reference information when the acquisition unit 232 cannot acquire location information labeled as main information.

[0078] If the determination in S101 shows that the accuracy indicating the accuracy of the vehicle type of the registered vehicle is less than the first accuracy threshold α (NO in S101), the determination unit 233 discards the location information and time information corresponding to the feature information acquired by the acquisition unit 232 (S108). If the determination in S102 shows that the accuracy indicating the accuracy of the color of the registered vehicle is less than the second accuracy threshold β (NO in S102), the determination unit 233 proceeds to the process of S108. If the determination in S103 shows that the first similarity is less than the first threshold or the second similarity is less than the second threshold (NO in S103), the determination unit 233 proceeds to the process of S108. If the determination in S105 shows that the third similarity is less than the third threshold (NO in S105), the determination unit 233 proceeds to the process of S108.

[0079] The storage control unit 236 determines whether or not a predetermined period of time has elapsed since the storage unit 22 associated the position information and time information indicating the position where the captured image containing the similar vehicle was captured with the identification information of the registered vehicle, labeled them as main information or reference information (S109). If the storage control unit 236 determines that the predetermined period of time has elapsed since the storage unit 22 associated the position information and time information with the identification information of the registered vehicle and stored them (YES in S109), the storage control unit 236 deletes the position information and time information from the storage unit 22 (S110) and ends the processing.

[0080] If the memory control unit 236 determines in S109 that a predetermined period has not elapsed since the location information and time information indicating the location where the image containing the similar vehicle was taken were associated with the identification information of the registered vehicle and stored in the memory unit 22 (NO in S109), the memory control unit 236 terminates the processing without deleting the location information from the memory unit 22.

[0081] 8 is a flowchart showing the processing procedure for estimating the route of a registered vehicle by the information processing device 200. This processing procedure starts, for example, when second request information requesting information on the route traveled by the registered vehicle is received from the information terminal of the user of the registered vehicle.

[0082] The estimation unit 234 determines whether or not the location information labeled as main information and the time information are stored in the storage unit 22 in association with the identification information of the registered vehicle included in the second request information received by the reception unit 231 (S201). If the location information labeled as main information and the time information are stored in the storage unit 22 (YES in S201), the estimation unit 234 deletes the location information labeled as reference information and the time information from the storage unit 22 (S202).

[0083] The estimation unit 234 estimates the route taken by the similar vehicle by plotting the position indicated by the position information labeled as the main information on a map (S203). The output unit 235 outputs the route estimated by the estimation unit 234 to the information terminal of the user of the registered vehicle (S204), and the process ends.

[0084] When the estimation unit 234 determines in S201 that no location information or the like labeled as primary information is stored in association with the identification information of the registered vehicle included in the second request information received by the reception unit 231 (NO in S201), the estimation unit 234 identifies the parking location of the registered vehicle indicated by the user-provided information received by the reception unit 231 and the time the user last confirmed the registered vehicle. The estimation unit 234 identifies location information and time information labeled as reference information that indicates a location that is estimated to be inaccessible to the registered vehicle from the parking location. The estimation unit 234 deletes the identified location information and the time information corresponding to this location information from the storage unit 22 (S205). The estimation unit 234 estimates the route traveled by the similar vehicle by plotting the positions indicated by the remaining location information labeled as reference information stored in the storage unit 22 (S206), and proceeds to the processing of S204.

[0085] [Effects of the information processing device 200 of the present disclosure] In the information processing device 200 of this embodiment, the output unit 235 selects a route to output as the route taken by the registered vehicle based on the accuracy of the characteristics of the similar vehicle, or outputs the route taken by the registered vehicle in a manner corresponding to the accuracy of the characteristics of the similar vehicle. In this way, the output unit 235 can output the route taken by the stolen vehicle in the event of the theft of a vehicle. At this time, the output unit 235 can prevent the misidentification of the stolen vehicle from occurring due to excessive trust in information indicating vehicle characteristics with low accuracy.

[0086] The present invention has been described above using embodiments, but the technical scope of the present invention is not limited to the scope described in the above embodiments, and various modifications and changes are possible within the scope of the gist of the present invention. For example, all or part of the device can be configured by functionally or physically distributing or integrating any unit. Furthermore, new embodiments resulting from any combination of multiple embodiments are also included in the embodiments of the present invention. The effects of the new embodiments resulting from the combination also have the effects of the original embodiments. [Explanation of symbols]

[0087] 1. Control device 2. Imaging device 3. LIDAR 4 Position Sensor 5. Communications Department 11 Storage section 12 Control Unit 21 Communications Department 22 Memory section 23 Control Unit 100 Imaging vehicle 101 Imaging vehicle 121 Generation part 122 Extraction part 123 Specific part 124 Transmission control section 200 Information processing device 231 Reception Department 232 Acquisition Department 233 Judgment section 234 Estimation Department 235 Output section 236 Memory control unit

Claims

1. an acquisition unit that acquires, from the plurality of image capturing vehicles, feature information indicating features of a target vehicle included in captured images of the surroundings of each of the plurality of image capturing vehicles, accuracy information indicating accuracy of the feature information, and location information indicating the location where the captured images were captured; a storage unit that stores the characteristics of registered vehicles; a determination unit that determines whether the target vehicle is a similar vehicle based on the characteristics of the registered vehicle and the characteristics of the target vehicle indicated by the characteristic information; an output unit that outputs a route traveled by the similar vehicle based on a plurality of positions at which a plurality of captured images including the similar vehicle were captured and the accuracy of the characteristics of the similar vehicle; An information processing device comprising:

2. the output unit outputs a trajectory of the similar vehicle along the route based on the times at which the plurality of captured images were captured. The information processing device according to claim 1 .

3. the output unit outputs the route that does not include the position where the second captured image is captured when a distance from a position where the first captured image including the similar vehicle is captured to a position where the second captured image which is captured after the first captured image is captured and includes the similar vehicle is greater than a predetermined value. The information processing device according to claim 1 .

4. the output unit outputs the route that passes through a position where one or more captured images having the certainty equal to or greater than a predetermined threshold are captured, in preference to the route that passes through a position where one or more captured images having the certainty lower than the threshold are captured.

3. The information processing device according to claim 1.

5. When the output unit is unable to output as the route a first certainty route along which the similar vehicle has traveled based on the positions of the plurality of similar vehicles acquired by the acquisition unit in association with the plurality of pieces of certainty information indicating a certainty equal to or greater than a predetermined threshold, the output unit outputs as the route a second certainty route along which the similar vehicle has traveled based further on the positions of the plurality of similar vehicles, including the position of the similar vehicle, acquired by the acquisition unit in association with the certainty information indicating a certainty less than the threshold.

3. The information processing device according to claim 1.

6. the output unit stops outputting the second accurate path when a transition is made from a state in which the first accurate path cannot be outputted to a state in which the first accurate path can be outputted. The information processing device according to claim 5 .

7. a storage control unit that, when the determination unit determines that the similarity between the features of the registered vehicle stored in the storage unit and the features of the target vehicle indicated by the feature information is equal to or greater than a threshold, causes the storage unit to store, for a predetermined period of time, the location information indicating the location at which the captured image containing the target vehicle was taken, in association with the registered vehicle; and, when the determination unit determines that the similarity between the features of the registered vehicle stored in the storage unit and the features of the target vehicle indicated by the feature information is less than the threshold, deletes from the storage unit the location information indicating the location at which the captured image containing the target vehicle was taken, without storing the location information in the storage unit for the predetermined period of time; a receiving unit that receives request information requesting information on the location of the registered vehicle from a user terminal, When the receiving unit receives the request information, the output unit transmits the location information stored in the storage unit in association with the registered vehicle to the user terminal.

3. The information processing device according to claim 1.

8. The computer executes acquiring, from the plurality of image capturing vehicles, feature information indicating features of a target vehicle included in captured images of the surroundings of each of the plurality of image capturing vehicles, accuracy information indicating accuracy of the feature information, and location information indicating locations where the captured images were captured; a step of referring to a storage unit that stores features of registered vehicles and determining whether the target vehicle is a similar vehicle based on the features of the registered vehicle and the features of the target vehicle indicated by the feature information; outputting a route traveled by the similar vehicle based on a plurality of positions at which a plurality of captured images including the similar vehicle were captured and the accuracy of the characteristics of the similar vehicle; An information processing method comprising:

9. An information processing system including a plurality of imaging vehicles that capture images of the surroundings and an information processing device that communicates with the plurality of imaging vehicles, The imaging vehicle is an imaging device that generates an image of the surroundings of the imaging vehicle; an extraction unit that extracts features of a target vehicle included in the captured image; a transmission control unit that transmits, to the information processing device, feature information indicating features of the target vehicle included in the captured image, accuracy information indicating accuracy of the feature information, and location information indicating a location where the captured image was captured; The information processing device includes: an acquisition unit that acquires, from the plurality of image capturing vehicles, feature information indicating features of a target vehicle included in captured images of the surroundings of each of the plurality of image capturing vehicles, accuracy information indicating accuracy of the feature information, and location information indicating the location where the captured images were captured; a storage unit that stores the characteristics of registered vehicles; a determination unit that determines whether the target vehicle is a similar vehicle based on the characteristics of the registered vehicle and the characteristics of the target vehicle indicated by the characteristic information; an output unit that outputs a route traveled by the similar vehicle based on a plurality of positions at which a plurality of captured images including the similar vehicle were captured and the accuracy of the characteristics of the similar vehicle; An information processing system comprising:

Images