Information processing device, information processing method, and program

Abstract

To provide a technique that makes it possible to efficiently collect images taken by vehicles.SOLUTION: An information processing device includes: a storage unit that stores travelling histories including travelling positions and directions and photographing times, of a plurality of vehicles that photograph the outside of the respective vehicles; a reception unit that accepts specification of a target location and a target time period; an extraction unit that extracts one or more vehicles in which a photographing range and a target location generated from a photographing possible range on the basis of the travelling position and direction of the vehicle in the target time period satisfy an overlapping condition, based on the travelling histories; and a transmitting unit that transmits an upload instruction to the one or more extracted vehicles, instructing them to upload image data taken during the target time period.SELECTED DRAWING: Figure 6

Description

Technical Field

[0001] The present invention relates to an information processing apparatus, an information processing method, and a program.

Background Art

[0002] Currently, while a vehicle is running, an in-vehicle device called a drive recorder that can record images such as the traveling direction is provided (for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Currently, there are a large number of vehicles running on the road, and images of various locations are taken by the drive recorders installed in these vehicles. Therefore, it is considered possible to obtain information on various locations by collecting and analyzing images from these numerous vehicles.

[0005] However, since there are an enormous number of vehicles running on the road, it is difficult to grasp which vehicle was running at which location and when, and to collect images of necessary locations. <00,00029>

[0006] Therefore, an object of the present invention is to provide a technology that enables efficient collection of images taken by vehicles.

Means for Solving the Problems

[0007] An information processing device according to one aspect of the present invention includes: a storage unit that stores driving history including driving position, driving direction, and shooting time for a plurality of vehicles that photograph the outside of a vehicle; a reception unit that receives a specification of a target location and a target time period; an extraction unit that extracts one or more vehicles that satisfy the condition of overlapping between the shooting range generated from the shooting range based on the driving position and driving direction of the vehicles during the target time period and the target location, based on the driving history; and a transmission unit that transmits an upload instruction to the extracted one or more vehicles instructing them to upload image data taken during the target time period. [Effects of the Invention]

[0008] According to the present invention, it is possible to provide a technology that enables the efficient collection of images taken by a vehicle. [Brief explanation of the drawing]

[0009] [Figure 1] This figure shows an example of an image processing system according to this embodiment. [Figure 2] This figure shows an example of the hardware configuration of the imaging device. [Figure 3] This figure shows an example of the hardware configuration of an image processing device. [Figure 4] This figure shows an example of the functional block configuration of a camera. [Figure 5] This figure shows an example of a collected data database and an instruction content database. [Figure 6] This figure shows an example of the functional block configuration of an image processing device. [Figure 7] This figure shows an example of a collected data database and a detection results database. [Figure 8] This flowchart shows the processing steps performed by the image processing device in the image acquisition and locking function (part 1). [Figure 9] This figure shows an example of a vehicle search screen that accepts instructions for uploading or locking image data. [Figure 10]It is a diagram showing an example of a vehicle search screen that receives an instruction to upload or lock image data. [Figure 11] It is a diagram for explaining the shooting range. [Figure 12] It is a diagram for explaining the duplication condition. [Figure 13] It is a diagram for explaining the duplication condition. [Figure 14] It is a diagram for explaining a method of calculating the time zone during which the upload or lock target location was photographed. [Figure 15] It is a flowchart showing the processing procedure performed by the photographing device in the image collection / lock function (Part 1). [Figure 16] It is a flowchart showing the processing procedure performed by the image processing device for the image collection / lock function (Part 2). [Figure 17] It is a flowchart showing the processing procedure performed by the photographing device for the image collection / lock function (Part 2). [Figure 18] It is a flowchart showing the processing procedure performed by the photographing device for the image collection / lock function (Part 2). [Figure 19] It is a flowchart showing an example of the processing procedure when the image processing device displays image data. [Figure 20] It is a diagram showing the image display screen (Part 1). [Figure 21] It is a diagram for explaining the method of extracting image data. [Figure 22] It is a diagram for explaining the method of extracting image data. [Figure 23] It is a diagram showing the image display screen (Part 2). [Figure 24] It is a flowchart showing an example of the processing procedure when the image processing device performs image detection. [Figure 25] It is a diagram showing an example of a screen for displaying the detected image detection target.

Embodiments for Carrying Out the Invention

[0010] Embodiments of the present invention will be described with reference to the attached drawings. In each drawing, components denoted by the same reference numerals have the same or similar configurations.

[0011] <System Configuration> Figure 1 shows an example of an image processing system 1 according to this embodiment. The image processing system 1 includes a shooting device 10, an image processing device 20, and a terminal 30. The shooting device 10 and the image processing device 20 can communicate with each other, for example, by using wireless communication. The image processing device 20 and the terminal 30 can also communicate with each other, for example, by using wired communication or wireless communication.

[0012] Vehicle C includes, but is not limited to, passenger cars, cargo vans, trucks, buses, and motorcycles. Any vehicle capable of mounting the camera device 10 is acceptable.

[0013] The imaging device 10 determines the position and direction of vehicle C and periodically notifies the image processing device 20. The position of vehicle C can be measured, for example, using a GNSS (Global Navigation Satellite System) or GPS (Global Positioning System) mounted on the imaging device 10 or vehicle C. The direction of vehicle C can be measured, for example, by using the temporal change in the position of vehicle C, or information from various sensors (gyro sensor, geomagnetic sensor, etc.) mounted on the imaging device 10 or vehicle C. The imaging device 10 includes, but is not limited to, a drive recorder, an electronic mirror with recording function, a mobile phone, a smartphone, and a tablet terminal. For example, the imaging device 10 may be a part of vehicle C. The imaging device 10 may be any device that has a recording function.

[0014] Furthermore, the camera 10 captures images of the exterior (surroundings) of vehicle C and stores the captured image data in the memory held by the camera 10. The camera 10 is intended to capture a predetermined angle including the direction of travel of vehicle C, like the front camera of a dashcam, but is not limited to this. For example, it may also capture images in the opposite direction to the direction of travel (i.e., behind vehicle C), like the rear camera of a dashcam, or capture images in all directions including the direction of travel, like a 360-degree camera. When the free memory space of the camera 10 becomes insufficient, it overwrites the oldest image data with new image data.

[0015] The shooting device 10 may continuously capture still images at a predetermined interval, or it may capture video (moving images) at a predetermined frame rate. In other words, the term "image" in this embodiment means a sequence of still images and / or video in chronological order. Also, "frame" means each of the sequence of still images in chronological order, and each of the still images that make up the video.

[0016] The image processing device 20 determines the location of vehicle C based on notifications from the camera 10 and collects image data from multiple vehicles C (from camera 10 mounted on multiple vehicles C). The image processing device 20 also performs analysis of the collected image data. Furthermore, the image processing device 20 displays the image data collected from the camera 10 on the terminal 30.

[0017] In this embodiment, the image processing system 1 may include at least one of the following functions. 1. "Image Collection and Lock Function": This function allows the image processing device 20 to collect (upload) image data taken by the user from the camera device 10 of vehicle C that was traveling during the specified time period and location. It also allows the image processing device 20 to instruct vehicle C, which was traveling during the specified time period and location, to lock the image data taken by the user during the specified time period and location without deleting it. 2. "Image display function": A function that displays image data collected by the image processing device 20 using the "image collection function" on the screen of the terminal 30. The image processing device 20 extracts image data taken at a location specified by the user and at a time specified by the user, and displays it on the screen of the terminal 30. 3. "Image Detection Function": This function allows the image processing device 20 to detect objects (e.g., a specific face, a specific license plate, etc.) in the image data by analyzing the image data collected from the camera 10. It also has a function to search for a specified BLE (Bluetooth L Energy) tag among the BLE tags detected by the camera 10.

[0018] In the following explanation, the location where the image data to be uploaded or locked is taken is referred to as the "upload or lock target location." The time period during which the image data to be uploaded or locked is taken is referred to as the "upload or lock target time period." When explicitly distinguishing between upload and lock, the terms "upload target location," "upload target time period," "lock target location," or "lock target time period" may be used. In the following explanation, for convenience, "image data taken by vehicle C" may be used, but more precisely, it means image data taken by the camera 10 mounted on vehicle C. In convenience, "image data collected from vehicle C" may be used, but more precisely, it means image data collected from the camera 10 mounted on vehicle C. In convenience, "send instructions to vehicle C" may be used, but more precisely, it means send instructions to the camera 10 mounted on vehicle C.

[0019] <Hardware Configuration> (Photography device) Figure 2 shows an example of the hardware configuration of the imaging device 10. The imaging device 10 includes a processor 11 such as a CPU (Central Processing Unit) and a GPU (Graphical Processing Unit), a storage device 12 such as memory, an HDD (Hard Disk Drive) and / or an SSD (Solid State Drive), a communication interface 13 for wired or wireless communication, sensors 14 such as an acceleration sensor, a geomagnetic sensor and / or a gyroscope, an input device 15 for receiving input operations, an output device 16 for outputting information, and a camera 17 for capturing still images or videos. The input device 15 is, for example, an operation button, a touch panel and / or a microphone. The input device 15 may also include an interface for inputting information measured by the vehicle C (vehicle speed and acceleration, etc.). The output device 16 is, for example, a display, a touch panel and / or a speaker.

[0020] (Image processing device) Figure 3 shows an example of the hardware configuration of the image processing device 20. The image processing device 20 has a processor 21 such as a CPU (Central Processing Unit) and a GPU (Graphical Processing Unit), a storage device 22 such as memory, an HDD (Hard Disk Drive) and / or an SSD (Solid State Drive), a communication interface 23 for wired or wireless communication, an input device 24 for receiving input operations, and an output device 25 for outputting information. The input device 24 is, for example, a keyboard, a touch panel, a mouse and / or a microphone. The output device 25 is, for example, a display, a touch panel and / or a speaker.

[0021] The image processing device 20 may consist of one or more physical servers, or it may be configured using a virtual server operating on a hypervisor, or it may be configured using a cloud server. The image processing device 20 may also be called an information processing device.

[0022] <Functional Block Configuration> (Photography device) Figure 4 shows an example of the functional block configuration of the imaging device 10. The imaging device 10 includes a storage unit 100, a measurement unit 101, a notification unit 102, an imaging unit 103, a receiving unit 104, an image management unit 105, a transmission unit 106, and a detection unit 107. The storage unit 100 can be implemented using a storage device 12 provided by the imaging device 10. The measurement unit 101 can be implemented using a sensor 14 provided by the imaging device 10. The imaging unit 103 can be implemented using a camera 17 provided by the imaging device 10. Furthermore, the measurement unit 101, notification unit 102, imaging unit 103, receiving unit 104, image management unit 105, transmission unit 106, and detection unit 107 can be implemented by the processor 11 of the imaging device 10 executing a program stored in the storage device 12. This program can be stored on a storage medium. The storage medium containing the program may be a non-transitory computer-readable medium. The non-transitory medium is not particularly limited, but may include, for example, a USB memory stick or a CD-ROM.

[0023] The storage unit 100 stores a collected data DB (DataBase) 100a for storing driving history and image data, and an instruction content DB 100b for storing instructions received from the image processing device 20.

[0024] Figure 5 shows an example of the collected data DB100a and the instruction content DB100b. The collected data DB100a stores the driving history DB100-1a and the image data DB100-2a. The "driving history" in the driving history DB100-1a is a history that shows when, where, and in what direction vehicle C was traveling. For example, data that associates the driving position, driving direction, and time of vehicle C is accumulated at predetermined intervals (e.g., every 1 second or every 10 seconds). For example, the driving position may be expressed in latitude and longitude, and the driving direction may be expressed as a clockwise angle with the reference direction (e.g., north) as 0 degrees. In the example in Figure 5, the data recorded as driving history is "North latitude x1, East longitude y1, z1 degrees, 2021 / 10 / 22_10:10_00". This means that at the time of the photograph taken on October 22, 2021, at 10:10:00, the vehicle's position was x1 degree North latitude and y1 degree East longitude, with a direction of travel of z1 degree. Similarly, the data "x2 North latitude, y2 East longitude, z2 degrees, 2021 / 10 / 22_10:10_01" means that at the time of the photograph taken on October 22, 2021, at 10:10:01, the vehicle's position was x2 North latitude and y2 degrees East longitude, with a direction of travel of z1 degree. In other words, in the example in Figure 5, the travel history is recorded every second.

[0025] Furthermore, the "image data" in the image data DB100-2a stores the image data captured by the camera 10. The image data may also include information indicating the date and time of capture for each frame. The lock flag indicates the time period during which deletion of the image data is prohibited. In the example in Figure 5, the time period from 10:10:00 to 10:15:59 on October 22, 2021 is set to be prohibited from deletion. Note that the method of recording the driving history and image data in the collected data DB100a is merely an example, and this embodiment is not limited to the example in Figure 5. For example, all captured image data may be stored in a single file, or it may be divided and stored in multiple files (for example, a file for each minute). Also, in the example in Figure 5, the driving history and image data are stored in separate databases, but this is not the only option. A configuration in which all data included in the driving history and image data is stored in a single database is also possible.

[0026] The instruction content DB100b stores an "instruction ID (Identifier)," which is an identifier for uniquely identifying instructions from the image processing device 20. The "instruction ID" may be notified by the image processing device 20, or it may be generated by the imaging device 10 itself. The "instruction content" stores the content of the instructions from the image processing device 20. More specifically, it stores either whether to lock the image data or upload the image data. The "target location" stores information specifying where the image data captured should be locked or uploaded. In this embodiment, the method for specifying where the image data was captured may include specifying a range (area) and specifying a location. For example, "range specification" may be an instruction to lock or upload image data captured within the specified range, and "location specification" may be an instruction to lock or upload image data captured at the specified location. The "target time period" stores information specifying when the image data captured should be locked or uploaded. Note that the specification of either the target location or the target time period may be omitted. Return to Figure 4 and continue the explanation.

[0027] The measurement unit 101 measures the driving position and direction of the vehicle C on which the imaging device 10 is mounted. The measurement unit 101 also associates the measured driving position, direction, and time of measurement of the vehicle C and stores them in the "driving history" of the collected data DB 100a.

[0028] The notification unit notifies the image processing device 20 of the vehicle C's position, direction of travel, and time of measurement at predetermined intervals.

[0029] The camera unit 103 photographs the exterior of the vehicle and stores the captured image data in the collected data DB 100a of the storage unit 100. The camera unit 103 may also store the captured image data in the collected data DB 100a in association with the time of capture. For example, the time of capture for each frame may be embedded in the image data stored in the collected data DB 100a. The camera unit 103 may also store the captured image data in the collected data DB 100a in association with the vehicle's position and direction of travel.

[0030] The receiving unit 104 receives a lock instruction from the image processing device 20, which specifies the conditions for image data that should not be deleted. The receiving unit 104 also receives an upload instruction from the image processing device 20, which instructs the receiving unit 104 to upload the image data.

[0031] When the image management unit 105 receives a lock instruction from the image processing device 20, it changes the state of image data stored in the collected data DB 100a that meets the conditions for image data to be prohibited from being deleted to a state where deletion is prohibited.

[0032] Furthermore, the lock instruction includes specifying a time period to be locked as a condition for image data whose deletion is prohibited. The image management unit 105 may change the state to prohibit deletion for image data whose shooting time corresponds to the specified time period, as this data satisfies the condition for image data whose deletion is prohibited.

[0033] Furthermore, the lock instruction includes the specification of a lock target location as a condition for image data whose deletion is prohibited. The image management unit 105 may change the state to prohibit deletion of image data stored in the collected data DB 100a if the shooting range generated from the shooting range based on the vehicle C's position and direction of travel during the lock target time period overlaps with the target location. Alternatively, the lock target location may be specified as a predetermined range (for example, a 1km square area around a station). In this case, the image management unit 105 may change the state to prohibit deletion of image data in which the shooting range and at least a part of the predetermined range overlap, as image data that satisfies the overlap condition.

[0034] Furthermore, the location to be locked may be specified as a predetermined point (for example, a specific building). In this case, the image management unit 105 may change the state to prohibit deletion of image data in which the predetermined point is included in the shooting range, as image data that satisfies the overlapping condition.

[0035] Furthermore, the lock instruction may specify a predetermined location as the location to be locked, and may also include a specification of the shooting direction as a condition for image data whose deletion is prohibited. "Specification of shooting direction" means specifying from which direction image data of the predetermined location should be locked. In this case, the image management unit 105 may change the state to prohibit deletion of image data that satisfies the overlapping condition, such that the predetermined location is included in the shooting range, and the shooting direction is included within the field of view (field of view of the camera provided by the shooting device 10) based on the direction of travel corresponding to the shooting range.

[0036] The transmitting unit 106 retrieves the image data specified in the upload instruction from the collected data DB 100a stored in the storage unit 100 and transmits it to the image processing device 20. The receiving unit 104 may also receive an upload instruction from the image processing device 20 that includes the upload target time period. In this case, the transmitting unit 106 may retrieve the image data from the collected data DB 100a whose shooting time corresponds to the upload target time period specified in the upload instruction and transmit it to the image processing device 20. If the upload target time period is in the future, the transmitting unit 106 may transmit the image data captured by the shooting unit 103 at the time the future time arrives to the image processing device 20.

[0037] Furthermore, the receiving unit 104 may be configured to receive upload instructions from the image processing device 20, including the upload target time period and the upload target location. In this case, the transmitting unit 106 may transmit to the image processing device 20 image data from the storage unit 100 that satisfies the condition of overlapping between the shooting range generated from the shooting range based on the driving position and driving direction during the upload target time period and the upload target location.

[0038] The detection unit 107 detects wireless signals transmitted from BLE tags located near vehicle C. The detection unit 107 also associates the ID of the BLE tag contained in the detected wireless signal with the detection location and time of the BLE tag, and transmits this information to the image processing device 20.

[0039] (Image processing device) Figure 6 shows an example of the functional block configuration of the image processing device 20. The image processing device 20 includes a storage unit 200, an acquisition unit 201, a receiving unit 202, an extraction unit 203, a transmission unit 204, an image detection unit 205, and a display control unit 206. The storage unit 200 can be implemented using a storage device 22 provided by the image processing device 20. The acquisition unit 201, the receiving unit 202, the extraction unit 203, the transmission unit 204, the image detection unit 205, and the display control unit 206 can be implemented by the processor 21 of the image processing device 20 executing a program stored in the storage device 22. The program can be stored in a storage medium. The storage medium on which the program is stored may be a computer-readable non-temporary storage medium. The non-temporary storage medium is not particularly limited, but may be, for example, a USB memory or a CD-ROM.

[0040] The storage unit 200 stores the collected data DB200a and the detection result DB200b. The collected data DB200a stores the travel history, including the travel position, travel direction, and time, for multiple vehicles that photograph the exterior of the vehicle.

[0041] Figure 7 shows an example of the collected data DB200a and detection result DB200b. The collected data DB200a stores the driving history DB200-1a, image data DB200-2a, and BLE data 200-3a for each vehicle C. The "vehicle ID" is an identifier that uniquely identifies vehicle C, and may be a registration number such as a license plate number. The image data DB200-2a may be a folder that stores image data files. The driving history DB200-1a stores the driving history of vehicle C and image data captured by the camera 10. The driving history DB200-1a and image data DB200-2a are the same as the driving history DB100-1a and image data DB100-2a described in Figure 5, so their explanation is omitted. BLE data 200-3a is a database that stores data received from the camera 10, associating the ID of the BLE tag, the detection location where the BLE tag was detected, and the detection time.

[0042] The detection result DB200b stores the results of an analysis of whether or not the detection target is captured in the image data collected from the imaging device 10. "Detection ID" is an identifier that uniquely identifies the detection result. "Type" indicates the type of the detected object. "Location Information" indicates the location where the detection target was detected. "Shooting Time" indicates the time when the detection target was captured in the image data. "Vehicle ID" indicates the vehicle ID of vehicle C that took the image data containing the detection target. Return to Figure 6 and continue the explanation.

[0043] The acquisition unit 201 acquires the driving position, driving direction, and time from multiple vehicles C at predetermined intervals, associates the acquired driving position, driving direction, and time, and stores them in the "driving history" of the collected data DB 200a.

[0044] The reception unit 202 receives requests from users to specify the upload location and time period, or the location and time period to be locked.

[0045] The extraction unit 203 extracts one or more vehicles C that satisfy the overlapping condition between the shooting range generated from the shooting range based on the driving position and direction of vehicle C during the upload target time period (or lock target time period) and the upload target location (or lock target location), based on the "driving history" of the collected data DB200a.

[0046] Alternatively, the reception unit 202 may accept the designation of a predetermined range (for example, a 1km square area around a station) as the upload target location (or lock target location), and the extraction unit 203 may extract one or more vehicles C that satisfy the overlapping condition, such as one or more vehicles C whose shooting range and at least a part of the predetermined range overlap.

[0047] Furthermore, the reception unit 202 may also accept the designation of a predetermined location (for example, a specific building) as the upload target location (or lock target location). In this case, the extraction unit 203 may extract one or more vehicles that satisfy the overlapping condition, in which the predetermined location is included in the shooting range.

[0048] Furthermore, the reception unit 202 may also accept the specification of the shooting direction and the specification of a predetermined location as the upload target location (or lock target location). In this case, the extraction unit 203 may extract one or more vehicles C that satisfy the overlapping condition, in which the predetermined location is included in the shooting range and the shooting direction is included in the range of the angle of view (angle of view of the camera provided by the shooting device 10) based on the direction of travel corresponding to the shooting range.

[0049] The transmission unit 204 sends an upload instruction (or lock instruction) to one or more vehicles extracted by the extraction unit 203, instructing them to upload (or lock) image data taken during the upload target time period (or lock target time period).

[0050] Furthermore, the reception unit 202 may accept the selection of one or more vehicles from the extracted vehicles to which image data should be uploaded (or locked). In this case, the transmission unit 204 may send an upload instruction (or lock instruction) to the one or more vehicles selected from the extracted vehicles.

[0051] Furthermore, the reception unit 202 may accept the specification of a future time as the upload target time period (or lock target time period). In this case, the extraction unit 203 may store the driving position and driving direction associated with the time indicating the future time in the "driving history" of the collected data DB 200a, and then extract one or more vehicles that satisfy the duplicate condition. The transmission unit 204 may also send an upload instruction (or lock instruction) to the one or more extracted vehicles.

[0052] In addition, each functional unit of the image processing device 20 may perform the following processing in addition to, or instead of, the processing described above.

[0053] The reception unit 202 receives from the user the location and time period of the image data that they want to display on the screen of the terminal 30, selected from the image data stored in the image processing device 20.

[0054] The extraction unit 203 extracts image data from the collected data DB200a in which the shooting range generated from the shooting range based on the vehicle C's position and direction of travel during the specified shooting time period and the specified shooting location satisfy the overlap condition.

[0055] The display control unit 206 displays the image data extracted by the extraction unit 203.

[0056] The image detection unit 205 detects whether or not there is image data (image data containing the image detection target) that corresponds to the image detection target specified by the user in the collected data DB 200a. For example, the reception unit 202 receives the specification of the image detection target from the user, and if the image data stored in the collected data DB 200a contains image data that corresponds to the image detection target, the image detection unit 205 may record in the detection result DB 200b information indicating the shooting location of the image data corresponding to the image detection target and the type of image detection target.

[0057] Furthermore, the display control unit 206 may also display an icon indicating the type of image detection target recorded in the detection result DB 200b on the map screen at a location on the map corresponding to the shooting location of the image data corresponding to the image detection target.

[0058] <Processing Procedure> Next, the "image acquisition and locking function," "image display function," and "image detection function" of the image processing system 1 will be explained in detail. The camera 10 installed in each vehicle C transmits the vehicle C's position, direction of travel, and time to the image processing device 20 at predetermined intervals (for example, every 1 second or every 10 seconds), and the driving position, direction of travel, and time transmitted from each camera 10 are sequentially accumulated in the "driving history" of the collected data DB 200a of the image processing device 20.

[0059] (Image collection and lock function (part 1)) Figure 8 is a flowchart showing the processing procedure performed by the image processing device 20 in the image acquisition and locking function (part 1).

[0060] In step S11, the reception unit 202 receives instructions from the user via the screen of the terminal 30 regarding the location and time period for uploading or locking image data.

[0061] Figure 9 shows an example of a vehicle search screen that accepts instructions for uploading or locking image data. The vehicle search screen shown in Figure 9 displays an input field T100 for entering the time period to be uploaded or locked, a map display area M100 for specifying the location to be uploaded or locked, and an extracted vehicle list L100 that displays a list of extracted vehicles C. The reception unit 202 accepts the time entered in input field T100 as the time period to be uploaded or locked. The reception unit 202 also accepts the location specified in the map display area M100 as the location to be uploaded or locked. In the example in Figure 9, the time period to be uploaded or locked is specified as 9 minutes from 13:16 to 13:25 on December 8th. It is assumed that this time period to be uploaded or locked is a past time period.

[0062] Furthermore, the reception unit 202 accepts the designation of a "predetermined range" as the location to be uploaded or locked. In the example in Figure 9, a predetermined range P100 with a certain size is designated as the location to be uploaded or locked in the map display area M100.

[0063] Figure 10 shows an example of a vehicle search screen that accepts instructions for uploading or locking image data. The reception unit 202 accepts the designation of a "predetermined location" as the location to be uploaded or locked, as well as the designation of the "shooting direction". The shooting direction indicates the direction from which the user wants to upload or lock an image of the predetermined location. For example, if the user wants to see an image of building A viewed from the south, the user specifies the location of building A as the predetermined location, and the shooting direction is from south to north. In the example in Figure 10, location S110 is designated as the location to be uploaded or locked in the map display area M100. In addition to the designation of location S110, the shooting direction D110 is also designated as the direction from which location S110 will be photographed. Other points not mentioned are the same as in Figure 9. When the vehicle search button B100 in Figures 9 and 10 is pressed, the image processing device 20 proceeds to the processing procedure in step S12 of Figure 8.

[0064] In step S12, the extraction unit 203 refers to the "driving history" in the collected data DB 200a and extracts one or more vehicles C whose shooting range, generated from the shooting range based on the vehicle's driving position and direction during the upload or lock target time period, overlaps with the upload or lock target location. The processing procedure performed by the extraction unit 203 is described in detail below.

[0065] [Generating the shooting area] First, the extraction unit 203 refers to the "driving history" in the collected data DB 200a and obtains the driving position and direction of each vehicle C during the specified upload or lock target time period (December 8, 13:16 to 13:25). Next, for each vehicle C, the extraction unit 203 calculates the shooting range of the camera 10 mounted on the vehicle C based on the driving position, driving direction, orientation of the camera mounted on the camera 10 (hereinafter referred to as "shootable direction"), camera field of view, and camera shooting range during the target time period. The camera field of view may be, for example, the horizontal field of view of the lens equipped on the camera 10. The camera shooting range means how many meters ahead from the position of the lens is considered to be able to be photographed, and may be predetermined for each camera 10 based on lens performance, pixel count, etc. For example, it may be 100 meters or 200 meters.

[0066] Figure 11 is a diagram illustrating the shooting range. L indicates the shooting distance, a indicates the field of view, b indicates the direction of travel, and c indicates the shooting direction. The shooting direction c is expressed as a clockwise angle with the direction of travel as the reference (0 degrees). In the example in Figure 11, the shooting direction is the same as the direction of travel of vehicle C. Typically, for a front camera of a dashcam, the shooting direction is the same as the direction of travel of vehicle C (i.e., 0 degrees). For a rear camera of a dashcam, the shooting direction is the opposite direction to the direction of travel of vehicle C (i.e., 180 degrees). In this embodiment, the shooting range when vehicle C is stationary is called the "shooting range". RE300 in Figure 11 shows the shooting range, which can be determined from the direction of travel, shooting direction c, field of view a, and shooting distance.

[0067] The travel route R300 indicates the route taken by vehicle C during the upload or lock period. The travel route R300 can be determined by arranging the travel positions of vehicle C during the upload or lock period (December 8th, 13:16-13:25) in chronological order. The shooting range C300 indicates the range that can be photographed by the camera 10 mounted on vehicle C during the upload or lock period. The shooting range C300 can be calculated by moving the photographable range RE300 along the travel route R300. For example, the extraction unit 203 moves the photographable range RE300 along the travel route RE300 from the start point to the end point, and determines the area filled by the photographable range RE300 as the shooting range C300. In other words, the shooting range C300 means the entire range photographed by vehicle C (camera 10) while it was moving during the upload or lock period.

[0068] [Vehicle extraction] Next, the extraction unit 203 determines for each vehicle C whether the shooting range and the upload or lock target location satisfy the overlapping condition, and extracts one or more vehicles that satisfy the overlapping condition. First, let's explain the case where a "predetermined range" is specified as the upload or lock target location. In this case, the extraction unit 203 determines for each vehicle C whether the shooting range and at least a part of the predetermined range overlap.

[0069] Figure 12 is a diagram illustrating the overlap condition. In specific example E400 in Figure 12, the shooting range C400 and the predetermined range P400 do not overlap. Therefore, the extraction unit 203 determines that vehicle C does not satisfy the overlap condition. On the other hand, in specific example E410, the shooting range C410 overlaps with a part of the predetermined range P410. Therefore, the extraction unit 203 determines that vehicle C satisfies the overlap condition. Also in specific example E410, the shooting range C410 overlaps with the entire predetermined range P410. Therefore, the extraction unit 203 determines that vehicle C satisfies the overlap condition.

[0070] Next, we will explain the case where a "predetermined location" and a "shooting direction" are specified as the upload or lock target location. In this case, the extraction unit 203 determines, for each vehicle C, whether the predetermined location is included in the shooting range and whether the shooting direction is included in the range of the angle of view based on the direction of travel corresponding to the shooting range (i.e., the direction of travel at the time the shooting range was photographed). For example, the shooting direction D501, the direction of travel c, and the range of the angle of view are expressed as clockwise angles with the direction of travel being 0 degrees, where the reference direction (e.g., north) is 0 degrees. If the direction of travel c is 90 degrees and the angle of view is 90 degrees, the range of the angle of view based on the direction of travel c will be in the range of 45 degrees to 135 degrees. Similarly, if the direction of travel c is 180 degrees and the angle of view is 90 degrees, the range of the angle of view based on the direction of travel c will be in the range of 135 degrees to 215 degrees.

[0071] Figure 13 is a diagram illustrating the overlapping conditions. In the specific example E500 in Figure 13, the predetermined point S500 is included in the shooting range C500, and the shooting direction D501 is included in the field of view based on the direction of travel c. Therefore, the extraction unit 203 determines that vehicle C satisfies the overlapping conditions. On the other hand, in the specific example E510 in Figure 13, the predetermined point S510 is not included in the shooting range C510. Therefore, the extraction unit 203 determines that vehicle C does not satisfy the overlapping conditions. Also, in the specific example E520 in Figure 13, the predetermined point S520 is included in the shooting range C520, but the shooting direction D520 is not included in the field of view based on the direction of travel c. Therefore, the extraction unit 203 determines that vehicle C does not satisfy the overlapping conditions.

[0072] As mentioned above, the shooting range is the entire range captured by vehicle C while it is moving during the upload or lock target time period. However, since vehicle C travels on a road, its direction of travel may change during the upload or lock target time period. Therefore, if the direction of travel changes during the upload or lock target time period, the extraction unit 203 may determine that vehicle C satisfies the overlap condition if the condition of specific example E500 is met in any of the directions of travel. In other words, the extraction unit 203 may determine that vehicle C satisfies the overlap condition if the predetermined point S500 is included in the shooting range C500, and there is a time during the upload or lock target time period when the angle of view based on the direction of travel c is included in the shooting direction D501. In cases where the road on which vehicle C is traveling is curved, for example, it is conceivable that vehicle C that has not photographed the predetermined point from the shooting direction may be extracted depending on the direction of travel when vehicle C is traveling near the predetermined point. However, at the very least, it is possible to narrow down the vehicle C whose image data will be uploaded or locked.

[0073] The extraction unit 203 extracts one or more vehicles C that satisfy the duplication condition by performing the processing procedure described above. The display control unit 206 displays the list of the one or more extracted vehicles C in the extracted vehicle list L100 shown in Figures 9 and 10. Returning to Figure 8, the explanation continues.

[0074] In step S13, the reception unit 202 accepts the selection of a vehicle from among the one or more extracted vehicles for which image data upload or locking is to be instructed (hereinafter referred to as the "instructed vehicle"). For example, the user can select one or more vehicles from the extracted vehicle list L100 in Figures 9 and 10. Note that the processing procedure in step S13 in Figure 8 may be omitted. In that case, all vehicles C extracted in the processing procedure of step S12 become the instructed vehicles.

[0075] In step S14, the reception unit 202 receives a selection from the user: whether to instruct the vehicle to upload image data or to lock the image data. If the image upload button B101 in Figures 9 and 10 is pressed, the reception unit 202 accepts a selection to instruct the vehicle to upload image data. On the other hand, if the image lock button B102 in Figures 9 and 10 is pressed, the reception unit 202 accepts a selection to instruct the vehicle to lock the image data. If the user chooses to lock the image data, the process proceeds to step S15. If the user chooses to upload the image data, the process proceeds to step S16.

[0076] In step S15, the transmitting unit 204 sends a lock instruction to the target vehicle, which includes the lock target time period received from the user in the processing procedure of step S11 as the lock condition. If there are multiple target vehicles, the transmitting unit 204 sends a lock instruction including the lock target time period to each of the multiple vehicles C. For example, in the example in Figure 9, if the image lock button B102 is pressed with the target time period set to December 8th, 13:16 to 13:25, the transmitting unit 204 sends a lock instruction with the lock target time period of December 8th, 13:16 to 13:25 to all target vehicles.

[0077] In step S16, the transmission unit 204 sends an upload instruction to the target vehicle, which includes the upload target time period received from the user in the processing procedure of step S11 as the upload condition. If there are multiple target vehicles, the transmission unit 204 sends an upload instruction to each of the multiple vehicles C, which includes the upload or lock target time period. For example, in the example in Figure 9, if the image upload button B101 is pressed with the target time period set to December 8th, 13:16 to 13:25, the transmission unit 204 sends an upload instruction to all target vehicles with the upload target time period set to December 8th, 13:16 to 13:25.

[0078] In step S17, the acquisition unit 201 acquires image data from vehicle C that sent the upload instruction and stores it in the collected data DB 200a.

[0079] In step S18, the display control unit 206 displays the video data stored in the collected data DB 200a on the terminal 30.

[0080] [Example of upload lock time period] In the processing procedure of step S15 described above, the transmission unit 204 transmits a lock instruction that includes the lock target time period specified by the user. That is, if the user enters the 9 minutes from 13:16 to 13:25 on December 8th as the target time period in Figure 9 or Figure 10, the transmission unit 204 transmits a lock instruction to the target vehicle with the lock target time period being from 13:16 to 13:25 on December 8th. Similarly in the processing procedure of step S16, the transmission unit 204 transmits an upload instruction to all target vehicles with the upload target time period being from 13:16 to 13:25 on December 8th.

[0081] However, the target time period may include periods other than the time when the upload or lock target location is actually captured in the image. Therefore, the image processing device 20 may send an upload or lock instruction by specifying the time period within the upload or lock target time period specified by the user during which the shooting device 10 was actually capturing the upload or lock target location.

[0082] Figure 14 is a diagram illustrating a method for calculating the time period during which the upload or lock target location was being photographed. As shown in specific examples E550 and E560, if a predetermined range P550 is specified as the upload or lock target location, the extraction unit 203 may identify the time X at which the photographable range RE300 and the predetermined range P550 began to overlap, and the time X+t1 at which the photographable range RE300 and the predetermined range P550 no longer overlap, and set the time from the identified time X to time X+t1 as the time period during which the photographing device 10 was photographing the upload or lock target location. In other words, the extraction unit 203 may set the time period during which the photographable range RE300 and the predetermined range P550 overlapped as the time period during which the photographing device 10 was photographing the upload or lock target location.

[0083] Furthermore, as shown in specific examples E570 and E580, if a predetermined location S570 and shooting direction D570 are specified as the upload or lock target location, the extraction unit 203 identifies the time Y at which the shooting range RE300 and the predetermined location S570 begin to overlap, and the time Y+t2 at which the shooting range RE300 and the predetermined location S570 no longer overlap. Subsequently, the extraction unit 203 determines whether there is a time in the time period from time Y to time Y+t2 when the shooting direction D570 is included in the field of view range based on the driving direction b. If there is a time when the shooting direction D570 is included in the field of view range based on the driving direction b, the identified time from Y to time Y+t2 may be considered as the time period when the shooting device 10 was shooting the upload or lock target location. If there is no time when the shooting direction D570 is included in the field of view range based on the driving direction b, the identified time from Y to time Y+t2 may not be considered as the time period when the shooting device 10 was shooting the upload or lock target location.

[0084] This prevents the inclusion of image data that does not depict the location being uploaded or locked within the uploaded or locked image data.

[0085] Figure 15 is a flowchart showing the processing procedure performed by the imaging device 10 in the image acquisition and locking function (part 1).

[0086] In step S21, the receiving unit 104 receives either a lock instruction including lock conditions or an upload instruction including upload conditions from the image processing device 20, which collects image data from multiple vehicles C. If an upload instruction is received, the process proceeds to step S22. If a lock instruction is received, the process proceeds to step S23.

[0087] In step S22, the transmission unit 106 retrieves image data from the collected data DB 100a stored in the storage unit 100 that satisfies the upload conditions specified in the upload instruction and transmits it to the image processing device 20. For example, if the upload target time period is specified as December 8th, 13:16 to 13:25, the transmission unit 106 retrieves image data from the collected data DB 100a corresponding to December 8th, 13:16 to 13:25 and transmits it to the image processing device 20.

[0088] In step S23, the image management unit 105 changes the state of image data stored in the collected data DB 100a that satisfies the lock conditions specified in the lock instruction to a state where deletion is prohibited. For example, if the lock target time period is specified as December 8th, 13:16 to 13:25, the image management unit 105 changes the state of image data stored in the collected data DB 100a that corresponds to December 8th, 13:16 to 13:25 to a state where deletion is prohibited.

[0089] The image collection and locking function described above using Figures 8 to 15 was explained assuming that a past time period was specified as the time period to be uploaded or locked. However, the reception unit 202 of the image processing device 20 can also accept the specification of a future time period as the time period to be uploaded or locked in the processing procedure of step S11 in Figure 8. In this case, the extraction unit 203 of the image processing device 20 may wait until the specified future time period, and after the driving position and driving direction associated with the time indicating the future time period are stored in the "driving history" of the collected data DB 200a, the processing procedure of step S12 in Figure 8 may be executed.

[0090] (Image collection and lock function (part 2)) In the image acquisition and locking function (part 1) described above, the image processing device 20 extracts the target vehicle and executes an upload or lock command for the extracted vehicle C. On the other hand, in the image acquisition and locking function (part 2), the camera 10 of each vehicle C is instructed on the conditions for uploading image data or locking image data, and the camera 10 of each vehicle C automatically uploads or locks the image data if it matches the instructed conditions.

[0091] Figure 16 is a flowchart showing the processing procedure performed by the image processing device 20 for the image acquisition and locking function (part 2).

[0092] In step S31, the reception unit 202 receives instructions from the user via the screen of the terminal 30 regarding the target location and target time period, which are conditions for uploading or locking the image data captured by the imaging device 10.

[0093] For example, the reception unit 202 accepts the time entered in the input field T100 in Figure 9 or Figure 10 as the time period to be uploaded or locked. The reception unit 202 also accepts the location specified in the map display area M100 in Figure 9 or Figure 10 as the location to be uploaded or locked. More specifically, as shown in Figure 9, if a predetermined range P100 having a certain size is specified as the location to be uploaded or locked, the reception unit 202 accepts the specification of the predetermined range as the location to be uploaded or locked.

[0094] Furthermore, as shown in Figure 10, if location S110 is specified as the upload or lock target location, and shooting direction D110 is specified as the direction in which to photograph location S110, the reception unit 202 accepts the specification of the predetermined location as the upload or lock target location, as well as the specification of the shooting direction. Note that the upload or lock target time period may be a past time or a future time period. When the user presses the image upload button B101 or the image lock button B102, the process proceeds to step S32 in Figure 16.

[0095] In step S32, the transmission unit transmits to each vehicle C stored in the "Vehicle ID" of the collected data DB200a an upload instruction including the upload target time period and upload target location received from the user in the processing procedure of step S31, or a lock instruction including the lock target time period and lock target location received from the user in the processing procedure of step S31.

[0096] For example, in the example in Figure 9, if the target time period is set to December 8th, 13:16 to 13:25, and the predetermined range P100 is set as the lock target location, and the image lock button B102 is pressed, the transmission unit 204 will send a lock instruction to vehicle C stating that the lock target location is the predetermined range P100 and the lock target time period is December 8th, 13:16 to 13:25. Also, for example, in the example in Figure 9, if the target time period is set to December 8th, 13:16 to 13:25, and the image upload button B101 is pressed, the transmission unit 204 will send an upload instruction to vehicle C stating that the upload target location is the predetermined range P100 and the upload target time period is December 8th, 13:16 to 13:25.

[0097] Figure 17 is a flowchart showing the processing procedure performed by the imaging device 10 for the image acquisition and locking function (part 2). The processing procedure shown in Figure 17 is executed repeatedly at a predetermined cycle.

[0098] In step S41, the receiving unit 104 receives / waits for a lock instruction or upload instruction from the image processing device 20.

[0099] In step S42, the receiving unit 104 stores the lock conditions included in the lock instruction or the upload conditions included in the upload instruction received from the image processing device 20 in the instruction content DB 100b. For example, suppose a lock instruction is received in which a predetermined range is specified as the location to be locked, and the lock time period is specified as from 0:00 on November 1, 2021 to 23:59 on November 20, 2021. In this case, the receiving unit 104 stores the lock conditions in the instruction content DB 100b shown in Figure 5 by generating a record with instruction ID J001.

[0100] Figure 18 is a flowchart showing the processing procedure performed by the imaging device 10 for the image acquisition and locking function (part 2). The processing procedure shown in Figure 18 is executed repeatedly at predetermined intervals.

[0101] In step S51, the image management unit 105 searches the collected data DB 100a for any image data that satisfies the upload conditions or lock conditions. If such data exists, the process proceeds to step S52; otherwise, the process terminates.

[0102] In step S52, the transmission unit 106 transmits to the image processing device 20 image data that meets the upload conditions from among the image data stored in the collected data DB 100a. The image management unit 105 also changes the state of image data that meets the lock conditions from among the image data stored in the collected data DB 100a to a state where deletion is prohibited.

[0103] Here, the processing procedures for steps S51 and S52 will be explained in detail, divided into several cases.

[0104] [Case 1] If a past time period is specified as the time period to be locked, the image management unit 105 may change the state of the image data stored in the collected data DB 100a to a state where deletion is prohibited for image data whose shooting time corresponds to the specified past time period.

[0105] Furthermore, if a past time period is specified as the time period to be uploaded, the transmission unit 106 may transmit to the image processing device 20 image data from the collected data DB 100a whose capture time corresponds to the specified past time period.

[0106] [Case 2] If a future time period is specified as the time period to be locked, the image management unit 105 may wait until the future time period, and when image data for that future time period is stored in the collected data DB 100a, change the state of said image data to one where deletion is prohibited.

[0107] Furthermore, if a future time period is specified as the upload target time period, the transmission unit 106 may wait until the future time period, and when image data for that future time period is stored in the collected data DB 100a, it may transmit the image data to the image processing device 20.

[0108] [Case 3] If a past time period is specified as the time period to be locked, and a location to be locked is also specified, the image management unit 105 may change the state of image data stored in the collected data DB 100a to a state where deletion is prohibited if the image data whose shooting time falls within that past time period, and whose shooting range generated from the shooting range based on the driving position and direction of driving during that past time period, and the location to be locked, satisfy the overlapping condition. For example, if time A to time B is specified as the time period to be locked, and the shooting range generated from the shooting range based on the driving position and direction of driving during that time period, and the location to be locked, satisfy the overlapping condition, the image management unit 105 will change the state of image data from time A to time B among the image data stored in the collected data DB 100a to a state where deletion is prohibited.

[0109] More specifically, if the location to be locked is specified as a predetermined range, the image management unit 105 may change the state to prohibit deletion of image data that satisfies the overlapping condition, such as image data where the shooting range in the specified past time period overlaps with at least a part of the predetermined range. Furthermore, if the lock instruction includes the specification of the shooting direction as a condition for image data to be prohibited from being deleted, and the location to be locked is specified as a predetermined point, the image management unit 105 may change the state to prohibit deletion of image data that satisfies the overlapping condition, such as image data where the predetermined point is included in the shooting range in the specified past time period, and the shooting direction is included in the field of view based on the direction of travel when shooting the shooting range.

[0110] The image management unit 105 may also lock image data for the time period during which the camera 10 actually photographed the target location, within the time period designated as the lock target time period. The image management unit 105 may also calculate the time period during which the camera 10 actually photographed the target location using the same method as the method used by the extraction unit 203 of the image processing device 20 to calculate the lock target time period, as described in the "[Modification of Upload Lock Target Time Period]" section above.

[0111] Furthermore, the processing when a past time period is specified as the upload target time period and an upload target location is specified may be the same as the processing described in Case 3 when a past time period is specified as the lock target time period and a lock target location is specified, but the word "lock" is replaced with the word "upload" and the phrase "change to a state where deletion is prohibited" is replaced with the phrase "send to the image processing device 20".

[0112] The method by which the image management unit 105 or the transmission unit 106 extracts image data that satisfies the duplication condition may be the same as the method by which the extraction unit 203 of the image processing device 20 extracts image data that satisfies the duplication condition, as explained using the processing procedure of step S12 in Figure 9 and Figures 11 to 15.

[0113] [Case 4] If a future time period is specified as the time period to be locked, and a location to be locked is also specified, the image management unit 105 may wait until the future time period, and after the image data for that future time period is stored in the collected data DB 100a, it may change the state of the image data stored in the collected data DB 100a that corresponds to that future time period and satisfies the overlap condition between the shooting range generated from the shooting range based on the driving position and direction of driving in the past time period and the location to be locked, to a state where deletion is prohibited. Other points that are not mentioned may be the same as in Case 3.

[0114] Furthermore, if a future time period is specified as the upload target time period and a predetermined range is specified as the upload target location, the transmission unit 106 may wait until the future time period, and when image data for that future time period is stored in the collected data DB 100a, it may transmit to the image processing device 20 image data that corresponds to that future time period and is stored in the collected data DB 100a, where the shooting range generated from the shooting range based on the driving position and direction of driving in the past time period and the target location satisfy the overlap condition. Other points not mentioned may be the same as in Case 3.

[0115] In step S53, the image management unit 105 or the transmission unit 106 returns to the processing procedure of step S51 unless the process is terminated, for example, when the imaging device 10 shuts down.

[0116] As described above, the image collection and locking function allows only image data that requires verification to be uploaded from the camera 10 to the image processing device 20. This reduces the processing load on the image processing device 20 and the communication network compared to the case where images are constantly uploaded from each camera 10 to the image processing device 20. Furthermore, the image collection and locking function allows image data that may or may not need verification later to be locked, temporarily stored on the camera 10 side, and then uploaded when verification becomes necessary. Therefore, the load on the communication network, which is required to instruct the camera 10 to lock image data and then upload it when needed, can be further reduced. In addition, by having the camera 10 receive instructions from the image processing device 20, which collects image data from multiple vehicles C, and perform image data locking / uploading, multiple vehicles C and the image processing device 20 can cooperate to collect appropriate image data at the appropriate time.

[0117] (Image display function) The image processing device 20 displays the image data stored in the collected data DB 200a on the screen of the terminal 30. The image data stored in the collected data DB 200a may be image data uploaded according to the processing procedure shown in Figure 8. Furthermore, the processing procedure is not limited to that shown in Figure 8, and the collected data DB 200a may store image data uploaded from all vehicles C at predetermined intervals (for example, every 10 minutes or every hour).

[0118] Figure 19 is a flowchart showing an example of the processing procedure when the image processing device 20 displays image data.

[0119] In step S61, the reception unit 202 receives from the user via the terminal 30 the location where the image data to be displayed on the screen of the terminal 30 was taken (shooting location) and the time period when the image data to be displayed on the screen of the terminal 30 was taken (shooting time period).

[0120] Figure 20 shows the image display screen (part 1). The image display screen shown in Figure 20 displays an input field T600 for entering the shooting time and a map display area M600 for specifying the shooting location. The reception unit 202 accepts the date and time entered in input field T600 as the shooting time. The reception unit 202 also accepts the location specified in the map display area M600 as the shooting location. In the example in Figure 20, the shooting time is specified as a 9-minute period from 13:16 to 13:25 on December 8th. Note that, unlike the screen shown in Figure 9, the image display screen displays image data that has already been uploaded, so it is possible to make it possible to enter only past time periods in input field T600.

[0121] Furthermore, the reception unit 202 accepts the designation of a predetermined area as the shooting location. In the example in Figure 20, a predetermined area P600 with a certain size is designated as the shooting location in the map display area M600. Note that in the map display area M600, it is also possible to designate a predetermined point (for example, a specific building) as the shooting location. In this case, the reception unit 202 accepts the designation of the predetermined point as the shooting location. In addition to designating a predetermined point, it is also possible to specify the shooting direction in which the predetermined point was photographed. In this case, the reception unit 202 accepts the designation of both the predetermined point and the shooting direction in which the predetermined point was photographed. Returning to Figure 19, we continue the explanation.

[0122] In step S62, the extraction unit 203 extracts image data from the collected data DB200a that satisfies the overlapping conditions between the shooting range, which is generated from the shooting range based on the vehicle C's position and direction of travel during the shooting time period, and the shooting location, and that was taken during the shooting time period.

[0123] The method by which the extraction unit 203 extracts image data from the collected data DB 200a is the same as the processing procedure in step S12 of Figure 9 and the processing procedure explained using Figures 11 to 15, provided that "upload or lock target time period" and "upload or lock target location" are replaced with "shooting time period" and "shooting location," respectively.

[0124] In step S63, the display control unit 206 displays a list of extracted image data on the terminal 30. The reception unit 202 also receives a selection of image data to be displayed from the list from the user. For example, as shown in Figure 20, the display control unit 206 may overlay and display one or more vehicles C that captured the extracted image data on the map display area M600. The reception unit 202 may also receive a selection of vehicles C from the map display area M600 for which to view the image data.

[0125] In step S64, the display control unit 206 displays the selected image data on the terminal 30. For example, in the image display screen of Figure 20, the selected image data is displayed in the image display area V600. Also, if the user operates the slider S600, the display control unit 206 displays a still image taken at the time specified by the slider S600 in the image display area V600.

[0126] [Displaying multiple image data for a specific location] In this embodiment, the image processing device 20 may receive a designation from the user for a designated location and time to be displayed on the screen of the terminal 30, and display multiple image data images taken at the designated location side by side. In this case, in step S61, the reception unit 202 receives the designation of a designated location (predetermined location) and a designated time (predetermined time) from the user via the terminal 30. In step S62, the extraction unit 203 extracts first extracted image data and second extracted image data from the collected data DB 200a as image data for the time period in which the designated location overlaps with the shooting range based on the driving position and direction of the vehicle C.

[0127] More specifically, the extraction unit 203 extracts image data from the image data DB 200-2a as first extracted image data, which includes time periods in which the specified location overlaps with the shooting range based on the vehicle C's position and direction of travel, and which include shooting times earlier than the specified time. The extraction unit 203 also extracts image data from the image data DB 200-2a as second extracted image data, which includes time periods in which the specified location overlaps with the shooting range based on the vehicle C's position and direction of travel, and which include shooting times later than the specified time.

[0128] The first extracted image data and the second extracted image data may be extracted from image data taken by different vehicles C, or they may be extracted from image data taken by the same vehicle C. For example, the first extracted image data may be extracted from image data taken by vehicle C in the morning at a specific location, and the second extracted image data may be extracted from image data taken by the same vehicle C in the evening.

[0129] Figure 21 is a diagram illustrating the method for extracting image data. In specific examples E700 and E710 of Figure 21, the shooting range RE300 is the same as the shooting range RE300 explained in Figure 11. The extraction unit 203 uses the driving history and image data of each vehicle C stored in the collected data DB200a to perform the following steps A and B for each vehicle C. After steps A and B are completed for all vehicle C, the extraction unit 203 performs steps C and onward.

[0130] "Procedure A": The extraction unit 203 refers to the "driving history" and determines whether there is a time period (hereinafter referred to as "overlapping time period") in which the designated point S700 overlaps with the shooting range RE300. If there is, proceed to "Procedure B"; otherwise, terminate the process.

[0131] "Procedure B": The extraction unit 203 identifies the overlapping time period from time Z, when the shooting range RE300 and the designated point S700 begin to overlap, to time Z+t3, when the designated point S700 leaves the shooting range RE300.

[0132] "Procedure C": The extraction unit 203 determines whether there is one or more vehicles C whose time is earlier than the specified time in the overlapping time period identified in Procedure B, and whether there is one or more vehicles C whose time is later than the specified time in the overlapping time period identified in Procedure B. If they exist, the process proceeds to Procedure D; otherwise, the process ends.

[0133] "Procedure D": The extraction unit 203 selects one vehicle C (hereinafter referred to as "Vehicle 1") from among one or more vehicles C whose time slots include a time earlier than the specified time within the overlapping time slots identified in Procedure B. It also selects one vehicle C (hereinafter referred to as "Vehicle 2") from among one or more vehicles C whose time slots include a time later than the specified time within the overlapping time slots identified in Procedure B.

[0134] "Procedure E": The extraction unit 203 extracts image data taken during the time period from time Z to time Z+t3 from the image data DB200-2a of the first vehicle as the first extracted image data. The extraction unit 203 also extracts image data taken during the time period from time Z to time Z+t3 from the image data DB200-2a of the second vehicle as the second extracted image data.

[0135] Figure 22 is a diagram illustrating the image data extraction method. First, the extraction unit 203 performs procedures A and B to identify overlapping time periods for vehicles C1 to C4. It is assumed that there are no overlapping time periods for vehicle C5. Next, the extraction unit 203 performs procedure C. According to Figure 22, the vehicles C whose overlapping time periods include a time earlier than the specified time are vehicles C1, C3, and C4, and the vehicles C whose overlapping time periods include a time later than the specified time are vehicles C1, C2, and C4. Therefore, the extraction unit 203 proceeds to procedure D.

[0136] In step D, the extraction unit 203 selects a first vehicle C from among vehicles C2, C3, and C4, and a second vehicle C from among vehicles C1 and C2. The selection method is arbitrary; for example, it may select a vehicle C whose specified time is included in the overlapping time period (vehicles C1 and C4 in Figure 22), or it may select a vehicle C whose specified time is not included in the overlapping time period (vehicles C2 and C3 in Figure 22). Here, let's assume that vehicle C2 is selected as the first vehicle C and vehicle C3 is selected as the second vehicle C. Subsequently, in step E, the extraction unit 203 extracts the first extracted image data from the image data of vehicle C2 and the second extracted image data from the image data of vehicle C3.

[0137] For example, if the overlapping time period for vehicle C2 is from 13:19:30 to 13:20:00, the extraction unit 203 extracts image data from vehicle C2 whose shooting time is between 13:19:30 and 13:20:00 as the first extracted image data. Similarly, if the overlapping time period for vehicle C3 is from 13:34:45 to 13:35:10, the extraction unit 203 extracts image data from vehicle C3 whose shooting time is between 13:34:45 and 13:35:10 as the second extracted image data.

[0138] Figure 23 shows the image display screen (part 2). The image display screen shown in Figure 23 displays an input field T800 for entering a specified time and a map display area M800 for specifying the shooting location. In the example in Figure 23, the shooting location is specified as a designated point (the point indicated by S850), and the specified time is 13:30:00. Furthermore, the first extracted image data is image data taken between 13:19:30 and 13:20:00, and the second extracted image data is image data taken between 13:34:45 and 13:35:10.

[0139] As shown in Figure 23, the display control unit 206 displays the first extracted image data in the image display area V800 and the second extracted image data in the image display area V810. At this time, the display control unit 206 may also display a still image (first still image) taken at a time before the specified time for the first extracted image data, and a still image (second still image) taken at a time after the specified time for the second extracted image data.

[0140] In the example shown in Figure 23, the image display area V800 displays a still image taken at 13:19:40, which is before the specified time (13:30:00), while the image display area V810 displays a still image taken at 13:35:00, which is after the specified time (13:30:00). When slider S800 is operated, a still image taken at the time specified by slider S800 is displayed. When slider S810 is operated, a still image taken at the time specified by slider S800 is displayed. As a result, the image display screen shows still images taken at the same location at different times (for example, before and after the specified time), allowing the user to check for temporal changes at the same location, for example, based on the time an accident occurred.

[0141] Furthermore, the display control unit 206 displays an icon N800 in the map display area M800 indicating the vehicle C's position and direction of travel at the time the image data displayed in the image display area V800 was taken (i.e., 13:19:40). Similarly, the display control unit 206 displays an icon N810 in the map display area M810 indicating the vehicle C's position and direction of travel at the time the image data displayed in the image display area V810 was taken (i.e., 13:35:00). This allows the user to understand where the image data was taken.

[0142] Furthermore, the extraction unit 203 may extract the first and second image data such that the direction of travel of vehicle C at the time the first image data was taken and the direction of travel of vehicle C at the time the second image data was taken are within a predetermined angle range (for example, within 30 degrees). For example, let's assume that the extraction unit 203 extracted image data from 13:19:30 to 13:20:00 as the first image data. In this case, the extraction unit 203 extracts the second image data such that the direction of travel of vehicle C that took the first image data during the time period from 13:19:30 to 13:20:00 and the direction of travel of vehicle C that took the image data to be extracted as the second image data are within a predetermined angle range. When checking for the presence or absence of temporal changes at the same location, it is considered easier to identify differences by comparing images taken from the same direction at the same location than by comparing images taken from different directions at the same location. This makes it possible for the user to easily check for the presence or absence of temporal changes at the same location.

[0143] (Image detection function) The image processing device 20 detects the object to be detected in the image data by analyzing the image data collected from the imaging device 10. The image data stored in the collected data DB 200a may be image data uploaded according to the processing procedure shown in Figure 8. Furthermore, the processing procedure is not limited to that shown in Figure 8, and the collected data DB 200a may also store image data uploaded from all vehicles C at predetermined intervals (for example, every 10 minutes or every hour).

[0144] Figure 24 is a flowchart showing an example of the processing procedure when the image processing device 20 performs image detection.

[0145] In step S71, the reception unit 202 receives a specification of an image detection target from the user via the terminal 30. The image detection targets may be divided into categories such as a specific person, a specific vehicle license plate, flood damage, or cracks. When a specific person is accepted as the image detection target, the reception unit 202 may accept that the type of detection target is a specific person and that an image of the specific person has been taken. When a specific vehicle license plate is accepted as the image detection target, the reception unit 202 may accept that the type of detection target is a vehicle license plate and that the license plate number of the vehicle C to be detected is. When flood damage is accepted as the image detection target, the reception unit 202 may accept that the type of detection target is "flood damage". When cracks are accepted as the image detection target, the reception unit 202 may accept that the type of detection target is "cracks".

[0146] In step S72, the image detection unit 205 searches the collected data DB 200a for image data (image data containing the image detection target) that corresponds to the image detection target specified in step S71. If an image corresponding to the image detection target is found, the process proceeds to step S73. The image detection unit 205 may, for example, search for the existence of image data corresponding to the image detection target by inputting the image data stored in the collected data DB 200a into a trained model that has the capability to identify the image of the image detection target.

[0147] More specifically, if the image detection target is a specific person, the image detection unit 205 may search for whether image data corresponding to the image detection target exists by inputting the image data stored in the collected data DB 200a and the image of the specific person received from the user into a trained model that has the capability to identify the person in the input image. Also, if the image detection target is a specific vehicle license plate, the image detection unit 205 may search for whether image data corresponding to the image detection target exists by inputting the image data stored in the collected data DB 200a and the vehicle license plate received from the user into a trained model that has the capability to identify vehicle license plates. Furthermore, if the image detection target is a flood, the image detection unit 205 may search for whether image data corresponding to the image detection target exists by inputting the image data stored in the collected data DB 200a into a trained model that has the capability to identify images of floods. Furthermore, if the image detection target is a crack in the road, the image detection unit 205 may input the image data stored in the collected data DB 200a into a trained model capable of identifying images of cracks, and search whether or not there is image data corresponding to the image detection target. Moreover, the method is not limited to using a trained model; known image detection techniques such as template matching may also be used.

[0148] In step S73, the image detection unit 205 tags the image data in which an image detection target has been detected. More specifically, the image detection unit 205 stores the type of image detection target, the location where the image detection target was photographed, the time when the image detection target was photographed, and the vehicle ID of the vehicle C on which the photographing device 10 that took the image data was mounted in the detection result DB 200b.

[0149] In step S74, the image detection unit 205 repeatedly executes the processing procedures from steps S72 to S73 until the search is completed.

[0150] Figure 25 shows an example of a screen displaying detected image targets. As shown in Figure 25, the display control unit 206 may display icons corresponding to the types of image targets recorded in the detection result DB200b on the map screen at the locations where the images that are image targets were taken. Icons TA900 and TA901 indicate locations where road cracks were detected. Icon TA910 indicates locations where a specific person was detected. Icons TA920, TA921, and TA922 indicate locations where flood damage was detected.

[0151] The image processing device 20 may also have a function to detect BLE tags in addition to the detection targets captured in the images collected from the imaging device 10. For example, the detection unit 107 of the imaging device 10 may detect BLE tags present around vehicle C by searching for beacons transmitted from BLE tags while vehicle C is in motion. The transmission unit 106 of the imaging device 10 may transmit information about the detected BLE tag (e.g., BLE tag ID, detection location, detection time) to the image processing device 20 when it detects a BLE tag. The image processing device 20 stores the received BLE tag information in the "BLE data" of the collected data DB 200a.

[0152] Furthermore, the image processing device 20 may accept the specification of the BLE tag to be detected (for example, the BLE tag ID to be detected) in the processing procedure of step S71 in Figure 24. The image detection unit 205 searches the "BLE data" in the collected data DB 200a to see if the specified BLE tag exists, and if the specified BLE tag exists, the display control unit 206 may display an icon on the map screen indicating the location where the BLE tag was detected.

[0153] According to the image detection function described above, it is possible to detect objects in an image by using the image data collected from the camera 10. This makes it possible to detect events such as cracks in a road or flooding in a river.

[0154] <Additional information> The embodiments described above are provided to facilitate understanding of the present invention and are not intended to limit its interpretation. The flowcharts, sequences, elements, and their arrangement, materials, conditions, shapes, and sizes described in the embodiments are not limited to those exemplified and can be modified as appropriate. Furthermore, configurations shown in different embodiments can be partially substituted or combined. [Explanation of Symbols]

[0155] 1…Image processing system, 10…Shooting device, 11…Processor, 12…Storage device, 13…Communication IF, 14…Sensor, 15…Input device, 16…Output device, 17…Camera, 20…Image processing device, 21…Processor, 22…Storage device, 23…Communication IF, 24…Input device, 25…Output device, 30…Terminal, 100…Storage unit, 100a…Collected data DB, 100b…Instruction content DB, 101…Measurement unit, 102…Notification unit, 103…Shooting unit, 104…Receiving unit, 105…Image management unit, 106…Transmission unit, 107…Detection unit, 200…Storage unit, 200a…Collected data DB, 200b…Detection result DB, 201…Acquisition unit, 202…Reception unit, 203…Extraction unit, 204…Transmission unit, 205…Image detection unit, 206…Display control unit

Claims

1. A storage unit that stores the travel history of multiple vehicles that photograph the exterior of the vehicle, including the travel position, travel direction, and time of photography, A reception desk that accepts requests for the location and time slot, Based on the aforementioned driving history, an extraction unit extracts one or more vehicles whose shooting range, generated from the shooting range based on the vehicle's driving position and direction during the target time period, satisfies the condition of overlapping with the target location. A transmission unit that sends an upload instruction to one or more extracted vehicles, instructing them to upload image data taken during the specified time period, It has, The vehicle has an acquisition unit that acquires the driving position and direction from the aforementioned multiple vehicles at predetermined intervals, and stores the acquired driving position and direction in the driving history in association with the time of capture. The reception desk accepts the designation of a future time as the target time period. The extraction unit, after storing the driving position and driving direction associated with the shooting time indicating the future time in the driving history, extracts one or more vehicles that satisfy the overlapping condition. The transmission unit transmits the upload instruction to the one or more vehicles that have been extracted. Information processing device.

2. The reception unit receives the designation of a predetermined range as the target location. The extraction unit extracts one or more vehicles that satisfy the overlapping condition, in which at least a portion of the shooting range and the predetermined range overlap. The information processing apparatus according to claim 1.

3. The reception unit receives the designation of the shooting direction and the designation of a predetermined location as the target location. The extraction unit extracts one or more vehicles that satisfy the overlapping conditions, in which the predetermined point is included in the shooting range, and the shooting direction is included in the field of view range based on the direction of travel corresponding to the shooting range. The information processing apparatus according to claim 1 or 2.

4. The reception unit receives the selection of one or more vehicles from the extracted vehicles for which the upload of the image data is to be instructed. The transmission unit transmits the upload instruction to one or more vehicles selected from the one or more extracted vehicles. The information processing apparatus according to any one of claims 1 to 3.

5. A step of storing in a memory unit the travel history of multiple vehicles that photograph the exterior of the vehicle, including the travel position, travel direction, and time of photography. Steps include accepting requests for the location and time slot to be specified, Based on the aforementioned driving history, the step of extracting one or more vehicles whose shooting range, generated from the shooting range based on the vehicle's driving position and direction during the target time period, and the target location satisfy the overlapping condition. The steps include sending an upload instruction to one or more selected vehicles, instructing them to upload image data taken during the specified time period, The steps include acquiring the driving position and direction from the aforementioned multiple vehicles at predetermined intervals, and storing the acquired driving position and direction in the driving history in association with the time of capture, Includes, The aforementioned acceptance step accepts the designation of a future time as the target time period, The extraction step involves, after the driving history stores the driving position and driving direction associated with the time of capture indicating the future time, extracting one or more vehicles that satisfy the overlapping condition. The aforementioned transmission step involves sending the upload instruction to the one or more extracted vehicles. An information processing method performed by an information processing device.

6. A step of storing in a memory unit the travel history of multiple vehicles that photograph the exterior of the vehicle, including the travel position, travel direction, and time of photography. Steps include accepting requests for the location and time slot to be specified, Based on the aforementioned driving history, the step of extracting one or more vehicles whose shooting range, generated from the shooting range based on the vehicle's driving position and direction during the target time period, and the target location satisfy the overlapping condition. The steps include sending an upload instruction to one or more selected vehicles, instructing them to upload image data taken during the specified time period, The steps include acquiring the driving position and direction from the aforementioned multiple vehicles at predetermined intervals, and storing the acquired driving position and direction in the driving history in association with the time of capture, Have the computer run it, The aforementioned acceptance step accepts the designation of a future time as the target time period, The extraction step involves, after the driving history stores the driving position and driving direction associated with the time of capture indicating the future time, extracting one or more vehicles that satisfy the overlapping condition. The aforementioned transmission step involves sending the upload instruction to the one or more extracted vehicles. program.

Images