Vehicle appearance information collection device, vehicle appearance information collection method, and computer program for vehicle appearance information collection

Abstract

To provide a vehicle information acquisition device capable of transmitting information describing an appearance of a vehicle that is a monitoring target to a server, the information being suitable for an appearance state inspection of the vehicle, while suppressing an increase in a communication volume.SOLUTION: A vehicle appearance information collection device comprises: a detection unit 42 that detects a target vehicle from a sensor signal generated by a sensor 21 when a position of the target vehicle indicated by vehicle position information is included in a detection range of the sensor 21 of the own vehicle; a calculation unit 43 that calculates a confidence level that a reliability of at least one component of the target vehicle is represented in the sensor signal within a range in which the target vehicle is represented when the target vehicle is detected from the sensor signal; and a transmission processing unit 44 that transmits the sensor signal to a server 4 via a wireless communication terminal 23 mounted on the own vehicle when the target vehicle is detected and the reliability of at least one component is below a predetermined detection threshold.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to a vehicle exterior information collection device, a vehicle exterior information collection method, and a vehicle exterior information collection computer program for acquiring appearance information representing the appearance of a vehicle. [Background technology]

[0002] A monitoring service using vehicles has been proposed (see Patent Document 1).

[0003] In the technology disclosed in Patent Document 1, a server, which is a monitoring device, instructs a vehicle to acquire appearance data of a monitoring target specified by a user, and detects whether or not there has been a change in the state of the monitoring target based on the appearance data received from the vehicle. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Publication No. 2022-158104 Summary of the Invention [Problem to be solved by the invention]

[0005] If all images showing the appearance of a monitored object obtained by photographing the object from a vehicle are transmitted to a server, images that are insufficient for use in inspecting the appearance of the monitored object will also be transmitted to the server, resulting in an unnecessary increase in communication traffic.

[0006] Therefore, the present invention aims to provide a vehicle information acquisition device that can transmit exterior information of a vehicle to a server, which is suitable for inspecting the exterior condition of the vehicle being monitored, while suppressing an increase in communication volume. [Means for solving the problem]

[0007] According to one embodiment, there is provided a vehicle exterior information collecting device including: a determination unit that determines whether a position of a target vehicle indicated by vehicle position information is within a detection range of a sensor of the host vehicle; a detection unit that detects the target vehicle from a sensor signal generated by the sensor when the position of the target vehicle is within the detection range; a calculation unit that, when the target vehicle is detected from the sensor signal, inputs the range in which the target vehicle is represented in the sensor signal to a classifier that has been trained in advance to detect at least one component of the target vehicle, thereby calculating, for each of the at least one component, a reliability that the component is represented in the range; and a transmission processing unit that, when the target vehicle is detected from the sensor signal and the reliability of any of the at least one component is less than a predetermined detection threshold, transmits the sensor signal to a server via a wireless communication terminal mounted on the host vehicle.

[0008] In one embodiment, the determination unit determines the relative positional relationship between the target vehicle and the host vehicle when the position of the target vehicle is included in the detection range, the calculation unit calculates the reliability of at least one component that is visible to the sensor in the relative positional relationship between the target vehicle and the host vehicle, and the transmission processing unit transmits the sensor signal to the server only if the reliability of the at least one component that is visible to the sensor is less than the detection threshold. [Effects of the Invention]

[0009] The vehicle exterior information collection device of the present disclosure has the advantage of being able to transmit exterior information of a vehicle to a server that is suitable for inspecting the exterior condition of the vehicle being monitored while suppressing an increase in communication volume. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a schematic configuration diagram of a vehicle management system in which a vehicle appearance information collection device is implemented. [Figure 2] 1 is a hardware configuration diagram of a vehicle appearance information collection device according to an embodiment. [Figure 3]FIG. 2 is a functional block diagram of a processor of the vehicle exterior information collection device. [Figure 4] FIG. 10 is a diagram illustrating an outline of a vehicle appearance information collection process. [Figure 5] 10 is an operation flowchart of a vehicle appearance information collection process. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, a vehicle exterior information collecting device, a vehicle exterior information collecting method executed by the vehicle exterior information collecting device, and a computer program for collecting vehicle exterior information will be described with reference to the drawings. Hereinafter, a vehicle equipped with the vehicle exterior information collecting device may be referred to as the "host vehicle." The vehicle exterior information collecting device determines whether the position of a vehicle to be monitored (hereinafter, sometimes referred to as the "target vehicle") is within the detection range of a sensor mounted on the host vehicle for detecting the surrounding conditions of the host vehicle. If the target vehicle is within the detection range, the vehicle information acquisition device attempts to detect the target vehicle from a sensor signal obtained by the sensor. Furthermore, when the target vehicle is detected from the sensor signal, the vehicle exterior information collecting device inputs the range in which the target vehicle is represented in the sensor signal to a vehicle part detection classifier, thereby calculating the reliability of at least one part of the target vehicle being represented within that range. If the target vehicle is detected from the sensor signal and the reliability of any part is below a predetermined detection threshold, the part may be damaged, deformed, or dirty. Therefore, the vehicle exterior information collection device transmits the sensor signal as appearance information representing the appearance of the target vehicle to a server via a wireless communication terminal installed in the vehicle. On the other hand, if the target vehicle is not detected from the sensor signal, it is highly likely that the target vehicle is not represented in the sensor signal, and therefore the vehicle exterior information collection device does not transmit the sensor signal to the server. Also, if the target vehicle is detected from the sensor signal and the reliability of each component is equal to or greater than the detection threshold, it is highly likely that the target vehicle does not have an abnormality, and therefore the vehicle exterior information collection device does not transmit the sensor signal to the server. In this way, by limiting the sensor signals to be transmitted to the server to those that are highly likely to represent defects in the target vehicle's appearance, the vehicle exterior information collection device aims to transmit sensor signals to the server that are suitable for inspecting the exterior condition of the target vehicle while suppressing an increase in communication volume.

[0012] FIG. 1 is a schematic configuration diagram of a vehicle management system in which a vehicle exterior information collection device is implemented. In this embodiment, the vehicle management system 1 includes a plurality of vehicles 2, a vehicle exterior information collection device 3 mounted on each of the plurality of vehicles 2, and a server 4. The vehicle exterior information collection device 3 is connected to the server 4 via the wireless base station 6 and the communication network 5 by accessing the wireless base station 6 which is connected to the communication network 5 to which the server 4 is connected via a gateway (not shown) or the like. Note that while FIG. 1 illustrates only one wireless base station 6, a plurality of wireless base stations 6 may be connected to the communication network 5.

[0013] The server 4 transmits vehicle position information indicating the position of any other vehicle among the plurality of vehicles 2 to one of the plurality of vehicles 2 in order to instruct the acquisition of appearance information of that other vehicle. Of the plurality of vehicles 2, a vehicle that receives the vehicle position information attempts to photograph any other vehicle among the plurality of vehicles 2 identified by the vehicle position information. Note that, of the plurality of vehicles 2, the vehicle that receives the vehicle position information is an example of the subject vehicle, and the vehicle identified by the vehicle position information is an example of the target vehicle.

[0014] First, the individual vehicles 2 and the vehicle exterior information collection device 3 mounted on each vehicle 2 will be described. In this embodiment, each of the multiple vehicles 2 may be a taxi vehicle controlled by automatic driving, but is not limited to this. At least one of the multiple vehicles 2 may be a taxi vehicle manually driven by a driver. Furthermore, each of the multiple vehicles 2 may be a vehicle used for purposes other than taxis, for example, a vehicle for delivering luggage. Furthermore, the purpose of some of the multiple vehicles 2 may be different from the purpose of the other vehicles of the multiple vehicles 2.

[0015] Each vehicle 2 has a camera 21, a GPS receiver 22, a wireless communication terminal 23, and a vehicle exterior information collection device 3. The camera 21, the GPS receiver 22, the wireless communication terminal 23, and the vehicle exterior information collection device 3 are communicably connected via an in-vehicle network. The vehicle 2 may further have a distance measurement sensor (not shown), such as a LiDAR sensor, for measuring distances to objects around the vehicle 2.

[0016] Camera 21 is an example of a sensor for detecting the situation around vehicle 2, and is attached to vehicle 2 facing a predetermined area around vehicle 2 (for example, the area in front of vehicle 2) so that the predetermined area around vehicle 2 is included in the imaging range of camera 21. Camera 21 then images the predetermined area at predetermined imaging intervals (for example, 1 / 30 to 1 / 10 seconds) and generates an image showing that area. The image generated by camera 21 is an example of a sensor signal. The imaging range of camera 21 is an example of a detection range of a sensor. Note that vehicle 2 may be provided with multiple cameras 21 with different imaging directions or focal lengths.

[0017] Every time the camera 21 generates an image, it outputs the generated image to the vehicle exterior information collection device 3 via the in-vehicle network.

[0018] The GPS receiver 22 receives GPS signals from GPS satellites at predetermined intervals and determines the own position of the vehicle 2 based on the received GPS signals. Then, the GPS receiver 22 outputs positioning information representing the positioning results of the own position of the vehicle 2 based on the GPS signals at predetermined intervals to the vehicle exterior information collection device 3 via the in-vehicle network. Note that the vehicle 2 may have a receiver that complies with a satellite positioning system other than the GPS receiver 22, and that receiver may determine the own position of the vehicle 2.

[0019] The wireless communication terminal 23 is a device that executes wireless communication processing in accordance with a predetermined wireless communication standard, and, for example, by accessing the wireless base station 6, is connected to the server 4 via the wireless base station 6 and the communication network 5. The wireless communication terminal 23 receives a downlink wireless signal including vehicle position information from the server 4 via the communication network 5 and the wireless base station 6, and outputs the received vehicle position information to the vehicle exterior information collection device 3. The wireless communication terminal 23 also generates an uplink wireless signal including an image of the target vehicle, which is received from the vehicle exterior information collection device 3. The wireless communication terminal 23 then transmits the uplink wireless signal to the wireless base station 6, thereby transmitting the image to the server 4. Furthermore, the wireless communication terminal 23 may include planned route information, received from a navigation device (not shown) installed in the vehicle 2, which indicates a planned driving route of the vehicle 2, one or more points on the planned driving route, and the planned times at which the points are to be passed, in the uplink wireless signal to be transmitted to the server 4 together with identification information of the vehicle 2. Furthermore, the wireless communication terminal 23 outputs the vehicle dispatch information received from the server 4, including the planned parking position of the vehicle 2 or the planned boarding position of the user, to an electronic control unit (ECU, not shown) for controlling the driving of the vehicle 2. Furthermore, the wireless communication terminal 23 may include information indicating the position of the vehicle 2 indicated in the positioning information and the orientation of the vehicle 2 indicated by an orientation sensor (not shown) when the vehicle 2 reaches the designated planned parking position, together with the identification information of the vehicle 2, in an uplink wireless signal to be transmitted to the server 4.

[0020] 2 is a hardware configuration diagram of the vehicle exterior information collection device 3. The vehicle exterior information collection device 3 temporarily stores images from the camera 21, positioning information from the GPS receiver 22, and vehicle position information from the server 4. Furthermore, the vehicle exterior information collection device 3 executes vehicle exterior information collection processing based on the images, positioning information, and vehicle position information. To this end, the vehicle exterior information collection device 3 has a communication interface 31, a memory 32, and a processor 33.

[0021] The communication interface 31 is an example of an in-vehicle communication unit, and has an interface circuit for connecting the vehicle exterior information collection device 3 to an in-vehicle network. That is, the communication interface 31 is connected to the camera 21, the GPS receiver 22, and the wireless communication terminal 23 via the in-vehicle network. The communication interface 31 then passes the image received from the camera 21, the positioning information received from the GPS receiver 22, and the vehicle position information received from the server 4 via the wireless communication terminal 23 to the processor 33. Furthermore, the communication interface 31 outputs the image in which the target vehicle is detected, received from the processor 33, to the wireless communication terminal 23 via the in-vehicle network.

[0022] The memory 32 is an example of a storage unit and includes, for example, a volatile semiconductor memory and a nonvolatile semiconductor memory. The memory 32 may further include other storage devices such as a hard disk drive. The memory 32 stores various data used in the vehicle exterior information collection process executed by the processor 33 of the vehicle exterior information collection device 3. For example, the memory 32 stores identification information of the vehicle 2, parameters of the camera 21 such as the focal length, shooting direction, and installation position, and shooting range information (e.g., the angle of view and the maximum distance at which details of the target vehicle can be identified in the image) that represents the shooting range of the camera 21 based on the installation position and shooting direction of the camera 21 relative to the vehicle 2. The memory 32 also stores various parameters for specifying the configuration of various classifiers. The memory 32 also temporarily stores images from the camera 21, positioning information from the GPS receiver 22, and vehicle position information received from the server 4. The memory 32 may also store computer programs for implementing the various processes executed by the processor 33.

[0023] The processor 33 has one or more central processing units (CPUs) and their peripheral circuits. The processor 33 may further have other arithmetic circuits such as a logic unit, a numerical calculation unit, or a graphics processing unit. The processor 33 stores images received from the camera 21, positioning information received from the GPS receiver 22, and the like in the memory 32. Furthermore, upon receiving vehicle position information from the server 4, the processor 33 executes vehicle exterior information collection processing.

[0024] 3 is a functional block diagram of the processor 33 of the vehicle appearance information collection device 3. The processor 33 has a determination unit 41, a detection unit 42, a calculation unit 43, and a transmission processing unit 44. Each of these units included in the processor 33 is a functional module realized by, for example, a computer program running on the processor 33. Alternatively, each of these units included in the processor 33 may be a dedicated arithmetic circuit provided in the processor 33.

[0025] The determination unit 41 determines whether or not the position of the target vehicle indicated by the vehicle position information is included in the photographing range of the camera 21 of the subject vehicle.

[0026] Here, the vehicle location information includes, for example, a planned driving route along which the target vehicle is scheduled to travel, the positions of one or more points on the planned driving route, and the planned time the target vehicle will pass at those points. Alternatively, the vehicle location information may include information indicating the parking position of the target vehicle, the orientation of the parking vehicle, and the time period during which the target vehicle is parked at that parking position (hereinafter referred to as the parking time period). Furthermore, the vehicle location information may include information for identifying the target vehicle, for example, the vehicle registration number of the target vehicle.

[0027] The determination unit 41 identifies the image capture range in real space based on the position of the vehicle 2 (i.e., the position of the vehicle itself) indicated by the latest positioning information, the traveling direction of the vehicle 2 indicated by a direction sensor (not shown), and the image capture range information stored in the memory 32. If the vehicle 2 is equipped with multiple cameras 21, the determination unit 41 identifies the image capture range for each camera. Then, the determination unit 41 identifies the positions of points or parking positions on the planned driving route included in the vehicle position information that are included in the image capture range in real space. If there is no position included in the image capture range, it determines that the position of the target vehicle is not included in the image capture range of the camera of the vehicle itself. On the other hand, if the position of any point or parking position on the planned driving route is included in the image capture range in real space, the determination unit 41 compares the planned passing time of the position of the point included in the image capture range or the parking time zone at the parking position with the current time. If the time difference between the scheduled passing time of the position of the point included in the shooting range and the current time is within a predetermined allowable time, or if the current time is included in the parking time period at the parking position included in the shooting range, the determination unit 41 determines that the position of the target vehicle is included in the shooting range of the camera 21 of the host vehicle. On the other hand, if the time difference is outside the allowable time, or if the current time is not included in the parking time period, the determination unit 41 determines that the position of the target vehicle is not included in the shooting range of the camera of the host vehicle.

[0028] Furthermore, when the position of the target vehicle is included in the capture range of the camera 21 of the host vehicle, the determination unit 41 determines the relative positional relationship between the target vehicle and the host vehicle based on the orientation of the target vehicle along the planned driving route or while parked, and the position and orientation of the host vehicle, thereby determining the orientation of the target vehicle visible from the camera 21. For example, when the camera 21 is mounted facing forward and it is assumed that the host vehicle and the target vehicle will approach each other head-on at a point on the planned driving route included in the capture range, the determination unit 41 determines that the orientation of the target vehicle visible from the camera 21 is the front. Furthermore, when the target vehicle is parked and the host vehicle approaches the target vehicle from behind, causing the target vehicle to be included in the capture range, the determination unit 41 determines that the orientation of the target vehicle visible from the camera 21 is the rear. Furthermore, when the target vehicle passes in front of the host vehicle from right to left and is included in the capture range during the passage, the determination unit 41 determines that the orientation of the target vehicle visible from the camera 21 is the left side. Similarly, when a target vehicle passes in front of the vehicle from left to right and the target vehicle is included in the shooting range during the passage, the determination unit 41 determines that the orientation of the target vehicle visible from the camera 21 is the right side.

[0029] The determination unit 41 notifies the detection unit 42 of the determination result as to whether or not the position of the target vehicle is included in the imaging range of the camera of the host vehicle. Furthermore, if the position of the target vehicle is included in the imaging range of the camera of the host vehicle, the determination unit 41 notifies the calculation unit 43 and the transmission processing unit 44 of the direction of the target vehicle visible from the camera 21.

[0030] When the determination unit 41 determines that the position of the target vehicle is within the image capture range, the detection unit 42 detects the target vehicle from the image generated by the camera 21. To this end, the detection unit 42 inputs at least one image generated by the camera 21 within a predetermined period (e.g., several seconds) before and after the determination unit 41 determines that the position of the target vehicle is within the image capture range to a classifier that has been trained in advance to detect the target vehicle. Then, for any of the input images, if the confidence level output by the classifier that a target vehicle is represented in any region of the image is equal to or higher than a predetermined detection threshold, the detection unit 42 determines that a target vehicle is represented in that region. In other words, the detection unit 42 determines that a target vehicle has been detected from the image. Note that a region determined to represent a target vehicle will be referred to as an object region hereinafter. On the other hand, if the confidence level output by the classifier for any region in any image is less than the detection threshold, the detection unit 42 determines that a target vehicle has not been detected.

[0031] The classifier may be a deep neural network (DNN) with a convolutional neural network (CNN)-type architecture, such as Faster R-CNN or Single Shot Multibox Detector. Alternatively, the classifier may be a classifier based on a machine learning method other than a DNN or neural network with an attention mechanism, such as a Vision Transformer. Such a classifier is trained in advance using a predetermined learning method, such as backpropagation, using a large number of training images including images of the target vehicle. In particular, in this embodiment, the target vehicles are limited to multiple vehicles 2 under the management of the server 4. Therefore, the classifier is trained to detect only vehicles that have the same vehicle model and exterior paint as each vehicle 2, thereby improving the detection accuracy of the target vehicle.

[0032] When a target vehicle is detected, the detection unit 42 notifies the calculation unit 43 and the transmission processing unit 44 of information indicating an image showing the detected target vehicle, and also notifies the calculation unit 43 of information indicating the object area including the target vehicle (for example, the coordinate values ​​of each corner of the object area).

[0033] When a target vehicle is detected from an image, the calculation unit 43 inputs the object region in which the target vehicle is represented in the image to a component detection classifier that has been trained in advance to calculate the reliability of at least one component of the target vehicle being represented. The component detection classifier then calculates a reliability indicating the likelihood that each of the at least one component is represented in the object region. In this embodiment, the at least one component of the target vehicle is a component to be inspected for appearance defects, such as any of the glass, headlights, brake lights, turn signals, license plate, doors, door mirrors, or bumpers. The component detection classifier can be a CNN type or a DNN with an attention mechanism. In this case, the component detection classifier calculates the reliability of each of the at least one component for various regions included in the input object region. The calculation unit 43 then determines the highest reliability of the calculated reliability as the reliability for that component.

[0034] The calculation unit 43 may resize the object region to a predetermined size (e.g., 32 × 32) by performing size conversion processing such as downsampling, upsampling, or bi-cubic interpolation on the object region. The calculation unit 43 may then input the resized object region to the part detection classifier. This allows the part detection classifier to treat the object region as having a constant size even if the relative distance between the host vehicle and the target vehicle is not constant and the size of the target vehicle in the image changes. This simplifies the configuration of the part detection classifier.

[0035] Note that, depending on the component for which reliability is calculated, it may be in a blind spot from the camera 21 and not appear in the image generated by the camera 21. Therefore, the calculation unit 43 identifies components visible from the camera 21, i.e., components that may appear in the image, based on the orientation of the target vehicle visible from the camera 21 notified by the determination unit 41. The calculation unit 43 then calculates the reliability for each identified component and notifies the transmission processing unit 44 of the calculated reliability. Note that it is sufficient if a table indicating, for each orientation of the target vehicle, the relationship between that orientation and the types of components visible when viewing the target vehicle from that orientation is stored in advance in the memory 32. Then, the calculation unit 43 may refer to the table to identify components that may appear in the image for the orientation of the target vehicle visible from the camera 21.

[0036] If a target vehicle is detected from the image and the reliability of at least one part is less than a predetermined detection threshold, the transmission processing unit 44 transmits the image to the server 4 via the wireless communication terminal 23. Note that this detection threshold is a value corresponding to the lower limit of the reliability at which it is determined that a part is represented in the image, and can be, for example, the same value as the detection threshold used by the detection unit 42 to determine whether or not the target vehicle has been detected.

[0037] For parts where the target vehicle is detected from an image but the reliability is below the detection threshold, the calculated reliability value for the part may be lowered due to dirt, scratches, or deformation of the part. Therefore, an image in which the target vehicle is detected in such a case may show some kind of defect in the target vehicle's appearance. Therefore, by uploading such images to server 4, it becomes possible to properly examine the condition of the target vehicle's appearance.

[0038] On the other hand, if a target vehicle is not detected from an image, the transmission processing unit 44 does not transmit the image to the server 4. For an image in which a target vehicle is not detected, it is highly likely that the target vehicle is not depicted for some reason. For example, when generating an image, the target vehicle may be hidden from the camera 21 due to an object such as another vehicle being positioned between the camera 21 of the vehicle and the target vehicle, resulting in the target vehicle not being depicted in the image. By not transmitting such an image, the amount of communication between the wireless communication terminal 23 and the server 4 is reduced. Furthermore, even if a target vehicle is detected from an image, if the reliability of each part is equal to or greater than the detection threshold, the transmission processing unit 44 does not transmit the image to the server 4. If the reliability of each part is equal to or greater than the detection threshold, it is assumed that there is no abnormality in the external condition of each part, i.e., that the dirt on each part is limited and that scratches and deformations are tolerable. Therefore, it is highly likely that such an image does not depict any defects in the external appearance of the target vehicle. Therefore, by omitting the transmission of such an image, the amount of communication between the wireless communication terminal 23 and the server 4 is reduced.

[0039] However, the transmission processing unit 44 may determine whether the reliability of each component visible from the camera 21 is less than the detection threshold value only for that component. The transmission processing unit 44 may transmit the image to the server 4 only when the reliability of any of the components visible from the camera 21 is less than the detection threshold value.

[0040] The transmission processing unit 44 may transmit, together with the image, identification information of the target vehicle, which is indicated by the vehicle position information for the target vehicle determined by the determination unit 41 to be located within the shooting range of the camera 21, to the server 4. This makes it easier for the server 4 to identify the target vehicle shown in the uploaded image, especially when there are multiple target vehicles.

[0041] FIG. 4 is a diagram illustrating an overview of the vehicle appearance information collection process. As shown in FIG. 4, when the target vehicle 400 is located within a shooting range 420 of the camera 21 of the host vehicle 410, which is an example of a detection range, the target vehicle 400 is detected from an image 430 generated by the camera 21. That is, the image 430 is an example of appearance information representing the appearance of the target vehicle 400. Then, for an object region 440 in the image 430 in which the target vehicle 400 is depicted, a reliability C of the part 401 of the target vehicle 400 is calculated. If the reliability C is less than the detection threshold Th, the image 430 is uploaded to the server. On the other hand, if the reliability C is equal to or greater than the detection threshold Th, the image 430 is not uploaded to the server.

[0042] 5 is an operational flowchart of the vehicle exterior information collection process. When the vehicle exterior information collection device 3 receives vehicle position information from the server 4, the processor 33 of the vehicle exterior information collection device 3 executes the vehicle exterior information collection process in accordance with the following operational flowchart.

[0043] The determination unit 41 determines whether the position of the target vehicle indicated by the vehicle position information is included in the image capturing range of the camera 21 of the host vehicle (step S101). If the position of the target vehicle is not included in the image capturing range (step S101-No), the processor 33 ends the vehicle appearance information collection process. On the other hand, if the position of the target vehicle is included in the image capturing range (step S101-Yes), the detection unit 42 detects the target vehicle from the image captured by the camera 21 (step S102).

[0044] If a target vehicle is not detected from the image (step S102-No), the transmission processing unit 44 does not transmit the image to the server 4 (step S103). On the other hand, if a target vehicle is detected from the image (step S102-Yes), the calculation unit 43 calculates the reliability that each part of the target vehicle is represented in the object region in the image where the target vehicle is represented (step S104).

[0045] The transmission processing unit 44 determines whether the reliability of each part is equal to or greater than the detection threshold Th (step S105). If the reliability of all parts is equal to or greater than the detection threshold Th (step S105-Yes), the transmission processing unit 44 does not transmit the image to the server 4 (step S103). On the other hand, if the reliability of any part is less than the detection threshold Th (step S105-No), the transmission processing unit 44 transmits the image to the server 4 via the wireless communication terminal 23 (step S106).

[0046] After step S103 or S106, processor 33 ends the vehicle appearance information collection process.

[0047] Next, the server 4 will be described. The server 4 manages multiple vehicles 2. In this embodiment, the server 4 generates vehicle location information for a target vehicle, which is a vehicle among the multiple vehicles 2 for which a predetermined period of time has elapsed since receiving an image of the vehicle, or a vehicle designated by an administrator via a user interface of the server 4. At this time, the server 4 includes, in the vehicle location information, the identification information of the target vehicle, a planned driving route included in the planned route information received from the target vehicle, or a planned driving route set by a vehicle dispatch server (not shown) in response to a vehicle dispatch request from a user, one or more points on the route, and the planned time of passing through each point. Alternatively, the server 4 may include, in the vehicle location information, the parking position and parking orientation received from the target vehicle, and a parking time zone set by vehicle dispatch management or the like. The server 4 then transmits the vehicle location information for the target vehicle to any of the multiple vehicles 2 other than the target vehicle via the communication network 5 and the wireless base station 6. In this case, the server 4 may select, for example, a vehicle located within a predetermined distance from the current location of the target vehicle from among the multiple vehicles 2 as a vehicle to which vehicle location information is to be transmitted, or may transmit vehicle location information to all vehicles 2 other than the target vehicle. Furthermore, when the server 4 receives an image of the target vehicle from one of the multiple vehicles 2, it determines whether or not there is any defect in the appearance of the target vehicle based on the received image. This determination may be made by an administrator visually inspecting the image, or by the server 4 itself inputting the image into a classifier for determining exterior defects, or by comparing it with a reference image depicting the target vehicle in a normal state. A target vehicle determined to have a defect may be managed so that it is not dispatched until the defect is repaired.

[0048] As described above, this vehicle exterior information collection device detects a target vehicle from an image generated by a camera mounted on the vehicle. Furthermore, this vehicle exterior information collection device calculates the reliability of at least one component of the target vehicle being represented in an object region in the image where the target vehicle is represented. When the target vehicle is detected from the image and the reliability of any component is less than a predetermined detection threshold, this vehicle exterior information collection device transmits the image to a server. This allows the vehicle exterior information collection device to limit the images to be transmitted to the server to those that are likely to represent defects in the target vehicle's appearance, thereby suppressing an increase in communication volume and transmitting images suitable for inspecting the exterior condition of the target vehicle to the server.

[0049] Instead of the camera 21, a distance measurement sensor such as a LiDAR mounted on the vehicle may be used as a sensor for detecting the situation around the vehicle. In this case, when the position of the target vehicle is included in the detection range of the distance measurement sensor, the detection unit 42 detects the target vehicle by inputting the distance measurement signal generated by the distance measurement sensor, which indicates the distance to an object in each direction, to a classifier for target vehicle detection. The distance measurement signal is another example of a sensor signal. The calculation unit 43 may calculate the reliability of each component by inputting the range of the direction in which the target vehicle is represented in the distance measurement signal to a classifier for reliability calculation. In this example, the classifier for target vehicle detection and the classifier for reliability calculation may each be a CNN-type architecture or a DNN with an attention mechanism. The transmission processing unit 44 then transmits to the server 4 a distance measurement signal in which the target vehicle is detected and the reliability of any component is below the detection threshold.

[0050] According to a modified example, when transmitting an image in which a target vehicle is detected to the server 4, the transmission processing unit 44 may also transmit to the server 4 a ranging signal generated by the ranging sensor within a predetermined period (e.g., several seconds) from the time the image was generated. Similarly, when transmitting a ranging signal in which a target vehicle is detected to the server 4, the transmission processing unit 44 may also transmit to the server 4 an image generated by the camera 21 within a predetermined period from the time the ranging signal was generated. Furthermore, if the host vehicle is equipped with a microphone for picking up sounds around the vehicle, when transmitting an image in which a target vehicle is detected or a ranging signal to the server 4, the transmission processing unit 44 may also transmit to the server 4 an audio signal generated by the microphone within a predetermined period from the time the image or ranging signal was generated.

[0051] As described above, those skilled in the art can make various modifications to the embodiments within the scope of the present invention. [Explanation of symbols]

[0052] 1 vehicle management system, 2 vehicle, 21 camera, 22 GPS receiver, 23 wireless communication terminal, 3 vehicle appearance information collection device, 31 communication interface, 32 memory, 33 processor, 41 determination unit, 42 detection unit, 43 calculation unit, 44 transmission processing unit, 4 server, 5 communication network, 6 wireless base station

Claims

1. a determination unit that determines whether the position of the target vehicle indicated by the vehicle position information is within a detection range of a sensor of the subject vehicle; a detection unit that detects the target vehicle from a sensor signal generated by the sensor when the position of the target vehicle is within the detection range; a calculation unit that, when a target vehicle is detected from the sensor signal, inputs a range in which the target vehicle is represented on the sensor signal into a classifier that has been trained in advance to detect at least one part of the target vehicle, and calculates, for each of the at least one part, a reliability that the part is represented in the range; a transmission processing unit that transmits the sensor signal to a server via a wireless communication terminal mounted on the vehicle when the target vehicle is detected and the reliability of any of the at least one component is less than a predetermined detection threshold; A vehicle exterior information collecting device having the same.

2. the determination unit determines a relative positional relationship between the target vehicle and the host vehicle when the position of the target vehicle is included in the detection range; the calculation unit calculates the reliability for a component visible from the sensor in a relative positional relationship between the target vehicle and the host vehicle, among the at least one component; 2. The vehicle exterior information collection device according to claim 1, wherein the transmission processing unit transmits the sensor signal to the server only when the reliability of a component visible from the sensor among the at least one component is less than the detection threshold.

3. determining whether the position of the target vehicle indicated by the vehicle position information is within the detection range of the sensor of the subject vehicle; detecting the target vehicle from a sensor signal generated by the sensor when the position of the target vehicle is within the detection range; When a target vehicle is detected from the sensor signal, a range in which the target vehicle is represented on the sensor signal is input to a classifier that has been trained in advance to detect at least one part of the target vehicle, thereby calculating a reliability that each of the at least one part is represented in the range; When the target vehicle is detected and the reliability of any of the at least one component is less than a predetermined detection threshold, the sensor signal is transmitted to a server via a wireless communication terminal mounted on the vehicle. A vehicle exterior information collection method including:

4. determining whether the position of the target vehicle indicated by the vehicle position information is within the detection range of the sensor of the subject vehicle; detecting the target vehicle from a sensor signal generated by the sensor when the position of the target vehicle is within the detection range; When a target vehicle is detected from the sensor signal, a range in which the target vehicle is represented on the sensor signal is input to a classifier that has been trained in advance to detect at least one part of the target vehicle, thereby calculating a reliability that each of the at least one part is represented in the range; When the target vehicle is detected and the reliability of any of the at least one component is less than a predetermined detection threshold, the sensor signal is transmitted to a server via a wireless communication terminal mounted on the vehicle. A computer program for collecting vehicle exterior information that causes a processor mounted on the vehicle to execute the above steps.

Images