Data collection device, data collection method, computer program for data collection and data collection instruction device

The data collection device adapts feature detection based on snow conditions, using trained classifiers to ensure continuous data collection and accurate map updates during snowy weather.

JP2024164457AActive Publication Date: 2024-11-27WOVEN BY TOYOTA INC

Patent Information

Application Number
JP2023079942
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-05-15
Publication Date
2024-11-27
Estimated Expiration
2043-05-15

AI Technical Summary

Technical Problem

Existing data collection systems struggle to accurately gather road feature information when there is snow on the road, leading to incomplete map updates during snowy conditions.

Method used

A data collection device equipped with a determination unit to detect snow, setting units to adjust feature detection types based on snow presence, and detection units using trained classifiers to identify either three-dimensional structures or road surface features, ensuring data collection continues during snowy conditions.

Benefits of technology

Enables the collection of data representing road features even when snow is present, preventing prolonged gaps in map updates and maintaining data accuracy.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024164457000001_ABST
    Figure 2024164457000001_ABST
Patent Text Reader

Abstract

To provide a data collection device capable of collecting data representing a feature even when there is snow accumulation on a road.SOLUTION: A data collection device comprises: a determination unit 42 which determines whether there is snow accumulation in the periphery of a vehicle 2; a setting unit 43 which sets the type of a feature being a detection object on the basis of the determination result regarding the presence / absence of snow accumulation; and a detection unit 44 which detects the feature of the set type from an image representing the periphery of the vehicle 2 generated by a camera 21 mounted on the vehicle 2, and generates probe data representing the detected feature.SELECTED DRAWING: Figure 4
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to a data collection device, a data collection method, a computer program for data collection, and a data collection instruction device that collect data used to generate or update a map. [Background technology]

[0002] A high-precision map that an automatic driving system of a vehicle refers to in order to control the automatic driving of the vehicle is required to accurately represent information about roads. Therefore, data representing roads or features around the roads in a specific area is collected from a vehicle that has actually traveled in the specific area, and the data is obtained by a sensor mounted on the vehicle. However, depending on the weather around the vehicle, it can be difficult to detect such features from the sensor signal. Therefore, a technology has been proposed that refers to weather information when collecting data (see Patent Document 1).

[0003] In the technology disclosed in Patent Document 1, a vehicle-side device for a road map update system sequentially judges whether or not the road on which the vehicle is traveling is a new road not included in the road map data. Then, when the vehicle-side device judges that the road on which the vehicle is traveling is a new road, and the vehicle-side device judges based on weather information that the image captured by the vehicle-mounted camera can be used to update the center-side road map data, the vehicle-side device transmits the image of the surroundings of the vehicle captured by the vehicle-mounted camera to the map management device. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2008-39687 A Summary of the Invention [Problem to be solved by the invention]

[0005] In some regions, snow may continue to accumulate on roads for long periods of time. In such cases, the above-mentioned technology does not collect images while the snow remains, and map information is not updated. Therefore, it is desirable to be able to collect data representing features that can be used to generate or update map information, even when there is snow on the road.

[0006] SUMMARY OF THE PRESENT EMBODIMENTS An object of the present invention is to provide a data collection device that can collect data representing features even when there is snow on the road. [Means for solving the problem]

[0007] According to one embodiment, there is provided a data collection device including a determination unit that determines whether or not there is snow around a vehicle, a setting unit that sets a type of feature to be detected based on the result of the determination of the presence or absence of snow, and a detection unit that detects the set type of feature from an image showing the surroundings of the vehicle generated by a camera mounted on the vehicle, and generates probe data representing the detected feature.

[0008] In addition, in this data collection device, when it is determined that there is snow around the vehicle, the setting unit sets the road or a three-dimensional structure located around the road as the type of feature to be detected, and on the other hand, when it is determined that there is no snow around the vehicle, it is preferable to set a predetermined feature including a road surface structure formed along the road surface or the ground around the road as the type of feature to be detected.

[0009] In this case, when it is determined that there is snow around the vehicle, the detection unit preferably detects a three-dimensional structure by inputting the image into a first classifier that has been pre-trained to detect three-dimensional structures from an image, and when it is determined that there is no snow around the vehicle, it preferably detects a specified feature by inputting the image into a second classifier that has been pre-trained to detect a specified feature from an image.

[0010] According to another aspect of the present invention, there is provided a data collection instruction device, the data collection instruction device having a determination unit which determines whether or not there is snowfall in a predetermined area, and a notification processing unit which, if there is snowfall, issues a collection instruction to a snow removal vehicle via a communication device to instruct the collection of probe data showing a predetermined type of feature in the predetermined area, and, if there is no snowfall, issues the collection instruction to a vehicle other than the snow removal vehicle via the communication device.

[0011] According to yet another aspect of the present invention, there is provided a data collection method including: determining whether or not there is snow around a vehicle, setting a type of feature to be detected based on the result of the determination of the presence or absence of snow, detecting the set type of feature from an image showing the surroundings of the vehicle generated by a camera mounted on the vehicle, and generating probe data representing the detected feature.

[0012] According to yet another aspect of the present invention, there is provided a computer program for collecting data, the computer program including instructions for causing a processor mounted on a vehicle to execute the following steps: determine whether or not there is snow around a vehicle, set a type of feature to be detected based on a result of the determination of the presence or absence of snow, detect the set type of feature from an image showing the surroundings of the vehicle generated by a camera mounted on the vehicle, and generate probe data representing the detected feature. Effect of the Invention

[0013] The data collection device according to the present disclosure has an advantage in that it is possible to collect data representing features even when there is snow on the road. [Brief description of the drawings]

[0014] [Figure 1] 1 is a schematic diagram of a data collection system in which a data collection device is implemented; [Diagram 2] FIG. 1 is a schematic configuration diagram of a vehicle. [Diagram 3] FIG. 2 is a hardware configuration diagram of a data collection device according to one embodiment. [Figure 4] FIG. 2 is a functional block diagram of a processor of the data collection device. [Diagram 5] 1A and 1B are diagrams showing examples of types of features to be detected when there is snow and when there is no snow, respectively. [Figure 6] 13 is an operation flowchart of data collection processing. [Figure 7] FIG. 2 is a hardware configuration diagram of a server which is an example of a data collection instruction device. [Figure 8] FIG. 2 is a functional block diagram of a processor of the server. [Figure 9] 13 is an operational flowchart of a process related to a data collection instruction. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0015] Hereinafter, a data collection device, a data collection method and a computer program for data collection executed by the data collection device, and a data collection instruction device will be described with reference to the drawings. The data collection device is mounted on a vehicle. The data collection device generates data (hereinafter referred to as probe data) representing specific features on or around a road, which is used for generating or updating a map, based on an image showing the surroundings of the vehicle. In this case, the data collection device determines whether there is snow around the vehicle, and sets the type of feature to be detected according to the determination result. In particular, when there is snow, the data collection device sets the type of feature that can be detected even when there is snow as the feature to be detected.

[0016] FIG. 1 is a schematic diagram of a data collection system in which a data collection device is implemented. In this embodiment, the data collection system 1 has a data collection device 3 mounted on at least one vehicle 2, and a server 4. The at least one vehicle 2 may be a snowplow vehicle, or may be a vehicle other than a snowplow vehicle, such as a regular passenger car, a bus, or a truck. The data collection device 3 is connected to the server 4 via the wireless base station 6 and the communication network 5 by accessing the wireless base station 6 connected to the communication network 5 to which the server 4 is connected via a gateway (not shown), for example. Note that in FIG. 1, one snowplow vehicle 2a and one regular passenger car 2b are shown as the at least one vehicle 2, but the data collection system 1 may have multiple snowplow vehicles mounted with the data collection device 3. Similarly, the data collection system 1 may have multiple vehicles other than snowplow vehicles mounted with the data collection device 3. In addition, as in a modified example described later, when the server 4 issues a collection instruction to each vehicle 2 uniformly regardless of whether there is snow or not, the data collection system 1 may have only snow removal vehicles or only vehicles other than snow removal vehicles. Furthermore, a plurality of wireless base stations 6 may be connected to the communication network 5.

[0017] First, the vehicle 2 and the data collection device 3 will be described. As described above, the data collection system 1 may include a plurality of vehicles 2 equipped with the data collection device 3, and the vehicles 2 may be either snow removal vehicles 2a or vehicles other than snow removal vehicles 2b. However, since each vehicle 2 and each data collection device 3 has the same configuration and executes the same process with respect to data collection processing, the following will describe one vehicle 2 and one data collection device 3.

[0018] 2 is a schematic diagram of the vehicle 2. The vehicle 2 has a camera 21 for capturing images of the surroundings of the vehicle 2, a GPS receiver 22, a wireless communication terminal 23, and a data collection device 3. The camera 21, the GPS receiver 22, the wireless communication terminal 23, and the data collection device 3 are communicatively connected via an in-vehicle network conforming to a standard such as a controller area network. The vehicle 2 may further have a distance measurement sensor (not shown) such as a LiDAR sensor for measuring distances to objects around the vehicle 2.

[0019] The camera 21 is an example of an imaging unit, and has a two-dimensional detector configured with an array of photoelectric conversion elements sensitive to visible light, such as a CCD or C-MOS, and an imaging optical system that forms an image of the area to be photographed on the two-dimensional detector. The camera 21 is attached, for example, in the cabin of the vehicle 2 so as to face the front of the vehicle 2. The camera 21 photographs the area in front of the vehicle 2 at a predetermined photographing period (for example, 1 / 30 to 1 / 10 seconds) and generates an image showing the area in front. The image obtained by the camera 21 may be a color image or a gray image. The photographing direction of the camera 21 is not limited to the forward direction of the vehicle 2. In particular, the camera 21 mounted on the snow removal vehicle 2a may be attached so as to photograph the area behind the snow removal vehicle 2a. As a result, an image showing the road surface after snow removal is generated. Furthermore, the vehicle 2 may be provided with a plurality of cameras 21 with different photographing directions or focal lengths.

[0020] Every time the camera 21 generates an image, it outputs the generated image to the data collection device 3 via the in-vehicle network.

[0021] The GPS receiver 22 receives GPS signals from GPS satellites at predetermined intervals and determines the own position of the vehicle 2 based on the received GPS signals. Then, the GPS receiver 22 outputs positioning information representing the positioning result of the own position of the vehicle 2 based on the GPS signals to the data collection device 3 via the in-vehicle network at predetermined intervals. Note that the vehicle 2 may have a receiver that complies with a satellite positioning system other than the GPS receiver 22. In this case, it is sufficient that the receiver determines the own position of the vehicle 2.

[0022] The wireless communication terminal 23 is an example of a communication device, and is a device that executes wireless communication processing conforming to a predetermined wireless communication standard. For example, by accessing the wireless base station 6, the wireless communication terminal 23 is connected to the server 4 via the wireless base station 6 and the communication network 5. That is, a communication line is established between the wireless communication terminal 23 and the server 4 via the wireless base station 6 and the communication network 5. Then, the wireless communication terminal 23 receives downlink wireless signals including a collection instruction signal or a signal indicating a collection target area from the server 4, and outputs the received signals to the data collection device 3. Also, the wireless communication terminal 23 generates uplink wireless signals including a specified type of collection target data (for example, probe data or image) received from the data collection device 3. Then, the wireless communication terminal 23 transmits the uplink wireless signals to the wireless base station 6, thereby transmitting the collection target data to the server 4.

[0023] 3 is a hardware configuration diagram of the data collection device 3. The data collection device 3 temporarily stores images received from the camera 21. Furthermore, the data collection device 3 generates probe data based on the images and the like, and temporarily stores the generated probe data. Then, the data collection device 3 transmits the probe data and images to the server 4 via the wireless communication terminal 23. To this end, the data collection device 3 has a communication interface 31, a memory 32, and a processor 33.

[0024] The communication interface 31 is an example of an in-vehicle communication unit, and has an interface circuit for connecting the data collection device 3 to an in-vehicle network. That is, the communication interface 31 is connected to the camera 21, the GPS receiver 22, and the wireless communication terminal 23 via the in-vehicle network. Then, the communication interface 31 passes the received image to the processor 33 every time it receives an image from the camera 21. Also, the communication interface 31 passes the received positioning information to the processor 33 every time it receives positioning information from the GPS receiver 22. Furthermore, the communication interface 31 passes the information from the server 4, such as a collection instruction signal, to the processor 33 every time it receives information from the wireless communication terminal 23. Furthermore, the communication interface 31 outputs the collection target data and the like received from the processor 33 to the wireless communication terminal 23 via the in-vehicle network.

[0025] The memory 32 is an example of a storage unit, and includes, for example, a volatile semiconductor memory and a non-volatile semiconductor memory. The memory 32 may further include another storage device such as a hard disk device. The memory 32 stores various data used in processing related to data collection executed by the processor 33 of the data collection device 3. For example, the memory 32 stores the identification information of the vehicle 2, and parameters of the camera 21 such as the focal length, shooting direction, and installation position of the camera 21. Furthermore, the memory 32 stores various parameters for specifying a classifier for determining the presence or absence of snowfall, and various parameters for specifying a classifier for detecting features from the image received from the camera 21. The memory 32 also stores positioning information received from the GPS receiver 22. Furthermore, the memory 32 stores information representing the collection target area received from the server 4. Furthermore, the memory 32 may store computer programs for implementing each processing executed by the processor 33.

[0026] Furthermore, the memory 32 is provided with a storage area for storing images (hereinafter referred to as the image storage area) and a storage area for storing probe data (hereinafter referred to as the probe storage area). Images received by the data collection device 3 from the camera 21 are temporarily stored in the image storage area. Furthermore, the probe data generated by the processor 33 is temporarily stored in the probe storage area.

[0027] The processor 33 has one or more central processing units (CPUs) and their peripheral circuits. The processor 33 may further have other arithmetic circuits such as a logic arithmetic unit, a numerical arithmetic unit, or a graphic processing unit. The processor 33 executes data collection processing while the vehicle 2 is traveling.

[0028] 4 is a functional block diagram of the processor 33 of the data collection device 3. The processor 33 has an image storage unit 41, a determination unit 42, a setting unit 43, a detection unit 44, and a communication processing unit 45. Each of these units in the processor 33 is, for example, a functional module realized by a computer program running on the processor 33. Alternatively, each of these units in the processor 33 may be a dedicated arithmetic circuit provided in the processor 33.

[0029] The image storage unit 41 writes the image into the image storage area of ​​the memory 32 every time the data collection device 3 receives an image from the camera 21. At that time, every time the image storage unit 41 receives an image from the camera 21, it associates the position of the vehicle 2 represented by the latest positioning information and the orientation of the vehicle 2 indicated by an orientation sensor (not shown) with the image. Furthermore, the image storage unit 41 may also associate parameters of the camera 21, such as the focal length, installation position, and shooting direction of the camera 21, with the image. When the image is uploaded to the server 4, this information associated with the image is uploaded to the server 4 together with the image itself.

[0030] The determination unit 42 determines whether or not there is snow around the vehicle 2. In this embodiment, the determination unit 42 inputs the latest image received from the camera 21 to a determiner that has been trained in advance to be able to distinguish between areas covered with snow and other areas. Then, when the size of the area covered with snow, which is indicated by the output result of the determiner, is equal to or larger than a predetermined snow accumulation determination threshold, the determination unit 42 determines that there is snow around the vehicle 2. The determination unit 42 can use a deep neural network (DNN) for semantic segmentation, such as a Fully Convolutional Network or U-Net, as such a determiner.

[0031] Alternatively, the determination unit 42 may determine whether or not there is snow around the vehicle 2 based on weather information received from a server (not shown) for distributing weather information via the wireless communication terminal 23. In this case, the determination unit 42 determines whether or not the current position of the vehicle 2 indicated by the latest positioning information received from the GPS receiver 22 is included in an area with snow, which is indicated by the weather information. Then, the determination unit 42 may determine that there is snow around the vehicle 2 if the current position of the vehicle 2 is included in an area with snow.

[0032] In the following, a state in which it is determined that there is snow around the vehicle 2 may be simply referred to as there is snow or when there is snow, and a state in which it is determined that there is no snow around the vehicle 2 may be simply referred to as there is no snow or when there is no snow. The determination unit 42 notifies the setting unit 43 of the determination result as to whether there is snow or not.

[0033] The setting unit 43 sets the type of feature to be detected based on the result of the determination of the presence or absence of snowfall notified by the determination unit 42. In this embodiment, when it is determined that there is snowfall around the vehicle 2, the setting unit 43 sets a three-dimensional structure located on the road or around the road as the type of feature to be detected. Such three-dimensional structures include, for example, poles or arrow signs indicating the position of the road shoulder, road signs, signs, and traffic lights. This is because such three-dimensional structures are highly likely to be visible without being covered by snow even if the road surface around the vehicle 2 is covered with snow. Note that the surface of the road sign may be covered with snow, so it is preferable that the setting unit 43 does not detect the display content of the road sign, but rather the road sign itself. Similarly, it is preferable that the setting unit 43 detects the sign itself, not the display content of the sign. Note that the positions of the road signs, the signs, and the traffic lights are represented on a so-called high-precision map or a dynamic map so that they can be used, for example, in a localization process to detect the position of the vehicle. Furthermore, the three-dimensional structures to be detected may include a wall of snow formed as a result of snow removal. However, since the position of the snow wall may change, it is preferable that the position of the snow wall is used for updating the dynamic map but not for updating the high precision map. Note that the high precision map is a map that includes information used to identify the vehicle position at the lane level, such as lane markings. In addition to such information, the dynamic map is a map that includes semi-dynamic or semi-static information such as traffic information and dynamic information such as information on surrounding vehicles.

[0034] On the other hand, when it is determined that there is no snow around the vehicle 2, the setting unit 43 sets predetermined features including road surface structures formed along the road surface or the ground around the road as the type of features to be detected. Such road surface structures include road markings such as lane markings, stop lines, and pedestrian crossings, and curbs. Note that the predetermined features to be detected when it is determined that there is no snow around the vehicle 2 may include three-dimensional structures such as poles and road signs that are detected when there is snow.

[0035] 5(a) and 5(b) are diagrams showing examples of types of features to be detected when there is snow and when there is no snow, respectively. In the example shown in FIG. 5(a), the presence of snow is detected in an image 500 showing the surroundings of a vehicle 2. The features on the road surface are not visible due to the snow. Therefore, three-dimensional structures such as a pole 501 and a traffic light 502 are set as features to be detected.

[0036] 5(b), no snow is detected and features on the road surface are visible in the image 510 showing the surroundings of the vehicle 2. Therefore, road markings such as lane markings 511 are set as features to be detected.

[0037] The setting unit 43 notifies the detection unit 44 of the set type of feature to be detected.

[0038] While the vehicle 2 is traveling, the detection unit 44 detects features of the type set by the setting unit 43 from the latest image generated by the camera 21 at predetermined intervals or each time the vehicle 2 travels a predetermined distance. The detection unit 44 then generates probe data indicating the type of feature detected on the image and the position of the feature.

[0039] For example, the detection unit 44 detects the type of feature to be detected that is shown in the input image by inputting the image to a classifier. The detection unit 44 may use a DNN having a convolutional neural network (CNN) type architecture, such as Single Shot MultiBox Detector or Faster R-CNN, as such a classifier. Alternatively, the detection unit 44 may use a DNN having a self attention network (SAN) type architecture, such as Vision Transformer, as such a classifier. Alternatively, the detection unit 44 may use a classifier based on other machine learning methods, such as an AdaBoost classifier, as such a classifier. Such a classifier is trained in advance according to a predetermined learning method, such as backpropagation, using a large number of teacher images in which the feature is shown, so as to detect the type of feature to be detected from the image. The classifier then outputs information representing an area (for example, a circumscribing rectangle of the feature to be detected, hereinafter referred to as an object area) in which the type of feature to be detected is included on the input image, and information representing the type of feature shown in the object area.

[0040] In this embodiment, a first classifier to be used when it is determined that there is snow around the vehicle 2 and a second classifier to be used when it is determined that there is no snow around the vehicle 2 are separately prepared. That is, the first classifier is trained in advance to detect from an image a type of feature to be detected when there is snow. In contrast, the second classifier is trained in advance to detect from an image a type of feature to be detected when there is no snow. The first classifier and the second classifier may be based on different architectures or may be based on the same architecture. Then, when the setting unit 43 notifies the detection unit 44 of the type of feature when it is determined that there is snow around the vehicle 2, the detection unit 44 inputs the image to the first classifier. As a result, a type of feature to be detected when there is snow is detected from the image. On the other hand, when the setting unit 43 notifies the detection unit 44 of the type of feature when it is determined that there is no snow around the vehicle 2, the detection unit 44 inputs the image to the second classifier. As a result, a type of feature to be detected when there is no snow is detected from the image. In this way, by preparing separate classifiers for when there is snow and when there is no snow, the types of features to be detected for each classifier can be limited, allowing the detection unit 44 to accurately detect the types of features to be detected.

[0041] Alternatively, a common classifier may be used when it is determined that there is snow around the vehicle 2 and when it is determined that there is no snow around the vehicle 2. In this case, the classifier is trained in advance to detect not only the types of features that are the detection target when snow is present, but also the types of features that are the detection target when there is no snow from the image. The detection unit 44 inputs the image to the classifier regardless of whether there is snow or there is no snow. However, when there is snow, the detection unit 44 may ignore the detection results output from the classifier of the types of features other than the types of features that are the detection target when there is no snow among the detection results output from the classifier. Conversely, when there is no snow, the detection unit 44 may ignore the detection results of the types of features other than the types of features that are the detection target when there is no snow among the detection results output from the classifier. In this way, by using a common classifier when there is snow and when there is no snow, the required hardware resources can be reduced.

[0042] The detection unit 44 estimates the position of the feature shown in the object region detected from the image based on parameters such as the direction from the camera 21, the position of the vehicle 2 at the time of image generation, the traveling direction, the shooting direction of the camera 21, the focal length, and the installation position, which correspond to the center of gravity of the object region. At this time, the detection unit 44 may estimate the position of the feature by so-called Structure from Motion (SfM). In this case, the detection unit 44 uses optical flow to associate object regions in which the same feature is shown between two images obtained at different times. Then, the detection unit 44 may estimate the position of the feature by triangulation based on the position and traveling direction of the vehicle 2 at the time when each of the two images was obtained, the parameters of the camera 21, and the position of the object region in each image. Then, the detection unit 44 generates probe data including information indicating the type and estimated position of the detected feature. The detection unit 44 may further include information indicating the position and traveling direction of the vehicle 2 at the time of image generation in the probe data. The detection unit 44 may further include information indicating the size of the object region and its position on the image in the probe data. The detection unit 44 generates one piece of probe data for each detected feature. Therefore, when multiple features are detected from one image, multiple pieces of probe data are generated from the one image.

[0043] Every time the detection unit 44 generates probe data, it writes the generated probe data into the probe storage area of ​​the memory 32 .

[0044] The communication processing unit 45 transmits the collection target data stored in the memory 32 to the server 4 via the wireless communication terminal 23.

[0045] The communication processing unit 45 refers to the information representing the collection target area and the latest positioning information to determine whether the vehicle 2 has entered the collection target area. When the vehicle 2 enters the collection target area, the communication processing unit 45 transmits the collection target data for the collection target area designated by the collection instruction signal to the server 4 via the wireless communication terminal 23.

[0046] The collection target area is specified, for example, by one or more continuous road sections, or by an area having a predetermined shape. When the collection target area is specified as a road section, the information representing the collection target area includes identification information for identifying the road section and information representing the positions of the ends at which the road section can be entered or exited. When the collection target area is an area having a predetermined shape, the information representing the collection target area includes information representing the positions of the outer edges of the area. When the position of vehicle 2 represented by the latest positioning information is included in the area or road section indicated by the information representing the collection target area, the communication processing unit 45 determines that vehicle 2 has entered the collection target area.

[0047] When the probe data is specified as data to be collected, the communication processing unit 45 transmits the probe data stored in the probe storage area of ​​the memory 32 to the server 4 via the wireless communication terminal 23 in order from the oldest to the newest.

[0048] In addition, when probe data and images are specified as data to be collected, the communication processing unit 45 transmits the probe data stored in the probe storage area of ​​the memory 32 and the images stored in the image storage area to the server 4 via the wireless communication terminal 23, in order from oldest to newest.

[0049] Furthermore, the images designated as the data to be collected may be partial images. In this case, the communication processing unit 45 generates partial images by cutting out an area that is estimated to represent the road surface from each image stored in the image storage area. The communication processing unit 45 then transmits the generated partial images, together with the probe data stored in the probe storage area of ​​the memory 32, to the server 4 via the wireless communication terminal 23 in order from the oldest to the newest.

[0050] In addition, the communication processing unit 45 erases from the memory 32 the collection target data that was transmitted to the server 4 via the wireless communication terminal 23 .

[0051] When the vehicle 2 exits the collection target area, the communication processing unit 45 refers to the memory 32 to identify collection target data that has not been transmitted at the time of the exit. Then, the communication processing unit 45 ends the transmission of the collection target data when the transmission of the identified collection target data to the server 4 is completed. Note that, similar to the determination of whether the vehicle 2 has entered the collection target area, the communication processing unit 45 may determine whether the vehicle 2 has exited the collection target area by referring to the information representing the collection target area and the latest positioning information.

[0052] 6 is an operational flowchart of the data collection process, particularly the process related to generation of probe data. The processor 33 executes the data collection process in accordance with the following operational flowchart at predetermined intervals or each time the vehicle 2 travels a predetermined distance.

[0053] The image storage unit 41 of the processor 33 writes the image received by the data collection device 3 from the camera 21 into the image storage area of ​​the memory 32 (step S101). In addition, the determination unit 42 of the processor 33 determines whether or not there is snow around the vehicle 2 (step S102).

[0054] If there is snow around the vehicle 2 (step S102-Yes), the setting unit 43 of the processor 33 sets a three-dimensional structure located on or around the road as the type of feature to be detected (step S103). On the other hand, if there is no snow around the vehicle 2 (step S102-No), the setting unit 43 sets a predetermined feature including a road surface structure formed along the road surface or the ground around the road as the type of feature to be detected (step S104).

[0055] After step S103 or S104, the detection unit 44 of the processor 33 detects the set type of feature from the image generated by the camera 21, and generates probe data representing the detected feature (step S105). The detection unit 44 then writes the generated probe data into the probe storage area of ​​the memory 32 (step S106). The processor 33 then ends the process related to the generation of the probe data.

[0056] Since it is expected that the presence or absence of snow accumulation does not change locally, the processor 33 may execute the processes of steps S102 to S104 only once before and after the vehicle 2 enters the collection target area.

[0057] As described above, the data collection device sets the type of feature to be detected depending on whether there is snow around the vehicle. In particular, when there is snow, the data collection device sets the type of feature that can be detected even when the road surface is covered with snow and is used for generating or updating a map as the type of feature to be detected. Therefore, the data collection device can collect probe data even when there is snow. Therefore, the data collection device can prevent probe data from being not collected for a long period of time.

[0058] According to a modified example, the setting unit 43 may set the type of feature to be detected for each region on the image. That is, for a region determined to have snowfall as indicated by the output result of the determiner of the determining unit 42, the setting unit 43 sets a three-dimensional structure as the type of feature to be detected as in the above embodiment. In contrast, for a region determined to have no snowfall as indicated by the output result of the determiner, a predetermined feature including a road surface structure is set as the type of feature to be detected. Then, the detection unit 44 may detect the type of feature set for each region. That is, the detection unit 44 may generate probe data only for features included in the region determined to have snowfall among the features detected by inputting the image to the first classifier. Similarly, the detection unit 44 may generate probe data only for features included in the region determined to have no snowfall among the features detected by inputting the image to the second classifier.

[0059] According to this modification, even when the area around the vehicle includes both snow-removed and unremoved areas, the data collection device can appropriately detect features.

[0060] Next, the server 4 will be described. The server 4 is an example of a data collection instruction device, and selects the type of vehicle to which the collection of the collection target data is instructed according to the presence or absence of snowfall. Furthermore, the server 4 specifies the type of data to be collected for the collection target area. The server 4 then notifies the selected type of vehicle of a collection instruction signal that indicates the specified type of data as collection target data and instructs the collection of the collection target data. Furthermore, the server 4 stores the probe data or images transmitted from the data collection device 3 mounted on each vehicle 2 that notified the collection instruction signal. The server 4 then generates or updates a map of the collection target area based on the probe data or images. Furthermore, the server 4 notifies each vehicle 2 of the collection target area.

[0061] 7 is a hardware configuration diagram of the server 4. The server 4 has a communication interface 51, a storage device 52, a memory 53, and a processor 54. The communication interface 51, the storage device 52, and the memory 53 are connected to the processor 54 via signal lines. The server 4 may further have input devices such as a keyboard and a mouse, and a display device such as a liquid crystal display.

[0062] The communication interface 51 is an example of a communication unit, and has an interface circuit for connecting the server 4 to the communication network 5. The communication interface 51 is configured to be able to communicate with the data collection device 3 mounted on each vehicle 2 via the communication network 5 and the wireless base station 6. That is, the communication interface 51 passes the collection target data and the like received from the data collection device 3 of each vehicle 2 via the wireless base station 6 and the communication network 5 to the processor 54. The communication interface 51 also transmits a collection instruction signal and the like received from the processor 54 to the data collection device 3 of each vehicle 2 via the communication network 5 and the wireless base station 6.

[0063] The storage device 52 is an example of a storage unit, and includes, for example, a hard disk device or an optical recording medium and an access device therefor. The storage device 52 stores type designation information and, for each of a plurality of road sections, collection target data collected for that road section. Furthermore, the storage device 52 stores vehicle type information and identification information indicating the type of each vehicle 2 (whether it is a snow removal vehicle or another vehicle). Furthermore, the storage device 52 may store a computer program for executing a data collection process on the server 4 side, which is executed on the processor 54. Furthermore, the storage device 52 may store a map that is generated or updated based on the collection target data.

[0064] The memory 53 is another example of a storage unit, and includes, for example, a non-volatile semiconductor memory and a volatile semiconductor memory. The memory 53 temporarily stores various data generated during the execution of the data collection process, and various data acquired through communication with each vehicle 2, such as probe data and images.

[0065] The processor 54 is an example of a control unit, and includes one or more CPUs (Central Processing Units) and their peripheral circuits. The processor 54 may further include other arithmetic circuits such as a logical arithmetic unit or a numerical arithmetic unit. The processor 54 executes the data collection process on the server 4 side.

[0066] 8 is a functional block diagram of the processor 54 of the server 4. The processor 54 has a determination unit 61 and a notification processing unit 62. Each of these units in the processor 54 is, for example, a functional module realized by a computer program running on the processor 54. Alternatively, each of these units in the processor 54 may be a dedicated arithmetic circuit provided in the processor 54.

[0067] The determination unit 61 determines whether or not there is snow in the collection target area. For example, the determination unit 61 determines whether or not there is snow in the collection target area based on weather information received from a server (not shown) for distributing weather information via the communication interface 51. To this end, the determination unit 61 compares the collection target area with an area with snow indicated by the weather information. If a predetermined percentage or more of the collection target area is included in the area with snow indicated by the weather information, the determination unit 61 may determine that there is snow in the collection target area. Alternatively, if a reference point (for example, the center or center of gravity of the collection target area) set in the collection target area is included in the area with snow indicated by the weather information, the determination unit 61 may determine that there is snow in the collection target area.

[0068] Furthermore, when the server 4 receives an image generated within the collection target area from any vehicle 2 within a recent predetermined period (for example, several hours to one day), the determination unit 61 may determine the presence or absence of snow based on the image. In this case, the determination unit 61 may determine the presence or absence of snow based on a determination result obtained by inputting the image into a determiner for determining the presence or absence of snow, similar to the determination unit 42 of the data collection device 3. Then, when the determination unit 61 determines that there is snow based on the image, it determines that there is snow in the collection target area.

[0069] The determination unit 61 notifies the notification processing unit 62 of the determination result as to whether or not there is snow in the collection target area.

[0070] The notification processing unit 62 selects the type of vehicle to which the collection instruction is to be sent based on the result of determining whether there is snow in the collection target area. Specifically, when there is snow in the collection target area, the notification processing unit 62 selects a snowplow vehicle as the type of vehicle to which the collection instruction is to be sent. On the other hand, when there is no snow in the collection target area, the notification processing unit 62 selects a vehicle other than a snowplow vehicle as the type of vehicle to which the collection instruction is to be sent. Then, the notification processing unit 62 refers to the vehicle type information and the identification information to identify the selected type of vehicle from among the vehicles 2.

[0071] The notification processing unit 62 refers to type designation information corresponding to the collection target area. Then, the notification processing unit 62 identifies the type of collection target data designated for the collection target area based on the type designation information. The type of collection target data includes an image representing the environment around the vehicle 2 generated by the camera 21 mounted on the vehicle 2, a partial image representing the road surface cut out from the image, or probe data representing features detected from the image. Therefore, the type designation information indicates, for each collection target area, the type of collection target data, which is either an image, a partial image, or probe data.

[0072] The notification processing unit 62 includes in the instruction signal information for specifying the type of the identified data to be collected. If the type designation information indicates probe data, the notification processing unit 62 specifies only the probe data as the data to be collected. If the type designation information indicates an image, the notification processing unit 62 specifies the probe data and the image as the data to be collected. If the type designation information indicates a partial image, the processor 54 specifies the probe data and the partial image as the data to be collected.

[0073] The notification processing unit 62 transmits the generated collection instruction signal to each vehicle 2 of the selected type via the communication interface 51, the communication network 5 and the wireless base station 6.

[0074] When a plurality of collection target areas are set, the processor 54 may execute the process of the determination unit 61 and the process of the notification processing unit 62 for each collection target area.

[0075] The processor 54 may set the collection target area based on the number of probe data collected for each road section stored in the storage device 52. For example, the processor 54 may set the road section for which the number of probe data collected in the most recent predetermined period (for example, several weeks to several months) is less than a predetermined collection threshold as the collection target area. Alternatively, the processor 54 may set the road section for which the elapsed time since the previous update is equal to or greater than a predetermined elapsed time threshold, among the individual road sections shown on the map to be updated, as the collection target area. The processor 54 may then notify the data collection device 3 of each vehicle 2 of the set collection target area via the communication interface 51.

[0076] Furthermore, the processor 54 may generate or update a map using the collected probe data and images. The map to be generated or updated is a map used for automatic driving control of a vehicle, and may be either a high-precision map or a dynamic map. The processor 54 aligns features represented in each of the probe data collected from one vehicle 2 for a continuous section with corresponding features represented in other probe data collected in the past or represented in the map to be updated. The processor 54 then identifies, as a newly installed feature, any feature that does not have a corresponding feature in other probe data or the map to be updated among the features represented in each of the probe data collected for the continuous section. The processor 54 then adds information (position and type) about the identified feature to the map to be generated or updated. The processor 54 may also identify, as removed, any feature that does not have a corresponding feature in the features represented in each of the probe data collected for the continuous section among the features represented in the map to be updated. The processor 54 may then delete information about the identified removed feature from the map to be updated.

[0077] 9 is an operational flowchart of processing related to a data collection instruction. At a predetermined timing (for example, a predetermined date and time), the processor 54 of the server 4 executes processing related to a data collection instruction according to the following operational flowchart.

[0078] The determination unit 61 of the processor 54 determines whether or not there is snow in the collection target area (step S201). If there is snow in the collection target area (step S201-Yes), the notification processing unit 62 of the processor 54 selects a snowplow as the type of vehicle to notify the collection instruction (step S202). On the other hand, if there is no snow in the collection target area (step S201-No), the notification processing unit 62 selects a vehicle other than a snowplow as the type of vehicle to notify the collection instruction (step S203).

[0079] After step S202 or S203, the notification processing unit 62 notifies the selected vehicle of a collection instruction signal via the communication interface 51, the communication network 5, and the wireless base station 6 (step S204). Then, the processor 54 executes processing related to the data collection instruction.

[0080] As described above, the data collection instruction device selects the type of vehicle to instruct collection of collection target data including probe data depending on the presence or absence of snowfall in the collection target area. In particular, when there is snowfall in the collection target area, the data collection instruction device selects a snowplow vehicle as the vehicle to instruct collection of collection target data. Therefore, the data collection instruction device can generate and collect probe data in a state where the road surface is visible to some extent even when snowfall is present. Therefore, the data collection instruction device can suppress a decrease in accuracy of features represented in the probe data even when snowfall is present, and can also cause the data collection device to collect probe data about features represented on the road surface, such as road markings. In addition, when there is no snowfall, the data collection instruction device causes vehicles other than the snowplow vehicle, which does not travel much except when snow is being removed, to generate and collect probe data, so that the data collection instruction device can appropriately collect probe data even when there is no snowfall.

[0081] According to a modified example, the server 4 may uniformly notify each vehicle 2 of a collection instruction regardless of the presence or absence of snow in the collection target area. Also, when the server 4 designates a vehicle to which data collection is instructed as described above, the processing of the determination unit 42 and the setting unit 43 may be omitted by assuming that the data collection device mounted on each vehicle collects probe data representing the type of features to be collected when there is no snow.

[0082] As described above, those skilled in the art can make various modifications to the embodiments within the scope of the present invention. [Explanation of symbols]

[0083] 1. Data collection system 2 Vehicles 2a Snow removal vehicle 2b Passenger car 21 Camera 22 GPS receiver 23 Wireless communication terminals 3 Data collection equipment 31 Communication Interface 32 Memory 33 Processors 41 Image storage section 42 Judgment section 43 Setting section 44 Detection section 45 Communication processing section 4 Server 51 Communication Interface 52 Storage Device 53 Memory 54 processors 61 Judgment section 62 Notification processing section 5. Communication Network 6 Radio base stations

Claims

1. A determination unit that determines whether or not there is snow around the vehicle; a setting unit that sets the type of feature to be detected based on the result of the determination of the presence or absence of snow; a detection unit that detects the set types of features from an image showing the surroundings of the vehicle generated by a camera mounted on the vehicle, and generates probe data showing the detected features; A data collection device having:

2. 2. The data collection device according to claim 1, wherein, when it is determined that there is snow around the vehicle, the setting unit sets a road or a three-dimensional structure located around the road as the type of feature to be detected, and, when it is determined that there is no snow around the vehicle, the setting unit sets a predetermined feature including a road surface structure formed along the road surface or the ground around the road as the type of feature to be detected.

3. The detection unit is When it is determined that snow is present around the vehicle, the image is input to a first classifier that has been trained in advance to detect the three-dimensional structure from the image, thereby detecting the three-dimensional structure; The data collection device according to claim 2, wherein when it is determined that there is no snow around the vehicle, the specified feature is detected by inputting the image into a second classifier that has been pre-trained to detect the specified feature from the image.

4. A determination unit that determines whether or not snow is present in a predetermined area; a notification processing unit that, when there is snowfall, sends a collection instruction to a snow removal vehicle via a communication device to instruct the snow removal vehicle to collect probe data showing a predetermined type of feature in the predetermined area, and, when there is no snowfall, sends the collection instruction to vehicles other than the snow removal vehicle via the communication device; A data collection instruction device having the following:

5. Determine whether there is snow around the vehicle, setting a type of feature to be detected based on the result of the determination of the presence or absence of snow; Detecting the set types of features from an image representing the surroundings of the vehicle generated by a camera mounted on the vehicle; generating probe data representative of the detected features; Data collection methods, including:

6. Determine whether there is snow around the vehicle, setting a type of feature to be detected based on the result of the determination of the presence or absence of snow; Detecting the set types of features from an image representing the surroundings of the vehicle generated by a camera mounted on the vehicle; generating probe data representative of the detected features; A data collection computer program that causes a processor mounted on the vehicle to execute the above steps.

Citation Information

Patent Citations

  • Road map generation system and road map generation method

    CN111480182A

  • Road condition recognizing device

    JP1996263784A

  • Road condition observation system

    JP2000346959A

  • Road snow-removal navigation system, road network data processing method and road network data processing program

    JP2002333325A

  • Image recognition device and method thereof, and own vehicle position recognition device and method

    JP2007147564A

Cited By

  • Cutoff control apparatus

    US20240120733A1