Congestion detection system
The system addresses high costs and improves reliability in traffic jam detection by selectively using vehicle images based on confidence levels and optimizing image processing and transmission.
Patent Information
- Application Number
- JP2023083861
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-05-22
- Publication Date
- 2024-12-04
- Estimated Expiration
- 2043-05-22
AI Technical Summary
Existing traffic jam detection systems incur high communication and analysis costs due to the need for image transmission and analysis at all detected locations.
A traffic jam detection system that acquires travel data from multiple vehicles, determines the necessity of image use based on confidence levels, selects vehicles to provide images, adjusts image specifications, and integrates detection information to reduce costs and improve reliability.
The system effectively suppresses communication and analysis costs while enhancing the reliability of traffic congestion detection by optimizing image processing and transmission.
Smart Images

Figure 2024167645000001_ABST
Abstract
Description
[Technical field]
[0001] The present invention relates to the technical field of congestion detection systems. [Background technology]
[0002] As one example of this type of system, a system has been proposed in which, when a traffic jam is detected by a traffic jam detection unit installed in a vehicle, driving images are acquired from a vehicle traveling on the road on which the traffic jam is detected, and the extent of the traffic jam is detected based on the driving images (see Patent Document 1). [Prior art documents] [Patent documents]
[0003] [Patent Document 1] Patent Publication No. 2022-152560 Summary of the Invention [Problem to be solved by the invention]
[0004] The technology described in Patent Document 1 has a technical problem in that image analysis is performed on all locations where congestion is detected, which increases the communication costs associated with sending and receiving images and the analysis costs associated with image analysis.
[0005] The present invention has been made in consideration of the above problems, and has an object to propose a congestion detection system that can reduce at least one of communication costs and analysis costs. [Means for solving the problem]
[0006] A traffic congestion detection system according to one embodiment of the present invention includes a driving data acquisition unit that acquires driving data from each of a plurality of vehicles, a first detection unit that generates first traffic congestion detection information based on the acquired driving data, an image acquisition unit that acquires image information relating to a peripheral image of at least one of the plurality of vehicles, a determination unit that determines whether or not to use the image information for traffic congestion detection based on a certainty factor relating to the first traffic congestion detection information, a second detection unit that generates second traffic congestion detection information using the image information acquired by the image acquisition unit when the determination unit determines that the image information is to be used for traffic congestion detection, and a generation unit that generates third traffic congestion detection information based on the second traffic congestion detection information. [Brief description of the drawings]
[0007] [Figure 1] 1 is a diagram showing an overview of a traffic congestion detection system according to a first embodiment. [Diagram 2] FIG. 2 is a block diagram showing a configuration of a server device according to the first embodiment. [Diagram 3] 1 is a block diagram showing an example of a configuration of a vehicle according to a first embodiment. [Figure 4] 4 is a flowchart showing the operation of the traffic congestion detection system according to the first embodiment. [Diagram 5] FIG. 13 is a diagram showing an example of a result of a fitting process. [Figure 6] FIG. 2 is a diagram showing an example of an image captured by an in-vehicle camera. [Figure 7] FIG. 11 is a diagram showing an example of a change over time in the x coordinate of a vehicle shown in an image. [Figure 8] FIG. 13 is a diagram showing an example of a change over time in the x coordinate of each of a plurality of vehicles shown in an image. [Figure 9] FIG. 2 is a diagram illustrating an example of nodes and links. [Figure 10] 10 is a diagram showing an example of time-dependent change in the positions of each of a plurality of vehicles traveling on a road corresponding to the link shown in FIG. [Figure 11] FIG. 11 is a conceptual diagram for explaining an example of a vehicle selection method. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0008] An embodiment of a traffic congestion detection system will be described with reference to the drawings.
[0009] First Embodiment A first embodiment of a traffic congestion detection system will be described with reference to Fig. 1 to Fig. 11. In Fig. 1, the traffic congestion detection system 1 includes a server device 10. The traffic congestion detection system 1 may include one or more vehicles capable of communicating with the server device 10 via a wireless base station BS and a network NW. The one or more vehicles may be so-called connected cars.
[0010] The server device 10 will be described with reference to Fig. 2. In Fig. 2, the server device 10 includes a calculation device 11, a storage device 12, and a communication device 13.
[0011] The arithmetic device 11 may include, for example, at least one of a central processing unit (CPU), a graphics processing unit (GPU), a field programmable gate array (FPGA), and a tensor processing unit (TPU).
[0012] The storage device 12 may include at least one of a RAM (Random Access Memory), a ROM (Read Only Memory), a hard disk device, a magneto-optical disk device, a SSD (Solid State Drive), and an optical disk array, for example. In other words, the storage device 12 may include a non-transitory recording medium.
[0013] The storage device 12 can store desired data. For example, the storage device 12 may temporarily store a computer program executed by the computing device 11. The storage device 12 may temporarily store data that is temporarily used by the computing device 11 when the computing device 11 is executing a computer program.
[0014] The communication device 13 may be capable of communicating with an external device (for example, a vehicle) of the server device 10 via the network NW. The network NW may be a wide area network such as the Internet.
[0015] The calculation device 11 may have, as a logically realized functional block or as a physically realized processing circuit, a driving data acquisition unit 111, a driving data congestion detection unit 112, a judgment unit 113, a vehicle selection unit 114, an image specification change unit 115, a vehicle data acquisition unit 116, a vehicle data congestion detection unit 117, and a congestion information integration unit 118.
[0016] Furthermore, at least one of the driving data acquisition unit 111, the driving data congestion detection unit 112, the judgment unit 113, the vehicle selection unit 114, the image specification change unit 115, the vehicle data acquisition unit 116, the vehicle data congestion detection unit 117, and the congestion information integration unit 118 may be realized in a form that mixes logical functional blocks and physical processing circuits (i.e., hardware).
[0017] When at least a part of the traveling data acquisition unit 111, the traveling data congestion detection unit 112, the judgment unit 113, the vehicle selection unit 114, the image specification change unit 115, the vehicle data acquisition unit 116, the vehicle data congestion detection unit 117, and the congestion information integration unit 118 are functional blocks, at least a part of the traveling data acquisition unit 111, the traveling data congestion detection unit 112, the judgment unit 113, the vehicle selection unit 114, the image specification change unit 115, the vehicle data acquisition unit 116, the vehicle data congestion detection unit 117, and the congestion information integration unit 118 may be realized by the calculation device 11 executing a predetermined computer program. The calculation device 11 may acquire the above-mentioned predetermined computer program from the storage device 12.
[0018] A vehicle 20 as an example of a vehicle capable of communicating with the server device 10 will be described with reference to Fig. 3. In Fig. 3, the vehicle 20 includes an ECU (Electronic Control Unit) 21, an on-board camera 22, an on-board sensor 23, and a communication device 24.
[0019] The on-board camera 22 is disposed on the vehicle 20 so as to be able to capture an image of the periphery of the vehicle 20. For example, the on-board camera 22 may be disposed inside the passenger compartment of the vehicle 20 so as to be able to capture an image of the area in front of the vehicle 20 through the front window of the vehicle 20. The on-board camera 22 may include two or more cameras. In this case, the on-board camera 22 may include at least two cameras, a camera capable of capturing an image of the area in front of the vehicle 20, a camera capable of capturing an image of the area behind the vehicle 20, and a camera capable of capturing an image of the side of the vehicle 20.
[0020] The on-board sensor 23 is a sensor that detects the behavior of the vehicle 20. For example, the on-board sensor 23 may include a speed sensor, an acceleration sensor, a yaw rate sensor, a steering angle sensor, and a position detection sensor that detects the position of the vehicle 20.
[0021] The ECU 21 may transmit the traveling data indicating the behavior of the vehicle 20 detected by the on-board sensor 23 to the server device 10 via the communication device 24. That is, the server device 10 may acquire the traveling data from the vehicle 20. The server device 10 may store the traveling data acquired from the vehicle 20 in the storage device 12 (see "traveling data 121" in FIG. 2). The traveling data acquired from the vehicle 20, which is a moving body, may be referred to as probe data. The traveling data may include position information indicating the position of the vehicle 20 and speed information indicating the speed of the vehicle 20. The traveling data 121 in FIG. 2 may include traveling data acquired from a vehicle other than the vehicle 20.
[0022] The ECU 21 may transmit a surrounding image showing the surroundings of the vehicle 20 captured by the on-board camera 22 to the server device 10 via the communication device 24. That is, the server device 10 may acquire the surrounding image from the vehicle 20. The server device 10 may store the surrounding image acquired from the vehicle 20 in the storage device 12 (see "image data 122" in FIG. 2). Note that the image data 122 in FIG. 2 may include a surrounding image acquired from a vehicle other than the vehicle 20.
[0023] The operation of the server device 10 will be described with reference to the flowchart of FIG. 4. In the first embodiment, each of the multiple vehicles including the vehicle 20 may sequentially transmit travel data to the server device 10. As a result, the travel data of each of the multiple vehicles including the vehicle 20 may be sequentially stored in the storage device 12 of the server device 10. The server device 10 may delete travel data stored in the storage device 12 that has been acquired by the server device 10 for a first predetermined time. Each of the multiple vehicles including the vehicle 20 may sequentially transmit surrounding images to the server device 10. As a result, the storage device 12 of the server device 10 may sequentially store surrounding images of each of the multiple vehicles including the vehicle 20. The server device 10 may delete surrounding images stored in the storage device 12 that have been acquired by the server device 10 for a second predetermined time. The second predetermined time may be different from or the same as the first predetermined time.
[0024] 4, the running data acquisition unit 111 of the server device 10 acquires running data from the storage device 12 (step S101). For example, the running data acquisition unit 111 may acquire running data for a predetermined period from the running data 121 stored in the storage device 12. For example, the running data acquisition unit 111 may acquire running data for a predetermined position range from the running data 121 stored in the storage device 12. For example, the running data acquisition unit 111 may acquire running data for a predetermined period and a predetermined position range from the running data 121 stored in the storage device 12.
[0025] The traveling data traffic jam detection unit 112 of the server device 10 detects traffic jam based on at least a part of the traveling data acquired in the processing of step S101 (step S102). The traveling data traffic jam detection unit 112 generates first traffic jam information related to the detected traffic jam. Various existing aspects can be applied to the method of detecting traffic jam based on the traveling data. That is, the method of detecting traffic jam based on the traveling data may be appropriately selected according to, for example, the specifications of the product. The following method is an example of the method of detecting traffic jam based on the traveling data. The traveling data traffic jam detection unit 112 may detect traffic jam based on a change over time in the position of the vehicle indicated by the position information included in the traveling data.
[0026] The determination unit 113 of the server device 10 determines whether or not to use an image for traffic congestion detection, depending on the confidence level related to the first traffic congestion information generated by the driving data traffic congestion detection unit 112 (step S103). The determination unit 113 may determine that an image is used for traffic congestion detection when the confidence level related to the first traffic congestion information is smaller than a first threshold value. In other words, the determination unit 113 may determine that an image is not used for traffic congestion detection when the confidence level related to the first traffic congestion information is equal to or greater than a first threshold value. Specific examples of the confidence level related to the first traffic congestion information will be described later.
[0027] In the process of step S103, if it is determined that the image is not used for traffic congestion detection (step S103: No), the operation shown in Fig. 4 is terminated. In this case, the server device 10 may distribute the traffic congestion information based on the first traffic congestion information to a plurality of vehicles including the vehicle 20 via the communication device 13.
[0028] In the process of step S103, if it is determined that an image is to be used for traffic congestion detection (step S103: Yes), the vehicle selection unit 114 of the server device 10 selects a vehicle that provides an image (e.g., a surrounding image) to be used for traffic congestion detection (step S104). For example, the vehicle selection unit 114 may select the location of traffic congestion indicated by the first traffic congestion information or one or more vehicles traveling in the vicinity of the location of traffic congestion as the vehicle that provides the image.
[0029] The image specification modification unit 115 of the server device 10 adjusts (or modifies) the specifications of the image used for congestion detection based on the confidence level related to the first congestion information (step S105). The process of step S105 may be performed in parallel with or before or after the process of step S104. The image specifications may include at least one of the image size, image resolution, and video length. A specific example of a method for adjusting the image specifications based on the confidence level will be described later. For example, the image specification modification unit 115 may lower the image resolution when the confidence level related to the first congestion information is smaller than the first threshold but close to the first threshold, compared to the image resolution when the confidence level related to the first congestion information is significantly lower than the first threshold.
[0030] The image data acquisition unit 116 of the server device 10 acquires image data (e.g., surrounding images) from the storage device 12 (step S106). Specifically, the image data acquisition unit 116 acquires images acquired from one or more vehicles selected by the vehicle selection unit 114 from the image data stored in the storage device 12, with the image specifications adjusted by the image specification change unit 115.
[0031] The image data congestion detection unit 117 of the server device 10 detects congestion using at least a part of the image data acquired in the process of step S106 (step S107). The image data congestion detection unit 117 generates second congestion information related to the detected congestion. Note that various existing aspects can be applied to the method of detecting congestion using image data. In other words, the method of detecting congestion using image data may be appropriately selected depending on, for example, the specifications of a product.
[0032] The following method is an example of a method for detecting a traffic jam using image data. The image data traffic jam detection unit 117 may detect a traffic jam by performing a predetermined image processing (for example, at least one of object detection processing, tracking processing, and lane detection processing) on an image (for example, a peripheral image). The image data traffic jam detection unit 117 may detect a traffic jam using a learning model that outputs a traffic jam detection result when image data is input. Such a learning model may be constructed by deep learning, which is one aspect of machine learning. The learning model constructed by deep learning may be a mathematical model constructed by machine learning using a multi-layered neural network having a plurality of intermediate layers (which may also be called hidden layers). The neural network may be, for example, a convolutional neural network.
[0033] The determination unit 113 determines whether or not to repeat the above-mentioned process, depending on the degree of certainty associated with the second congestion information generated by the image data congestion detection unit 117 (step S108). The determination unit 113 may determine to repeat the above-mentioned process when the degree of certainty associated with the second congestion information is smaller than a second threshold value. In other words, the determination unit 113 may determine not to repeat the above-mentioned process when the degree of certainty associated with the second congestion information is equal to or greater than the second threshold value. Specific examples of the degree of certainty associated with the second congestion information will be described later.
[0034] If it is determined in the process of step S108 that the above-mentioned process is to be repeated (step S108: Yes), the process of step S101 described above is performed. On the other hand, if it is determined in the process of step S108 that the above-mentioned process is not to be repeated (step S108: No), the traffic congestion information integration unit 118 of the server device 10 integrates the first traffic congestion information and the second traffic congestion information to generate integrated traffic congestion information (step S109). The server device 10 may distribute the traffic congestion information based on the integrated traffic congestion information to a plurality of vehicles including the vehicle 20 via the communication device 13.
[0035] The traffic congestion information integration unit 118 may generate the integrated traffic congestion information by adding the information indicated by the second traffic congestion information to the information indicated by the first traffic congestion information. The traffic congestion information integration unit 118 may generate the integrated traffic congestion information by changing (in other words, replacing) at least a part of the information indicated by the first traffic congestion information to the information indicated by the second traffic congestion information. The traffic congestion information integration unit 118 may generate the integrated traffic congestion information by correcting at least a part of the information indicated by the first traffic congestion information based on the information indicated by the second traffic congestion information.
[0036] (First concrete example) As a first specific example of the operation of the traffic congestion detection system 1, an operation of detecting a traffic congestion for each lane will be described with reference to FIGS. 5 to 8 in addition to the flowchart of FIG.
[0037] 4, the travel data acquisition unit 111 of the server device 10 may acquire travel data relating to each of a plurality of vehicles traveling on a predetermined road during a predetermined period from the storage device 12. The above-mentioned predetermined road may be, for example, a road corresponding to one link in a road map represented by a combination of nodes and links.
[0038] 4, the traveling data congestion detection unit 112 may obtain a speed distribution based on the speed information included in the traveling data acquired by the traveling data acquisition unit 111. The traveling data congestion detection unit 112 may perform a fitting process using a Gaussian mixture model on the obtained speed distribution, assuming that the number of distributions is "2".
[0039] The number of distributions may vary depending on the number of lanes on the road that vehicles traveling in one direction can travel on. For example, if the number of lanes on the road that vehicles traveling in one direction can travel on is "2" (so-called a road with two lanes on each side), the number of distributions may be "2". For example, if the number of lanes on the road that vehicles traveling in one direction can travel on is "3" (so-called a road with three lanes on each side), the number of distributions may be "3". The initial value of the number of distributions may be set regardless of the number of lanes on the road that vehicles traveling in one direction can travel on.
[0040] For example, when a traffic jam occurs in one lane where vehicles moving in one direction can pass, and another lane adjacent to the one lane where vehicles moving in the one direction can pass and there is no traffic jam, the speed of the vehicles traveling in the one lane will be slower than the speed of the vehicles traveling in the other lane. In this case, when the fitting process is performed in the process of step S102, two peaks appear, for example, as shown in Fig. 5(a). In Fig. 5, the "number of vehicles" on the vertical axis may be read as "frequency."
[0041] For example, when congestion occurs in both the first lane and the other lane, it is considered that there is almost no difference between the speed of the vehicle traveling in the first lane and the speed of the vehicle traveling in the other lane. In this case, when the fitting process is performed in the process of step S102, there is a high possibility that only one peak will appear, for example, as shown in Figure 5(b). Note that the results of the fitting process shown in Figures 5(a) and 5(b) correspond to an example of the above-mentioned first congestion information.
[0042] The driving data congestion detection unit 112 may calculate the confidence level (i.e., the confidence level related to the first congestion information) based on the result of the fitting process, etc. For example, the driving data congestion detection unit 112 may calculate the confidence level by the formula "conf=w1×x1+w2×x2+w3×x3+···+wn×xn". In the above formula, "conf" represents the confidence level, "w1,w2,w3,···,wn" represents weights, and "x1,x2,x3,···,xn" represents parameters.
[0043] The above parameters may include, for example, at least one of (1) the number of vehicles, (2) the ratio of the number of vehicles in one lane to the number of vehicles in another lane, (3) the fitting score of a Gaussian mixture model, (4) the average speed value of the distribution on the lower speed side among the multiple distributions, and (5) the difference between the average speeds of each of the multiple distributions. Note that the above formula is an example, and the confidence factor may be calculated by a method other than the above formula.
[0044] Research by the present inventors has revealed that in the operation of detecting congestion for each lane, the confidence level tends to decrease in at least one of the following cases: (1) when the number of vehicles traveling on a given road is small or there is a bias in the number of vehicles per lane, (2) when the difference between the average speed of the congestion lane and the average speed of the non-congested lanes is relatively small, (3) when vehicles in the congestion lane are moving at a certain speed (in other words, when vehicles in the congestion lane are not repeatedly stopping and starting), (4) when congestion is temporarily alleviated in the congestion lane, and (5) when the number of lanes available for vehicles traveling in one direction is three or more.
[0045] 4, if it is determined that an image is to be used for traffic congestion detection (step S103: Yes), the processes from step S104 onward are performed. In the process of step S104, the vehicle selection unit 114 may select, for example, one vehicle traveling at a relatively high speed among a plurality of vehicles traveling on a predetermined road during a predetermined period of time as the vehicle providing the image.
[0046] A vehicle traveling at a relatively high speed travels a relatively long distance in a certain period (speed is the amount of change in the position of the vehicle per unit time). For this reason, the surrounding image (e.g., video) captured by the vehicle's on-board camera during the period has higher spatial coverage than the surrounding image (e.g., video) captured by the vehicle's on-board camera of a vehicle traveling at a relatively low speed during the period. In other words, the range of space captured by the vehicle's on-board camera of a vehicle traveling at a relatively high speed during a certain period is wider than the range of space captured by the vehicle's on-board camera of a vehicle traveling at a relatively low speed. In other words, when capturing the same range of space, the amount of data related to the surrounding image (e.g., video) of a vehicle traveling at a relatively high speed is smaller than the amount of data related to the surrounding image (e.g., video) of a vehicle traveling at a relatively low speed. Therefore, if the surrounding image of one vehicle traveling at a relatively high speed is used, the load related to image processing in the image data congestion detection unit 117 can be reduced. For example, for this reason, the vehicle selection unit 114 may select one vehicle traveling at a relatively high speed from among a plurality of vehicles traveling on a predetermined road during a predetermined period.
[0047] In the process of step S104, the vehicle selection unit 114 may select, for example, one vehicle that is traveling at a relatively high speed and that has been on the predetermined road for a relatively long time during the predetermined time period among a plurality of vehicles traveling on a predetermined road during the predetermined time period. With this configuration, it is highly likely that the image (e.g., video) around the one vehicle includes a relatively large number of locations where congestion occurs, and it is expected that congestion will be appropriately detected in the process of step S107 in FIG.
[0048] 4, the image specification change unit 115 adjusts (or changes) the specification of the image used for traffic congestion detection based on the confidence level (i.e., the confidence level related to the first traffic congestion information) calculated by the driving data traffic congestion detection unit 112. For example, the image specification change unit 115 may adjust the image specification using the formula "l=max(σ(conf)×l_max,l_min)".
[0049] In the above formula, "l" represents the adjusted video length, "conf" represents the confidence level, "l_max" represents the video length before adjustment, "l_min" represents the minimum value of the video length, and "σ" represents an activation function such as an identity function or a sigmoid function. Here, a case has been described in which the image specification modification unit 115 adjusts the video length, but the image specification modification unit 115 may adjust at least one of the image size and the resolution in addition to or instead of the video length.
[0050] In the processing of step S106 in FIG. 4, the image data acquisition unit 116 acquires an image acquired from one vehicle selected by the vehicle selection unit 114 from the image data stored in the storage device 12, with the image specifications adjusted by the image specification modification unit 115.
[0051] In the process of step S107 in Fig. 4, the image data congestion detection unit 117 detects congestion using at least a part of the image data acquired by the image data acquisition unit 116. Here, an example of a method of congestion detection by the image data congestion detection unit 117 will be described with reference to Figs. 6 to 8.
[0052] FIG. 6(a) shows an example of an image captured by an on-board camera at time t1. FIG. 6(b) shows an example of an image captured by an on-board camera at time t2 after time t1. The first vehicle equipped with the on-board camera that captured the images shown in FIG. 6(a) and FIG. 6(b) is assumed to be traveling in the center lane shown in the images. The vehicle surrounded by the dotted rectangle in the image shown in FIG. 6(a) and the vehicle surrounded by the dotted rectangle in the image shown in FIG. 6(b) are assumed to be the same vehicle. The vehicle surrounded by the dotted rectangle will hereinafter be referred to as the "second vehicle" as appropriate.
[0053] The speed of the first vehicle is assumed to be higher than the speed of the second vehicle. In this case, as shown in FIG. 6(a) and FIG. 6(b), the distance between the first vehicle and the second vehicle decreases with time. Here, the horizontal direction of the image captured by the vehicle-mounted camera is taken as the x-axis. Also, the x-coordinate of the horizontal center of the image is taken as "x=0". Then, as is clear from FIG. 6(a) and FIG. 6(b), the value of the x-coordinate of the center point CP of the dotted rectangle increases with time. The time change in the x-coordinate of the center point CP can be represented as a graph, for example, as shown in FIG. 7.
[0054] 6(a) and 6(b), the second vehicle is in the lane to the right of the lane in which the first vehicle is traveling. If a third vehicle traveling at a speed slower than that of the first vehicle is in the lane to the left of the lane in which the first vehicle is traveling, the x-coordinate value of the center point of the dotted rectangle surrounding the third vehicle decreases over time.
[0055] For example, the image data congestion detection unit 117 may perform a predetermined image processing on the image data acquired by the image data acquisition unit 116. For example, the image data congestion detection unit 117 may obtain a time change in the x-coordinate of a point corresponding to the center point CP for each of a plurality of vehicles detected from the image data acquired by the image data acquisition unit 116. The image data congestion detection unit 117 may detect, as a congested section, a road section in which a predetermined number or more of vehicles exist within a third predetermined time period in which the time change in the x-coordinate of the above-mentioned point monotonically increases or decreases (for example, see FIG. 8).
[0056] Incidentally, the result of the image processing, for example, shown in FIG. 8, corresponds to an example of the second congestion information described above. The image data congestion detection unit 117 may calculate the confidence level (i.e., the confidence level related to the second congestion information) based on the result of the image processing, etc. For example, the image data congestion detection unit 117 may calculate the confidence level by the formula "conf=w1×x1+w2×x2+w3×x3+···+wn×xn" like the traveling data congestion detection unit 112. Incidentally, at least a part of the parameters (e.g., x1, x2, x3,···, xn) may be different from the parameters when the traveling data congestion detection unit 112 calculates the confidence level. When the image data congestion detection unit 117 uses a learning model that outputs a congestion detection result when image data is input, the confidence level output from the learning model may be used as the confidence level related to the second congestion information.
[0057] 4, if it is determined that the process is to be repeated from step S101 (step S108: Yes), for example, the traveling data congestion detection unit 112 may adjust the parameters related to the Gaussian mixture model based on the result of the process of step S107. Specifically, if the number of lanes is detected in the process of step S107, the traveling data congestion detection unit 112 may set the number of distributions as the parameters related to the Gaussian mixture model to be the same as the number of lanes detected.
[0058] (Second concrete example) As a second specific example of the operation of the traffic congestion detection system 1, an operation of detecting the end of a traffic congestion will be described with reference to Figures 9 to 11 in addition to the flowchart of Figure 4. Note that, regarding the second specific example, explanations that overlap with the above-mentioned first specific example will be omitted as appropriate.
[0059] In the process of step S101 in Fig. 4, the travel data acquisition unit 111 of the server device 10 may acquire travel data related to each of a plurality of vehicles traveling on a predetermined road during a predetermined period from the storage device 12. For example, the above-mentioned predetermined road may be a road corresponding to link L (see Fig. 9) connecting node N1 and node N2 in a road map represented by a combination of nodes and links.
[0060] 4, the travel data congestion detection unit 112 may extract, for example, a section in which the speed of each vehicle traveling from node N2 to node N1 is equal to or lower than a threshold speed, based on the speed information included in the travel data acquired by the travel data acquisition unit 111. The travel data congestion detection unit 112 may generate, for example, a time-distance diagram shown in FIG. 10, based on the extracted section.
[0061] In Fig. 10, the solid line segments represent the change in the position of each of the multiple vehicles over time. In Fig. 10, the node N1 is the starting point of the distance, and the distance increases as the distance from the node N1 toward the node N2 increases.
[0062] For example, the traveling data congestion detection unit 112 may detect the end of each of the multiple line segments in FIG. 10 on the side with a shorter distance as the beginning of the congestion. The traveling data congestion detection unit 112 may detect the end of each of the multiple line segments in FIG. 10 on the side with a longer distance as the end of the congestion. However, there is not always a sufficient number of vehicles that can communicate with the server device 10 on the specified road. For this reason, the traveling data congestion detection unit 112 may estimate the beginning and end of the congestion based on the time-distance diagram, for example, using a Kalman filter or the like. The beginning and end of the congestion detected or estimated by the traveling data congestion detection unit 112 correspond to another example of the above-mentioned first congestion information.
[0063] The driving data congestion detection unit 112 may calculate the confidence level (i.e., the confidence level related to the first congestion information) based on the detection result of the end of the congestion. For example, the driving data congestion detection unit 112 may calculate the confidence level by the formula "conf=w1×x1+w2×x2+w3×x3+···+wn×xn". Note that at least a part of the parameters (e.g., x1, x2, x3,···, xn) may be different from the parameters when the driving data congestion detection unit 112 calculates the confidence level in the first specific example described above.
[0064] Research by the present inventors has revealed that in the operation of detecting the end of a traffic jam, the confidence level tends to decrease in at least one of the following cases: (1) when the vehicle speed fluctuates around a threshold speed and the line segment is interrupted on the time-distance diagram, (2) when there is a vehicle at the end of the traffic jam that cannot communicate with server device 10, (3) when there is a period when there are no vehicles that can communicate with server device 10, and (4) when the position of the end of the traffic jam differs for each lane and there is a relatively large variation in the positions of the ends of the line segments on the time-distance diagram.
[0065] If it is determined in the process of step S103 in FIG. 4 that an image is to be used for traffic congestion detection (step S103: Yes), the processes from step S104 onward are performed. In the process of step S104, the vehicle selection unit 114 may select one vehicle by a method described below with reference to FIG. 11. In FIG. 11, the solid line segments represent the change in the position of each of the multiple vehicles over time (including the portion where the vehicle speed is higher than the above-mentioned threshold speed). In FIG. 11, the node N1 is the starting point of the distance, and the distance increases as the distance from node N1 to node N2 increases. In FIG. 11, each of the multiple black circles represents the position of a vehicle (i.e., a vehicle that can communicate with the server device 10) at time t3.
[0066] The vehicle selection unit 114 may calculate the average speed and variance of two or more vehicles included in the window W while changing the position of the window W (see "scan" in FIG. 11) along the direction in which the distance axis (i.e., vertical axis) extends in the time-distance diagram shown in FIG. 11, for example. When the window W includes only two or more vehicles traveling in the congestion section, the calculated average speed is relatively low (e.g., below the above-mentioned threshold speed) and the variance is relatively small. When the window W includes one or more vehicles traveling in the congestion section and one or more vehicles traveling in a location that is not a congestion section, the calculated average speed is relatively high and the variance is relatively large.
[0067] The vehicle selection unit 114 may identify the position of the window W where the calculated average speed and variance of two or more vehicles included in the window W are equal to or greater than the average speed threshold and variance threshold, respectively. The vehicle selection unit 114 may select one vehicle (i.e., the vehicle providing the image) based on the position of the identified window W.
[0068] 4, the image specification change unit 115 adjusts (or changes) the specification of the image used for traffic congestion detection based on the confidence level (i.e., the confidence level related to the first traffic congestion information) calculated by the driving data traffic congestion detection unit 112. The image specification change unit 115 may adjust the image specification by the formula "l=max(σ(conf)×l_max,l_min)" as in the first specific example described above.
[0069] In the process of step S107 in Fig. 4, the image data congestion detection unit 117 detects congestion using at least a part of the image data acquired by the image data acquisition unit 116. As in the first specific example described above, the image data congestion detection unit 117 may detect a congestion section based on a change over time (see Figs. 7 and 8) in the x-coordinate of the center point (see Fig. 6) of a dotted rectangle surrounding a vehicle shown in an image. The image data congestion detection unit 117 may detect the end of a congestion section based on the detected congestion section.
[0070] 4, if it is determined that the process is to be repeated from step S101 (step S108: Yes), for example, the driving data congestion detection unit 112 may adjust parameters related to the detection process of the end of the congestion based on the result of the process of step S107. For example, when the end of the congestion is detected by applying DBSCAN (Density-Based Spatial Clustering of Applications with Noise) to each line segment in the time-distance diagram shown in FIG. 10, the driving data congestion detection unit 112 may adjust the data point radius of DBSCAN based on the result of the process of step S107.
[0071] (Technical effect) In the traffic congestion detection system 1 according to the first embodiment, the vehicle selection unit 114 selects a vehicle that provides an image (e.g., a surrounding image) used for traffic congestion detection. Then, the image data traffic congestion detection unit 117 detects traffic congestion using an image acquired from the vehicle selected by the vehicle selection unit 114. That is, in the traffic congestion detection system 1 according to the first embodiment, only images acquired from the vehicle selected by the vehicle selection unit 114 are subject to processing by the image data traffic congestion detection unit 117. Therefore, according to the traffic congestion detection system 1 according to the first embodiment, it is possible to reduce analysis costs related to image analysis for traffic congestion detection.
[0072] In addition, in the traffic congestion detection system 1 according to the first embodiment, the image specification change unit 115 changes the image specification based on the confidence level related to the first traffic congestion information. For example, the image specification change unit 115 may lower the image resolution when the confidence level related to the first traffic congestion information is smaller than the first threshold value but close to the first threshold value, compared to the image resolution when the confidence level related to the first traffic congestion information is significantly lower than the first threshold value. With this configuration, it is possible to improve the reliability of traffic congestion detection while suppressing the analysis cost related to image analysis for traffic congestion detection.
[0073] In the first embodiment, each of the vehicles including the vehicle 20 may transmit a peripheral image to the server device 10 only when a predetermined event occurs (for example, when the speed becomes lower than a predetermined speed, when the hazard lamps are turned on, etc.). With this configuration, it is possible to reduce communication costs related to transmission and reception of images used for traffic congestion detection.
[0074] <Second embodiment> A second embodiment of the traffic congestion detection system will be described. In the second embodiment, the configuration and operation of the traffic congestion detection system 1 may be the same as those of the first embodiment described above, except for some differences in the operation of the traffic congestion detection system 1. Therefore, the description of the second embodiment that overlaps with the first embodiment will be omitted as appropriate.
[0075] In the second embodiment, each of the multiple vehicles including the vehicle 20 may transmit a peripheral image to the server device 10 when a request for transmitting a peripheral image is received from the server device 10. For example, when a request for transmitting a peripheral image is received from the server device 10 via the communication device 24, the ECU 21 of the vehicle 20 may transmit a peripheral image showing the periphery of the vehicle 20 captured by the in-vehicle camera 22 to the server device 10 via the communication device 24.
[0076] The operation of the server device 10 according to the second embodiment will be described with reference to the flowchart of Fig. 4. If it is determined in the process of step S103 in Fig. 4 that an image is to be used for congestion detection (step S103: Yes), the vehicle selection unit 114 selects a vehicle that provides an image (e.g., a surrounding image) to be used for congestion detection (step S104). The image specification change unit 115 adjusts (or changes) the specification of the image to be used for congestion detection based on the confidence level related to the first congestion information (step S105).
[0077] After processing of steps S104 and S105, the calculation device 11 of the server device 10 may send a transmission request via the communication device 13 to one or more vehicles selected by the vehicle selection unit 114, requesting the transmission of surrounding images with the image specifications adjusted by the image specification modification unit 115.
[0078] A vehicle that receives a transmission request from the server device 10 (i.e., a vehicle selected by the vehicle selection unit 114) transmits a surrounding image with image specifications adjusted by the image specification change unit 115 to the server device 10. The server device 10 may temporarily store the surrounding image transmitted from the vehicle in the storage device 12.
[0079] After that, in the process of step S106 in FIG. 4, the image data acquisition unit 116 acquires image data from the storage device 12 (that is, the surrounding image transmitted from the vehicle).
[0080] (Technical effect) According to the traffic congestion detection system 1 of the second embodiment, similarly to the above-described first embodiment, it is possible to improve the reliability of traffic congestion detection while suppressing the analysis cost related to image analysis for traffic congestion detection. As described above, in the traffic congestion detection system 1 of the second embodiment, a vehicle (e.g., vehicle 20) transmits a surrounding image to the server device 10 when a transmission request is received from the server device 10. Therefore, according to the traffic congestion detection system 1 of the second embodiment, it is possible to suppress the communication cost related to the transmission and reception of images used for traffic congestion detection.
[0081] <Third embodiment> A third embodiment of the traffic congestion detection system will be described. In the third embodiment, the configuration and operation of the traffic congestion detection system 1 may be the same as those of the first embodiment described above, except for some differences in the operation of the traffic congestion detection system 1. Therefore, the description of the third embodiment that overlaps with the first embodiment will be omitted as appropriate.
[0082] In the third embodiment, each of a plurality of vehicles including the vehicle 20 has functions corresponding to the image data acquisition unit 116 and the image data congestion detection unit 117 in the above-described first embodiment. For example, the ECU 21 of the vehicle 20 may have functions corresponding to the image data acquisition unit 116 and the image data congestion detection unit 117 in the above-described first embodiment. Therefore, the calculation device 11 of the server device 10 according to the third embodiment does not need to have the image data acquisition unit 116 and the image data congestion detection unit 117.
[0083] The operation of the traffic congestion detection system 1 according to the third embodiment will be described with reference to the flowchart of Fig. 4. If it is determined in the process of step S103 in Fig. 4 that an image is to be used for traffic congestion detection (step S103: Yes), the vehicle selection unit 114 selects a vehicle that provides an image (e.g., a surrounding image) to be used for traffic congestion detection (step S104). The image specification change unit 115 adjusts (or changes) the specification of the image to be used for traffic congestion detection based on the confidence level related to the first traffic congestion information (step S105).
[0084] After processing of steps S104 and S105, the calculation device 11 of the server device 10 may transmit image specification information indicating the image specifications adjusted by the image specification modification unit 115 to one or more vehicles selected by the vehicle selection unit 114 via the communication device 13.
[0085] 4, a vehicle that has received the image specification information from the server device 10 (i.e., a vehicle selected by the vehicle selection unit 114) acquires a surrounding image (in other words, image data) captured by the vehicle's on-board camera in accordance with the image specification indicated by the image specification information. For example, when the vehicle that has received the image specification information is the vehicle 20, the ECU 21 may acquire a surrounding image captured by the on-board camera 22 in accordance with the image specification indicated by the image specification information.
[0086] In the process of step S107 in Fig. 4, the vehicle that has received the image specification information from the server device 10 detects a traffic jam using at least a part of the surrounding image (in other words, image data) acquired in the process of step S106. The vehicle generates second traffic jam information related to the detected traffic jam. For example, when the vehicle that has received the image specification information is the vehicle 20, the ECU 21 may detect a traffic jam using at least a part of the surrounding image acquired in the process of step S106. The ECU 21 may generate second traffic jam information related to the detected traffic jam.
[0087] After the process of step S107, the vehicle transmits the second congestion information to the server device 10. In this case, the vehicle may transmit information indicating a degree of certainty related to the second congestion information to the server device 10, in addition to the second congestion information.
[0088] (Technical effect) According to the traffic congestion detection system 1 of the third embodiment, similarly to the above-described first embodiment, it is possible to improve the reliability of traffic congestion detection while suppressing the analysis cost related to image analysis for traffic congestion detection. In the traffic congestion detection system 1 of the third embodiment, as described above, the second traffic congestion information is generated in the vehicle (e.g., vehicle 20). Therefore, in the traffic congestion detection system 1 of the third embodiment, it is not necessary to transmit the image used for traffic congestion detection from the vehicle (e.g., vehicle 20) to the server device 10. Therefore, according to the traffic congestion detection system 1 of the third embodiment, it is possible to reduce the communication cost related to the transmission and reception of the image used for traffic congestion detection.
[0089] Various aspects of the invention derived from the above-described embodiment will be described below.
[0090] A traffic congestion detection system according to one embodiment of the invention includes a driving data acquisition unit that acquires driving data from each of a plurality of vehicles, a first detection unit that generates first traffic congestion detection information based on at least a portion of the acquired driving data, an image acquisition unit that acquires image information relating to a surrounding image of at least one of the plurality of vehicles, a determination unit that determines whether or not to use the image information for traffic congestion detection based on a certainty factor relating to the first traffic congestion detection information, a second detection unit that generates second traffic congestion detection information using the image information acquired by the image acquisition unit when the determination unit determines that the image information is to be used for traffic congestion detection, and a generation unit that generates third traffic congestion detection information based on the second traffic congestion detection information.
[0091] In the above-described embodiment, the "driving data acquisition unit 111" corresponds to an example of a "driving data acquisition unit", the "driving data traffic congestion detection unit 112" corresponds to an example of a "first detection unit", the "image data acquisition unit 116" corresponds to an example of an "image acquisition unit", the "judgment unit 113" corresponds to an example of a "judgment unit", the "image data traffic congestion detection unit 117" corresponds to an example of a "second detection unit", the "traffic congestion information integration unit 118" corresponds to an example of a "generation unit", and the "ECU 21" corresponds to another example of the "image acquisition unit" and "second detection unit".
[0092] In the traffic congestion detection system, the second detection unit may generate the second traffic congestion detection information by using image information whose image specifications have been changed according to a certainty factor related to the first traffic congestion detection information.
[0093] In the traffic congestion detection system, the second detection unit may generate the second traffic congestion detection information by using image information selected according to a degree of certainty related to the first traffic congestion detection information.
[0094] In the traffic congestion detection system, the generation unit may generate the third traffic congestion detection information based on the first traffic congestion detection information and the second traffic congestion detection information.
[0095] In the traffic congestion detection system, the image acquisition unit may acquire image information relating to an image of a periphery of the at least one vehicle when the determination unit determines that image information is to be used for traffic congestion detection.
[0096] The traffic congestion detection system may include a change unit that changes an image specification related to the image information according to a certainty level related to the first traffic congestion detection information, and the image acquisition unit may acquire image information related to a peripheral image of the at least one vehicle with the image specification changed by the change unit. In the above-mentioned embodiment, the "image specification change unit 115" corresponds to an example of the "change unit".
[0097] The traffic congestion detection system may include a selection unit that selects at least one vehicle from the plurality of vehicles according to a certainty factor related to the first traffic congestion detection information, and the image acquisition unit may acquire image information related to a peripheral image of the at least one vehicle selected by the selection unit. In the above-described embodiment, the "vehicle selection unit 114" corresponds to an example of the "selection unit."
[0098] The present invention is not limited to the above-described embodiments, but can be modified as appropriate within the scope that does not contradict the gist or concept of the invention that can be read from the claims and the entire specification, and congestion detection systems involving such modifications are also included in the technical scope of the present invention. [Explanation of symbols]
[0099] 1...traffic congestion detection system, 10...server device, 11...arithmetic device, 12...storage device, 13, 24...communication device, 20...vehicle, 21...ECU, 22...vehicle-mounted camera, 23...vehicle-mounted sensor, 111...driving data acquisition unit, 112...driving data congestion detection unit, 113...judgment unit, 114...vehicle selection unit, 115...image specification change unit, 116...image data acquisition unit, 117...image data congestion detection unit, 118...traffic congestion information integration unit
Claims
1. A travel data acquisition unit that acquires travel data from each of a plurality of vehicles; A first detection unit that generates first congestion detection information based on at least a part of the acquired running data; an image acquisition unit that acquires image information relating to a surrounding image of at least one vehicle of the plurality of vehicles; a determination unit that determines whether or not image information is to be used for traffic congestion detection according to a certainty factor related to the first traffic congestion detection information; a second detection unit that generates second congestion detection information using the image information acquired by the image acquisition unit when the determination unit determines that the image information is to be used for congestion detection; A generation unit that generates third congestion detection information based on the second congestion detection information; A traffic congestion detection system comprising:
2. The second detection unit generates the second congestion detection information using image information whose image specifications have been changed according to a certainty factor related to the first congestion detection information. The congestion detection system according to claim 1 .
3. The second detection unit generates the second congestion detection information using image information selected according to a certainty factor related to the first congestion detection information. The congestion detection system according to claim 1 .
4. The generation unit generates the third congestion detection information based on the first congestion detection information and the second congestion detection information. The congestion detection system according to claim 1 .
5. The image acquisition unit acquires image information relating to an image of a periphery of the at least one vehicle when the determination unit determines that image information is to be used for traffic congestion detection. The congestion detection system according to claim 1 .
6. A change unit changes an image specification related to the image information according to a certainty degree related to the first congestion detection information, The image acquisition unit acquires image information related to a surrounding image of the at least one vehicle with the image specifications changed by the change unit. The congestion detection system according to claim 1 .
7. A selection unit that selects at least one vehicle from the plurality of vehicles according to a certainty factor related to the first congestion detection information, The image acquisition unit acquires image information related to a surrounding image of at least one vehicle selected by the selection unit. The congestion detection system according to claim 1 .
Citation Information
Patent Citations
Road monitoring system
JP2003242594A
Traffic condition determination system and traffic condition determination device
JP2020004235A
Traffic information processor
JP2020107080A
Information processing device
JP2022152560A
Cited By
Traffic congestion prediction device and traffic congestion prediction method
JP7936964B1
