Congestion detection system

By determining whether to use image information based on the confidence of the congestion detection information in the congestion detection system and adjusting the image specifications, the problem of high image analysis cost in the prior art is solved, and cost reduction and detection reliability are improved.

CN119007429APending Publication Date: 2024-11-22TOYOTA JIDOSHA KK
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410396683.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2023-05-22
Filing Date
2024-04-03
Publication Date
2024-11-22

AI Technical Summary

Technical Problem

When existing congestion detection systems detect congestion, they need to perform image analysis on all detected parts, resulting in increased communication costs and analysis costs.

Method used

By acquiring the driving data of a plurality of vehicles and peripheral image information in the server device, whether to use image information for detection is determined based on the confidence of the congestion detection information, and the image specification is adjusted when using image information to reduce analysis cost.

Benefits of technology

It effectively suppresses image analysis and communication costs in congestion detection and improves detection reliability.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119007429A_ABST
    Figure CN119007429A_ABST
Patent Text Reader

Abstract

The invention relates to a congestion detection system. A congestion detection system is provided with a server device having a processor that executes commands for: acquiring travel data from each of a plurality of vehicles; generating first congestion detection information on the basis of at least a portion of the acquired travel data; acquiring image information of a peripheral image of at least one of the plurality of vehicles; determining whether the image information is used in the congestion detection according to the confidence of the first congestion detection information; generating second congestion detection information using the image information acquired by the image acquisition unit when the determination unit determines that the image information is used in congestion detection; and generating third congestion detection information on the basis of the second congestion detection information.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of congestion detection systems. Background Art

[0002] As such a system, for example, a system has been proposed in which, when congestion is detected by a congestion detection unit mounted on a vehicle, a driving image is acquired from a vehicle traveling on a road where the congestion is detected, and a range of the congestion is detected based on the driving image (see Japanese Patent Application Laid-Open No. 2022-152560). Summary of the invention

[0003] In the technology described in Japanese Patent Application Laid-Open No. 2022-152560, since image analysis is performed on all locations where congestion is detected, there is a technical problem in that the communication cost for sending and receiving images and the analysis cost for image analysis increase.

[0004] The present disclosure proposes a congestion detection system capable of suppressing at least one of communication cost and analysis cost.

[0005] A congestion detection system according to one embodiment of the present disclosure includes a server device, wherein the server device has a processor, and the processor executes the following commands: respectively acquiring driving data from a plurality of vehicles; generating first congestion detection information based on the acquired driving data; acquiring image information of a peripheral image of at least one of the plurality of vehicles; determining whether to use the image information in congestion detection based on the confidence of the first congestion detection information; generating second congestion detection information using the image information acquired by the image acquisition unit when the determination unit determines that the image information is to be used in congestion detection; and generating third congestion detection information based on the second congestion detection information. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] Features, advantages, and technical and industrial significance of exemplary embodiments of the present invention will be described below with reference to the accompanying drawings, in which like reference numerals represent like elements, and in which:

[0007] Figure 1 This is a diagram showing an overview of the congestion detection system according to the first embodiment.

[0008] Figure 2 This is a block diagram showing the configuration of a server device according to the first embodiment.

[0009] Figure 3 This is a block diagram showing an example of the configuration of a vehicle according to the first embodiment.

[0010] Figure 4 This is a flowchart showing the operation of the congestion detection system according to the first embodiment.

[0011] Figure 5A It is a diagram showing an example of the result of the fitting process.

[0012] Figure 5B It is a diagram showing an example of the result of the fitting process.

[0013] Fig. 6A This is a diagram showing an example of an image captured by a vehicle-mounted camera.

[0014] Figure 6B This is a diagram showing an example of an image captured by a vehicle-mounted camera.

[0015] Figure 7 This is a diagram showing an example of the temporal change of the x-coordinate of the vehicle shown in the image.

[0016] Figure 8 This is a diagram showing an example of temporal changes in the x-coordinates of each of a plurality of vehicles shown in an image.

[0017] Fig. 9 It is a graph showing an example of nodes and links.

[0018] Fig.10 It means that in the equivalent Fig. 9 FIG. 1 is a diagram showing an example of temporal changes in the positions of a plurality of vehicles traveling on a road of a link shown.

[0019] Fig.11 This is a schematic diagram for explaining an example of a method for selecting a vehicle. DETAILED DESCRIPTION

[0020] Embodiments of the congestion detection system will be described based on the drawings.

[0021] <First Embodiment>

[0022] Reference Figures 1 to 11 A first embodiment of the congestion detection system is described. Figure 1 In the example, the congestion detection system 1 includes a server device 10. The congestion detection system 1 may include one or more vehicles that can communicate with the server device 10 via a wireless base station BS and a network NW. The one or more vehicles may be so-called networked cars.

[0023] Reference Figure 2 The server device 10 is described. Figure 2 In the embodiment, the server device 10 includes a computing device 11 , a storage device 12 , and a communication device 13 .

[0024] The computing device 11 may include, for example, at least one of a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an FPGA (Field Programmable Gate Array), and a TPU (Tensor Processing Unit).

[0025] The storage device 12 may include, for example, at least one of a RAM (Random Access Memory), a ROM (Read Only Memory), a hard disk device, a magneto-optical device, an SSD (Solid State Drive), and an optical disk array. That is, the storage device 12 may include a non-temporary recording medium.

[0026] The storage device 12 can store desired data. For example, the storage device 12 can temporarily store a computer program executed by the computing device 11. The storage device 12 can temporarily store data temporarily used by the computing device 11 when the computing device 11 executes the computer program.

[0027] The communication device 13 can communicate with an external device (for example, a vehicle) of the server device 10 via the network NW. It should be noted that the network NW may be, for example, a wide area network such as the Internet.

[0028] As a logically implemented functional block or a physically implemented processing circuit, the computing device 11 may have a driving data acquisition unit 111, a driving data congestion detection unit 112, a determination unit 113, a vehicle selection unit 114, an image specification change unit 115, a vehicle data acquisition unit 116, a vehicle data congestion detection unit 117 and a congestion information integration unit 118.

[0029] It should be noted that at least one of the driving data acquisition unit 111, the driving data congestion detection unit 112, the determination unit 113, the vehicle selection unit 114, the image specification change unit 115, the vehicle data acquisition unit 116, the vehicle data congestion detection unit 117 and the congestion information integration unit 118 can be implemented in the form of a mixture of logical function blocks and physical processing circuits (i.e., hardware).

[0030] When at least a part of the driving data acquisition unit 111, the driving data congestion detection unit 112, the determination unit 113, the vehicle selection unit 114, the image specification change unit 115, the vehicle data acquisition unit 116, the vehicle data congestion detection unit 117, and the congestion information integration unit 118 is a functional block, at least a part of the driving data acquisition unit 111, the driving data congestion detection unit 112, the determination unit 113, the vehicle selection unit 114, the image specification change unit 115, the vehicle data acquisition unit 116, the vehicle data congestion detection unit 117, and the congestion information integration unit 118 can be realized by the computing device 11 executing a predetermined computer program. The computing device 11 can acquire the above-mentioned predetermined computer program from the storage device 12.

[0031] Reference Figure 3 , a vehicle 20 is described as an example of a vehicle that can communicate with the server device 10. Figure 3 In the figure, a vehicle 20 includes an ECU (Electronic Control Unit) 21 , a vehicle-mounted camera 22 , a vehicle-mounted sensor 23 , and a communication device 24 .

[0032] The vehicle-mounted camera 22 is arranged on the vehicle 20 in such a manner that it can capture the surroundings of the vehicle 20. For example, the vehicle-mounted camera 22 may be arranged in the vehicle compartment of the vehicle 20 in such a manner that it can capture the front of the vehicle 20 through the front window of the vehicle 20. It should be noted that the vehicle-mounted camera 22 may include more than two cameras. In this case, the vehicle-mounted camera 22 may also include at least two cameras among a camera that can capture the front of the vehicle 20, a camera that can capture the rear of the vehicle 20, and a camera that can capture the side of the vehicle 20.

[0033] The on-vehicle sensor 23 is a sensor that detects the behavior of the vehicle 20. For example, the on-vehicle sensor 23 may include a speed sensor, an acceleration sensor, a yaw rate sensor, and a steering angle sensor, and a position detection sensor that detects the position of the vehicle 20.

[0034] The ECU 21 can transmit the driving data indicating the behavior of the vehicle 20 detected by the vehicle-mounted sensor 23 to the server device 10 via the communication device 24. That is, the server device 10 can obtain the driving data from the vehicle 20. The server device 10 can store the driving data obtained from the vehicle 20 in the storage device 12 (see Figure 2 The driving data obtained from the vehicle 20 as a moving body may also be referred to as detection data. It should be noted that the driving data may include position information indicating the position of the vehicle 20 and speed information indicating the speed of the vehicle 20. Figure 2The driving data 121 may include driving data acquired from a vehicle different from the vehicle 20 .

[0035] The ECU 21 can transmit the surrounding image representing the surrounding of the vehicle 20 captured by the vehicle-mounted camera 22 to the server device 10 via the communication device 24. That is, the server device 10 can obtain the surrounding image from the vehicle 20. The server device 10 can store the surrounding image obtained from the vehicle 20 in the storage device 12 (see Figure 2 It should be noted that in Figure 2 The image data 122 may include a surrounding image acquired from a vehicle different from the vehicle 20 .

[0036] Reference Figure 4 The operation of the server device 10 is described with reference to the flowchart of FIG. 1 . In the first embodiment, a plurality of vehicles including the vehicle 20 may send driving data to the server device 10 in sequence. As a result, the driving data of each of the plurality of vehicles including the vehicle 20 may be stored in sequence in the storage device 12 of the server device 10. It should be noted that the server device 10 may also delete the driving data stored in the storage device 12 after the first predetermined time has passed since the data was acquired by the server device 10. In addition, a plurality of vehicles including the vehicle 20 may send peripheral images to the server device 10 in sequence. As a result, the peripheral images of each of the plurality of vehicles including the vehicle 20 may be stored in sequence in the storage device 12 of the server device 10. It should be noted that the server device 10 may also delete the peripheral images stored in the storage device 12 after the second predetermined time has passed since the data was acquired by the server device 10. It should be noted that the second predetermined time may be different from or the same as the first predetermined time.

[0037] exist Figure 4 In step S101, the driving data acquisition unit 111 of the server device 10 acquires driving data from the storage device 12. For example, the driving data acquisition unit 111 may acquire driving data of a predetermined period from the driving data 121 stored in the storage device 12. For example, the driving data acquisition unit 111 may acquire driving data of a predetermined position range from the driving data 121 stored in the storage device 12. For example, the driving data acquisition unit 111 may acquire driving data of a predetermined period and a predetermined position range from the driving data 121 stored in the storage device 12.

[0038] The driving data congestion detection unit 112 of the server device 10 detects congestion based on at least a part of the driving data acquired in the processing of step S101 (step S102). The driving data congestion detection unit 112 generates the first congestion information of the detected congestion. It should be noted that the method of detecting congestion based on driving data can be applied to various existing methods. That is, the method of detecting congestion based on driving data can be appropriately selected according to the specifications of the product, etc. It should be noted that as an example of a method of detecting congestion based on driving data, the following method can be listed. The driving data congestion detection unit 112 can detect congestion based on the time change of the position of the vehicle represented by the position information contained in the driving data.

[0039] The determination unit 113 of the server device 10 determines whether to use images in congestion detection based on the confidence of the first congestion information generated by the driving data congestion detection unit 112 (step S103). The determination unit 113 may determine that images are used in congestion detection when the confidence of the first congestion information is less than the first threshold. In other words, the determination unit 113 may determine that images are not used in congestion detection when the confidence of the first congestion information is greater than the first threshold. It should be noted that a specific example of the confidence of the first congestion information will be described later.

[0040] In the process of step S103, if it is determined that the image is not used for congestion detection (step S103: No), the process ends. Figure 4 In this case, the server device 10 may distribute the congestion information based on the first congestion information to a plurality of vehicles including the vehicle 20 via the communication device 13 .

[0041] When it is determined in the process of step S103 that an image is used for congestion detection (step S103: Yes), the vehicle selection unit 114 of the server device 10 selects a vehicle as a source of providing an image (e.g., a peripheral image) used for congestion detection (step S104). For example, the vehicle selection unit 114 may select one or more vehicles traveling at the congestion occurrence location indicated by the first congestion information or near the congestion occurrence location as vehicles as a source of providing an image.

[0042] The image specification changing unit 115 of the server device 10 adjusts (or changes) the specification of the image used in congestion detection based on the confidence of the first congestion information (step S105). It should be noted that the processing of step S105 can also be performed in parallel or successively with the processing of step S104. It should be noted that the specification of the image can include at least one of the size of the image, the resolution of the image, and the length of the dynamic image. A specific example of a method for adjusting the specification of an image based on the confidence will be described later. For example, the image specification changing unit 115 can make the resolution of the image when the confidence of the first congestion information is less than the first threshold but is a value near the first threshold lower than the resolution of the image when the confidence of the first congestion information is much lower than the first threshold.

[0043] The image data acquisition unit 116 of the server device 10 acquires image data (e.g., surrounding images) from the storage device 12 (step S106). Specifically, the image data acquisition unit 116 acquires images acquired from one or more vehicles selected by the vehicle selection unit 114 from the image data stored in the storage device 12 using the image specifications adjusted by the image specification change unit 115.

[0044] The image data congestion detection unit 117 of the server device 10 detects congestion using at least a portion of the image data acquired in the process of step S106 (step S107). The image data congestion detection unit 117 generates second congestion information of the detected congestion. It should be noted that various existing methods can be applied to the method of detecting congestion using image data. That is, the method of detecting congestion using image data can be appropriately selected according to, for example, the specifications of the product.

[0045] It should be noted that as an example of a method for detecting congestion using image data, the following method can be listed. The image data congestion detection unit 117 can also detect congestion by performing predetermined image processing (such as at least one of object detection processing, tracking processing, and lane detection processing) on ​​an image (such as a peripheral image). The image data congestion detection unit 117 can also detect congestion using a learning model that outputs a detection result of congestion when image data is input. Such a learning model can be constructed by deep learning, which is one of the methods of machine learning. The learning model constructed by deep learning can be a mathematical model constructed by machine learning using a neural network with a multi-layer structure having multiple intermediate layers (also referred to as hidden layers). The neural network can be, for example, a convolutional neural network.

[0046] The determination unit 113 determines whether to repeat the above process based on the confidence of the second congestion information generated by the image data congestion detection unit 117 (step S108). When the confidence of the second congestion information is less than the second threshold, the determination unit 113 may determine to repeat the above process. In other words, when the confidence of the second congestion information is greater than the second threshold, the determination unit 113 may determine not to repeat the above process. It should be noted that a specific example of the confidence of the second congestion information will be described later.

[0047] In the process of step S108, if it is determined that the above process is to be repeated (step S108: Yes), the process of step S101 is performed. On the other hand, if it is determined that the above process is not to be repeated (step S108: No), the congestion information integration unit 118 of the server device 10 integrates the first congestion information and the second congestion information to generate integrated congestion information (step S109). The server device 10 can distribute congestion information based on the integrated congestion information to a plurality of vehicles including the vehicle 20 via the communication device 13.

[0048] It should be noted that the congestion information integration unit 118 may also generate integrated congestion information by adding information represented by the second congestion information to information represented by the first congestion information. The congestion information integration unit 118 may also generate integrated congestion information by changing (in other words, replacing) at least a portion of the information represented by the first congestion information with information represented by the second congestion information. The congestion information integration unit 118 may also generate integrated congestion information by correcting at least a portion of the information represented by the first congestion information based on the information represented by the second congestion information.

[0049] (First specific example)

[0050] As a first specific example of the operation of the congestion detection system 1, in addition to Figure 4 In addition to the flowchart, refer to Figure 5A to Figure 8 The operation of detecting congestion for each lane will be described.

[0051] exist Figure 4 In the process of step S101, the driving data acquisition unit 111 of the server device 10 can acquire the driving data of each of the plurality of vehicles driving on the predetermined road within a predetermined period from the storage device 12. The predetermined road can be, for example, a road corresponding to a link in a road map represented by a combination of nodes and links.

[0052] exist Figure 4In the process of step S102, the driving data congestion detection unit 112 may obtain the speed distribution based on the speed information included in the driving data acquired by the driving data acquisition unit 111. The driving data congestion detection unit 112 may assume that the number of distributions is "2" and perform fitting processing using a mixed Gaussian model on the obtained speed distribution.

[0053] It should be noted that the number of distributions may vary depending on the number of lanes through which vehicles moving in one direction on the road can pass. For example, if the number of lanes through which vehicles moving in one direction on the road can pass is "2" (so-called single-side 2-lane road), the number of distributions may be "2". For example, if the number of lanes through which vehicles moving in one direction on the road can pass is "3" (so-called single-side 3-lane road), the number of distributions may be "3". It should be noted that the initial value of the number of distributions may also be set independently of the number of lanes through which vehicles moving in one direction on the road can pass.

[0054] For example, if congestion occurs in one lane through which vehicles moving in one direction can pass and vehicles moving in one direction can pass, and there is no congestion in other lanes adjacent to the one lane, the speed of vehicles traveling in the one lane is slower than that of vehicles traveling in other lanes. In this case, if fitting processing is performed in the processing of step S102, for example, Figure 5A As shown in Figure 2, there are two peaks. Figure 5A , Figure 5B In the figure, the “number of units” on the vertical axis can also be replaced by “frequency”.

[0055] For example, in the case where both the one lane and the other lane are congested, it is generally believed that there is almost no difference between the speed of vehicles traveling in the one lane and the speed of vehicles traveling in the other lanes. In this case, if fitting processing is performed in the process of step S102, for example, Figure 5B As shown, there is a high probability that only one peak appears. Figure 5A and Figure 5B The result of the fitting process shown corresponds to an example of the above-mentioned first congestion information.

[0056] The driving data congestion detection unit 112 can calculate the confidence (i.e., the confidence of the first congestion information) based on the result of the fitting process, etc. For example, the driving data congestion detection unit 112 can calculate the confidence by a mathematical formula such as "conf = w1×x1+w2×x2+w3×x3+…+wn×xn". It should be noted that in the above mathematical formula, "conf" represents the confidence, "w1, w2, w3, …, wn" represent weights, and "x1, x2, x3, …, xn" represent parameters.

[0057] The above parameters may include, for example, at least one of (1) the number of vehicles, (2) the ratio of the number of vehicles in one lane to the number of vehicles in other lanes, (3) the fitting score of the mixed Gaussian model, (4) the average speed of the distribution on the low-speed side of multiple distributions, and (5) the difference between the average speeds of multiple distributions. It should be noted that the above mathematical formula is an example, and the confidence level may also be calculated by a method different from the above mathematical formula.

[0058] Through the research of the inventors of the present application, it was found that in the action of detecting congestion for each lane, there is a tendency for the confidence to decrease in at least one of the following situations, for example: (1) the number of vehicles traveling on a predetermined road is small or there is a deviation in the number of vehicles in each lane; (2) the difference between the average speed of the lane where congestion occurs and the average speed of the lane where no congestion occurs is relatively small; (3) the vehicles in the lane where congestion occurs move at a certain speed (in other words, the vehicles in the lane where congestion occurs do not repeatedly stop and start); (4) the congestion in the lane where congestion occurs is temporarily relieved; and (5) the number of lanes through which vehicles moving in one direction can pass is 3 or more.

[0059] exist Figure 4 In the process of step S103, if it is determined that the image is used for congestion detection (step S103: Yes), the process after step S104 is performed. In the process of step S104, the vehicle selection unit 114 can select a vehicle with a relatively high speed among a plurality of vehicles traveling on a predetermined road within a predetermined period as the vehicle providing the image.

[0060] A vehicle with a relatively high speed travels a relatively long distance in a certain period (speed is the amount of change in the position of the vehicle per unit time). Therefore, the surrounding image (e.g., dynamic image) captured by the vehicle-mounted camera of the vehicle during the period becomes more spatially inclusive than the surrounding image (e.g., dynamic image) captured by the vehicle-mounted camera of the vehicle with a relatively low speed during the period. That is, in a certain period, the range of space captured by the vehicle-mounted camera of the vehicle with a relatively high speed is larger than the range of space captured by the vehicle-mounted camera of the vehicle with a relatively low speed. In other words, when capturing the same range of space, the data volume of the surrounding image (e.g., dynamic image) of the vehicle with a relatively high speed is smaller than the data volume of the surrounding image (e.g., dynamic image) of the vehicle with a relatively low speed. Therefore, if the surrounding image of a vehicle with a relatively high speed is used, the load of image processing in the image data congestion detection unit 117 can be reduced. For example, based on such a reason, the vehicle selection unit 114 can select a vehicle with a relatively high speed among multiple vehicles traveling on a predetermined road within a predetermined period.

[0061] In the process of step S104, the vehicle selection unit 114 may select, for example, one vehicle among a plurality of vehicles traveling on a predetermined road within a predetermined period, which has a relatively high speed and has been on the predetermined road for a relatively long time within the predetermined period. If configured in this way, it is highly likely that the surrounding image (e.g., dynamic image) of the one vehicle contains relatively many places where congestion occurs, and it can be expected that the vehicle will be selected in the predetermined road. Figure 4 Congestion is appropriately detected in the process of step S107.

[0062] exist Figure 4 In the process of step S105, the image specification changing unit 115 adjusts (or changes) the specification of the image used for congestion detection based on the confidence (i.e., the confidence of the first congestion information) calculated by the driving data congestion detection unit 112. For example, the image specification changing unit 115 can adjust the image specification by the mathematical formula "l = max (σ (conf) × l_max, l_min)".

[0063] It should be noted that, in the above mathematical formula, "l" represents the adjusted dynamic image length, "conf" represents the confidence, "l_max" represents the dynamic image length before adjustment, "l_min" represents the minimum value of the dynamic image length, and "σ" represents an activation function such as an identity function or a sigmoid function. Here, the case where the image specification changing unit 115 adjusts the dynamic image length is described, but the image specification changing unit 115 may also adjust at least one of the image size and resolution based on or instead of the dynamic image length.

[0064] exist Figure 4 In the process of step S106 , the image data acquisition unit 116 acquires an image acquired from one vehicle selected by the vehicle selection unit 114 among the image data stored in the storage device 12 , using the image specification adjusted by the image specification changing unit 115 .

[0065] exist Figure 4 In the process of step S107, the image data congestion detection unit 117 detects congestion using at least a portion of the image data acquired by the image data acquisition unit 116. Figure 6A to Figure 8 An example of a method of congestion detection by the image data congestion detection unit 117 will be described.

[0066] Fig. 6A An example of an image captured by the vehicle-mounted camera at time t1 is shown. Figure 6B An example of an image captured by a vehicle-mounted camera at time t2 after time t1 is shown. Fig. 6A and Figure 6BThe first vehicle captured by the vehicle-mounted camera in the image shown is traveling in the center lane of the image. Fig. 6A The vehicle enclosed by the dashed rectangle in the image shown is Figure 6B The vehicles enclosed by the dotted rectangle in the image shown are the same vehicle. Hereinafter, the vehicle enclosed by the dotted rectangle will be appropriately referred to as the "second vehicle".

[0067] The speed of the first vehicle is higher than that of the second vehicle. Fig. 6A and Figure 6B As shown in FIG. 1 , the distance between the first vehicle and the second vehicle decreases as time passes. Here, the horizontal direction of the image captured by the vehicle-mounted camera is set as the x-axis. In addition, the x-coordinate of the center of the horizontal direction of the image is set as "x=0". Fig. 6A and Figure 6B It can be seen that the value of the x-coordinate of the center point CP of the dotted rectangle increases with the passage of time. When the time change of the x-coordinate of the center point CP is represented by a curve graph, for example, Figure 7 shown.

[0068] exist Fig. 6A and Figure 6B In the example, the second vehicle is located in a lane adjacent to the right side of the lane in which the first vehicle is traveling. Assuming that a third vehicle is located in a lane adjacent to the left side of the lane in which the first vehicle is traveling and is traveling at a lower speed than the first vehicle, the value of the x-coordinate of the center point of the dotted rectangle surrounding the third vehicle decreases with the passage of time.

[0069] For example, the image data congestion detection unit 117 may perform predetermined image processing on the image data acquired by the image data acquisition unit 116. For example, the image data congestion detection unit 117 may determine the time variation of the x-coordinates of the points corresponding to the central point CP of each of the plurality of vehicles detected from the image data acquired by the image data acquisition unit 116. The image data congestion detection unit 117 may detect a road section where the time variation of the x-coordinates of the points monotonically increases or decreases and there are a predetermined number or more of vehicles within a third predetermined time as a congested section (for example, referring to Figure 8 ).

[0070] It should be noted that, for example Figure 8The result of the above-mentioned image processing shown is equivalent to an example of the above-mentioned second congestion information. The image data congestion detection unit 117 can calculate the confidence (i.e., the confidence of the second congestion information) based on the result of the image processing, etc. For example, the image data congestion detection unit 117 can calculate the confidence by a mathematical formula such as "conf=w1×x1+w2×x2+w3×x3+…+wn×xn" in the same way as the driving data congestion detection unit 112. It should be noted that at least a part of the parameters (such as x1, x2, x3,…, xn) may be different from the parameters when the driving data congestion detection unit 112 calculates the confidence. In the case where the image data congestion detection unit 117 uses a learning model that outputs a detection result of congestion when image data is input, the confidence output from the learning model may also be used as the confidence of the second congestion information.

[0071] exist Figure 4 In the process of step S108, when it is determined that the process is to be repeated from step S101 (step S108: Yes), for example, the driving data congestion detection unit 112 may adjust the parameters of the mixed Gaussian model based on the result of the process of step S107. Specifically, when the driving data congestion detection unit 112 detects the number of lanes in the process of step S107, the number of distributions as the parameters of the mixed Gaussian model may be made the same as the number of lanes detected.

[0072] (Second specific example)

[0073] As a second specific example of the operation of the congestion detection system 1, in addition to Figure 4 In addition to the flowchart, refer to Figures 9 to 11 The operation of detecting the end of the congestion will be described. It should be noted that regarding the second specific example, the description overlapping with the above-mentioned first specific example will be appropriately omitted.

[0074] exist Figure 4 In the process of step S101, the driving data acquisition unit 111 of the server device 10 can acquire the driving data of each of the plurality of vehicles traveling on the predetermined road within the predetermined period from the storage device 12. For example, the predetermined road may be a link L (see FIG. 1 ) connecting the node N1 and the node N2 in the road map represented by a combination of nodes and links. Fig. 9 ) road.

[0075] exist Figure 4 In the processing of step S102, the driving data congestion detection unit 112 can extract, for example, a section where the speed of each vehicle traveling from the node N2 to the node N1 is below the threshold speed based on the speed information included in the driving data acquired by the driving data acquisition unit 111. The driving data congestion detection unit 112 can generate, for example, Fig.10 Time distance graph shown.

[0076] It should be noted that in Fig.10 In , the solid line segments represent the temporal changes in the positions of multiple vehicles. Fig.10 In the figure, node N1 is taken as the starting point of the distance. From node N1, the farther away from node N2, the greater the distance.

[0077] For example, the driving data congestion detection unit 112 may Fig.10 The end of each of the plurality of line segments with a smaller distance is detected as the front of the congestion. The driving data congestion detection unit 112 can Fig.10 The end of each of the plurality of line segments with a larger distance is detected as the end of the congestion. However, there may not be enough vehicles that can communicate with the server device 10 on the predetermined road. Therefore, the driving data congestion detection unit 112 may also estimate the beginning and end of the congestion based on the time distance line diagram, for example, using a Kalman filter. The beginning and end of the congestion detected or estimated by the driving data congestion detection unit 112 are equivalent to another example of the first congestion information.

[0078] The driving data congestion detection unit 112 may calculate the confidence (i.e., the confidence of the first congestion information) based on the detection result of the end of the congestion, etc. For example, the driving data congestion detection unit 112 may calculate the confidence by a mathematical formula such as "conf = w1×x1+w2×x2+w3×x3+…+wn×xn". It should be noted that at least a part of the parameters (e.g., x1, x2, x3, …, xn) may be different from the parameters used when the driving data congestion detection unit 112 calculates the confidence in the first specific example described above.

[0079] Through the research of the inventors of the present application, it was found that in the action of detecting the end of congestion, there is a tendency for confidence to decrease in at least one of the following situations, such as (1) the speed of the vehicle fluctuates around the threshold speed and the line segment in the time-distance graph is interrupted, (2) there is a vehicle that cannot communicate with the server device 10 at the end of the congestion, (3) there is a period when there is no vehicle that can communicate with the server device 10, and (4) the position of the end of the congestion is different for each lane and the deviation of the position of the end of the line segment in the time-distance graph is relatively large.

[0080] exist Figure 4 In the process of step S103, if it is determined that the image is used for congestion detection (step S103: Yes), the process after step S104 is performed. In the process of step S104, the vehicle selection unit 114 can refer to the following Fig.11 Describes the method to select a vehicle. Fig.11In FIG. 1 , the solid line segments represent the temporal changes in the positions of the plurality of vehicles (including the portions where the speed of the vehicle is higher than the above-mentioned threshold speed). Fig.11 In , node N1 is taken as the starting point of the distance, and the distance increases as the distance moves away from node N1 toward node N2. Fig.11 In FIG. 1 , a plurality of black circles respectively represent the position of a vehicle (ie, a vehicle that can communicate with the server device 10 ) at time t3 .

[0081] The vehicle selection unit 114 may, for example, Fig.11 In the time distance diagram shown in FIG. 1 , the position of the window W is changed along the direction extending along the distance axis (i.e., the vertical axis) (refer to Fig.11 The average speed and variance of the two or more vehicles included in the window W are calculated while performing a "scan" ("scanning") of the window W. When the window W contains only two or more vehicles traveling in a congested area, the calculated average speed is relatively low (for example, below the threshold speed mentioned above) and the variance is relatively small. When the window W contains one or more vehicles traveling in a congested area and one or more vehicles traveling in a place that is not a congested area, the calculated average speed is relatively high and the variance is relatively large.

[0082] The vehicle selection unit 114 may determine the position of the window W where the average speed and variance of the two or more vehicles included in the calculated window W are respectively above the average speed threshold and the variance threshold. The vehicle selection unit 114 may select a vehicle (i.e., the vehicle providing the image) based on the determined position of the window W.

[0083] exist Figure 4 In the process of step S105, the image specification changing unit 115 adjusts (or changes) the specification of the image used for congestion detection based on the confidence (i.e., the confidence of the first congestion information) calculated by the driving data congestion detection unit 112. The image specification changing unit 115 can adjust the image specification by a mathematical formula such as "l = max (σ (conf) × l_max, l_min)" as in the above-mentioned first specific example.

[0084] exist Figure 4 In the process of step S107, the image data congestion detection unit 117 detects congestion using at least a portion of the image data acquired by the image data acquisition unit 116. The image data congestion detection unit 117 may detect congestion based on the center point of the dotted rectangle surrounding the vehicle shown in the image (see Fig. 6A , Figure 6B )'s x-coordinate changes over time (see Figure 7 and Figure 8 The image data congestion detection unit 117 can detect the end of the congestion based on the detected congested section.

[0085] exist Figure 4 In the process of step S108, if it is determined that the process from step S101 is repeated (step S108: Yes), for example, the driving data congestion detection unit 112 may adjust the parameters of the detection process of the end of the congestion based on the result of the process of step S107. Fig.10 When DBSCAN (Density-Based Spatial Clustering of Applications with Noise) is applied to each line segment in the time-distance line graph shown to detect the end of congestion, the driving data congestion detection unit 112 may adjust the data point radius of DBSCAN based on the processing result of step S107 .

[0086] (Technical Effect)

[0087] In the congestion detection system 1 of the first embodiment, the vehicle selection unit 114 selects a vehicle that is a source of an image (e.g., a peripheral image) used for congestion detection. And, the image data congestion detection unit 117 detects congestion using the image acquired from the vehicle selected by the vehicle selection unit 114. That is, in the congestion detection system 1 of the first embodiment, only the image acquired from the vehicle selected by the vehicle selection unit 114 becomes a processing target of the image data congestion detection unit 117. Therefore, according to the congestion detection system 1 of the first embodiment, the analysis cost of the image analysis for congestion detection can be suppressed.

[0088] In addition, in the congestion detection system 1 of the first embodiment, the image specification changing unit 115 changes the specification of the image based on the confidence level of the first congestion information. For example, the image specification changing unit 115 may make the resolution of the image lower when the confidence level of the first congestion information is lower than the first threshold value but is close to the first threshold value than when the confidence level of the first congestion information is much lower than the first threshold value. With such a configuration, the analysis cost of the image analysis for congestion detection can be reduced, and the reliability of congestion detection can be improved.

[0089] In the first embodiment, each of the plurality of vehicles including the vehicle 20 may transmit the surrounding image to the server device 10 only when a predetermined event occurs (for example, when the speed falls below a predetermined speed, when the hazard warning light is turned on, etc.). With such a configuration, the communication cost of sending and receiving images used for congestion detection can be suppressed.

[0090] <Second Embodiment>

[0091] A second embodiment of the congestion detection system will be described. It should be noted that, in the second embodiment, the structure and operation of the congestion detection system 1 may be the same as those of the first embodiment described above, except for a part of the operation of the congestion detection system 1. Therefore, for the second embodiment, the description repeated with the first embodiment described above is appropriately omitted.

[0092] In the second embodiment, each of the plurality of vehicles including the vehicle 20 can transmit a surrounding image to the server device 10 when a request to transmit a surrounding image is received from the server device 10. For example, when a request to transmit a surrounding image is received from the server device 10 via the communication device 24, the ECU 21 of the vehicle 20 can transmit a surrounding image representing the surroundings of the vehicle 20 captured by the vehicle-mounted camera 22 to the server device 10 via the communication device 24.

[0093] Reference Figure 4 The operation of the server device 10 according to the second embodiment is described with reference to the flowchart of Figure 4 In the process of step S103, when it is determined that the image is used for congestion detection (step S103: Yes), the vehicle selection unit 114 selects a vehicle as a source of the image (e.g., surrounding image) used for congestion detection (step S104). The image specification changing unit 115 adjusts (or changes) the specification of the image used for congestion detection based on the confidence level of the first congestion information (step S105).

[0094] After the processing of steps S104 and S105 , the computing device 11 of the server device 10 may send a request to send the surrounding image in the image format adjusted by the image format changing unit 115 to one or more vehicles selected by the vehicle selecting unit 114 via the communication device 13 .

[0095] The vehicle that receives the transmission request from the server device 10 (i.e., the vehicle selected by the vehicle selection unit 114) transmits the surrounding image of the image specification adjusted by the image specification changing unit 115 to the server device 10. The server device 10 may temporarily store the surrounding image transmitted from the vehicle in the storage device 12.

[0096] Afterwards, in Figure 4 In the process of step S106 , the image data acquisition unit 116 acquires the image data (ie, the surrounding image transmitted from the vehicle) from the storage device 12 .

[0097] (Technical Effect)

[0098] According to the congestion detection system 1 of the second embodiment, as in the first embodiment described above, the analysis cost of image analysis for congestion detection can be suppressed, and the reliability of congestion detection can be improved. In the congestion detection system 1 of the second embodiment, as described above, the vehicle (for example, the vehicle 20) sends the surrounding image to the server device 10 when there is a transmission request from the server device 10. Therefore, according to the congestion detection system 1 of the second embodiment, the communication cost of sending and receiving the image used for congestion detection can be suppressed.

[0099] <Third Embodiment>

[0100] The third embodiment of the congestion detection system is described. It should be noted that, in the third embodiment, the structure and operation are the same as those of the first embodiment described above, except for a part of the operation of the congestion detection system 1. Therefore, for the third embodiment, the description repeated with the first embodiment described above is appropriately omitted.

[0101] In the third embodiment, each of the plurality of vehicles including the vehicle 20 has a function equivalent to the image data acquisition unit 116 and the image data congestion detection unit 117 in the first embodiment. For example, the ECU 21 of the vehicle 20 may have a function equivalent to the image data acquisition unit 116 and the image data congestion detection unit 117 in the first embodiment. Therefore, the computing device 11 of the server device 10 in the third embodiment may not have the image data acquisition unit 116 and the image data congestion detection unit 117.

[0102] Reference Figure 4 The operation of the congestion detection system 1 according to the third embodiment will be described with reference to the flowchart. Figure 4 In the process of step S103, when it is determined that the image is used for congestion detection (step S103: Yes), the vehicle selection unit 114 selects a vehicle as a source of the image (e.g., surrounding image) used for congestion detection (step S104). The image specification changing unit 115 adjusts (or changes) the specification of the image used for congestion detection based on the confidence level of the first congestion information (step S105).

[0103] After the processing of steps S104 and S105 , the computing device 11 of the server device 10 may transmit the image specification information indicating the specification of the image adjusted by the image specification changing unit 115 to one or more vehicles selected by the vehicle selecting unit 114 via the communication device 13 .

[0104] exist Figure 4In the process of step S106, the vehicle that receives the image specification information from the server device 10 (i.e., the vehicle selected by the vehicle selection unit 114) acquires the surrounding image captured by the vehicle-mounted camera of the vehicle (in other words, image data) in the image specification indicated by the image specification information. For example, when the vehicle that receives the image specification information is the vehicle 20, the ECU 21 can acquire the surrounding image captured by the vehicle-mounted camera 22 in the image specification indicated by the image specification information.

[0105] exist Figure 4 In the process of step S107, the vehicle that receives the image specification information from the server device 10 detects congestion using at least a portion of the surrounding image (in other words, image data) acquired in the process of step S106. The vehicle generates second congestion information of the detected congestion. For example, in the case where the vehicle that receives the image specification information is the vehicle 20, the ECU 21 may detect congestion using at least a portion of the surrounding image acquired in the process of step S106. The ECU 21 may generate the second congestion information of the detected congestion.

[0106] After the process of step S107, the vehicle transmits the second congestion information to the server device 10. In this case, the vehicle may transmit information indicating the reliability of the second congestion information to the server device 10 in addition to the second congestion information.

[0107] (Technical Effect)

[0108] According to the congestion detection system 1 of the third embodiment, similarly to the above-mentioned first embodiment, the analysis cost of the image analysis used for congestion detection can be suppressed, and the reliability of congestion detection can be improved. In the congestion detection system 1 of the third embodiment, as described above, the second congestion information is generated in the vehicle (for example, the vehicle 20). Therefore, in the congestion detection system 1 of the third embodiment, it is not necessary to send the image used in congestion detection from the vehicle (for example, the vehicle 20) to the server device 10. Therefore, according to the congestion detection system 1 of the third embodiment, the communication cost of sending and receiving the image used in congestion detection can be reduced.

[0109] Various aspects of the invention derived from the above-described embodiment will be described below.

[0110] A congestion detection system according to one embodiment of the invention comprises: a driving data acquisition unit for acquiring driving data from a plurality of vehicles; a first detection unit for generating first congestion detection information based on at least a portion of the acquired driving data; an image acquisition unit for acquiring image information of a peripheral image of at least one of the plurality of vehicles; a determination unit for determining whether to use the image information in congestion detection based on a confidence level of the first congestion detection information; a second detection unit for generating second congestion detection information using the image information acquired by the image acquisition unit when the determination unit determines that the image information is to be used in congestion detection; and a generation unit for generating third congestion detection information based on the second congestion detection information.

[0111] In the above-mentioned embodiment, the “driving data acquisition unit 111” is equivalent to an example of a “driving data acquisition unit”, the “driving data congestion detection unit 112” is equivalent to an example of a “first detection unit”, the “image data acquisition unit 116” is equivalent to an example of an “image acquisition unit”, the “determination unit 113” is equivalent to an example of a “determination unit”, the “image data congestion detection unit 117” is equivalent to an example of a “second detection unit”, the “congestion information integration unit 118” is equivalent to an example of a “generation unit”, and the “ECU 21” is equivalent to another example of an “image acquisition unit” and a “second detection unit”.

[0112] In this congestion detection system, the second detection unit may generate the second congestion detection information using image information whose image specifications are changed according to the confidence level of the first congestion detection information.

[0113] In this congestion detection system, the second detection unit may generate the second congestion detection information using image information selected based on the confidence level of the first congestion detection information.

[0114] In the congestion detection system, the generation unit may generate the third congestion detection information based on the first congestion detection information and the second congestion detection information.

[0115] In the congestion detection system, the image acquisition unit may acquire image information of a surrounding image of the at least one vehicle when the determination unit determines that the image information is to be used for congestion detection.

[0116] The congestion detection system may include a change unit that changes the image specification of the image information according to the confidence of the first congestion detection information, and the image acquisition unit may acquire the image information of the surrounding image of the at least one vehicle with the image specification changed by the change unit. In the above embodiment, the "image specification change unit 115" is an example of the "change unit".

[0117] The congestion detection system may include a selection unit that selects at least one vehicle from the plurality of vehicles based on the confidence of the first congestion detection information, and the image acquisition unit may acquire image information of a peripheral image of the at least one vehicle selected by the selection unit. In the above embodiment, the "vehicle selection unit 114" is an example of a "selection unit".

[0118] The present invention is not limited to the above-described embodiments, and can be appropriately modified within the scope not departing from the gist or idea of ​​the invention as read from the claims and the entire specification, and a congestion detection system with such modifications is also included in the protection scope of the present invention.

Claims

1. A congestion detection system, characterized in that: comprising a server device, The server device has a processor. The processor executes the following commands: Acquire driving data from multiple vehicles respectively; generating first congestion detection information based on at least a portion of the acquired driving data; acquiring image information of a peripheral image of at least one of the plurality of vehicles; determining whether to use image information in congestion detection according to the confidence of the first congestion detection information; If it is determined that the image information is used for congestion detection, generating second congestion detection information using the acquired image information; as well as Based on the second congestion detection information, third congestion detection information is generated.

2. The congestion detection system according to claim 1, characterized in that: The processor further executes a command for generating the second congestion detection information using image information whose image specifications are changed according to the confidence level of the first congestion detection information.

3. The congestion detection system according to claim 1, characterized in that: The processor further executes a command for generating the second congestion detection information using image information selected according to the confidence level of the first congestion detection information.

4. The congestion detection system according to claim 1, characterized in that: The processor further executes a command to generate the third congestion detection information based on the first congestion detection information and the second congestion detection information.

5. The congestion detection system according to claim 1, characterized in that: The processor further executes the following command: when it is determined that the image information is used for congestion detection, the processor acquires image information of a surrounding image of the at least one vehicle.

6. The congestion detection system according to claim 1, characterized in that: The processor also executes the following commands: changing the image specification of the image information according to the confidence level of the first congestion detection information; and Image information of the surrounding image of the at least one vehicle is acquired in the changed image format.

7. The congestion detection system according to claim 1, characterized in that: The processor also executes the following commands: selecting at least one vehicle from the plurality of vehicles according to the confidence level of the first congestion detection information; and Image information of a surrounding image of the selected at least one vehicle is acquired.

8. The congestion detection system according to claim 4, characterized in that: The server device further includes a communication unit that distributes congestion information based on the third congestion detection information to a plurality of vehicles including the vehicle.