Method and apparatus for providing vehicle geometry estimates of surrounding vehicles

Abstract

The invention relates to a method for providing a vehicle geometry estimate of a surrounding vehicle (40). In a first step of the method, an image (I) of at least a portion of the surrounding vehicle (40) is received, the portion comprising a license plate (42) fixed to the surrounding vehicle (40). Next, the license plate (42) is detected in the received image (I) and an alphanumeric identification code of the license plate (42) is extracted. Thereafter, vehicle geometry parameters (P) associated to the alphanumeric identification code are requested from a database (36) via a wireless data connection (38). Thereafter, the vehicle geometry parameters (P) are received from the database (36) via the wireless data connection (38) and a vehicle geometry is estimated based on the received vehicle geometry parameters (P). Furthermore, a system (12) for providing a vehicle geometry estimate of a surrounding vehicle (40) and a vehicle (10) comprising such a system are described. Furthermore, a computer readable medium is proposed.

Description

Technical Field

[0001] This disclosure relates to a method for providing an estimation of the vehicle geometry of surrounding vehicles.

[0002] Furthermore, this disclosure relates to a system for providing an estimate of the vehicle geometry of surrounding vehicles.

[0003] In addition, this disclosure relates to vehicles that include such a system.

[0004] Furthermore, this disclosure relates to a computer-readable medium including instructions that, when executed by a computer, cause the computer to perform this method. Background Technology

[0005] This method, system, and computer-readable medium can be used in vehicles capable of autonomous driving. Furthermore, they can be used in vehicles with a human driver to assist that driver.

[0006] In both autonomous and assisted driving scenarios, the so-called autonomous vehicle, or primary vehicle, must possess information about surrounding vehicles that may pose potential obstacles. To this end, the vehicle is equipped with appropriate sensor units, including, for example, optical cameras, radar equipment, ultrasonic units, or lidar units. The vehicle's surrounding environment is continuously monitored by the sensor units, and the sensor measurements are converted into object information describing the surrounding vehicles. Summary of the Invention

[0007] The purpose of this disclosure is to improve this object information.

[0008] The subject matter of the independent claims of this disclosure at least partially addresses or mitigates this problem, wherein further examples are incorporated in the dependent claims.

[0009] According to a first aspect, a method is provided for estimating the vehicle geometry of surrounding vehicles, comprising:

[0010] a. Receive images of at least a portion of the surrounding vehicles, including license plates affixed to the surrounding vehicles.

[0011] b. Detect the license plate in the received image and extract the alphanumeric identification code of the license plate.

[0012] c. Request vehicle geometry parameters linked to the alphanumeric identifier from a database via a wireless data connection.

[0013] d. Receive vehicle geometry parameters from the database via a wireless data connection, and

[0014] e. Estimate the vehicle geometry based on the received vehicle geometry parameters.

[0015] The received images are typically in the form of image data, which is computer-readable data containing information about the image content.

[0016] The received image may have been captured by a vision sensor unit, such as an optical camera, radar device, ultrasonic unit, lidar unit, etc. In this case, the image can be static or moving. If the image is moving, it can also be a series of static image frames.

[0017] It should be noted that the steps of this method are executed automatically, i.e., by the data processing device.

[0018] The method disclosed herein has the advantage of providing highly stable, robust, and accurate estimates of vehicle geometry. This is due to the fact that methods relying solely on information detected by a vision sensor unit mounted on the vehicle must address the potentially limited field of view of the sensor unit. This means that objects occluded by other objects (e.g., surrounding vehicles) cannot be detected. Furthermore, the estimation of vehicle geometry may be distorted depending on the relative position between the vision sensor unit and the detected surrounding vehicles. In contrast, using vehicle geometry parameters from a database as the basis for geometry estimation is highly accurate because these parameters are unaffected by detection errors (e.g., distortion). Moreover, the vehicle geometry parameters received from the database are always complete. Therefore, occlusion of the detected object does not have a negative impact.

[0019] Therefore, the database includes at least one entry containing an alphanumeric identifier and at least one entry containing the corresponding vehicle geometry parameters. This is the case, for example, with official vehicle registration databases. Of course, this data can also be provided in private databases.

[0020] License plate detection algorithms can use known detection algorithms, such as the Maximum Stable Extremal Region (MSER) ​​algorithm or neural network-based deep learning detection algorithms, to locate the license plate in the image frame and extract the alphanumeric identification code.

[0021] Data querying and retrieval can utilize secure standard transport protocols such as TLS 1.2. Alternatively or additionally, the database can be securely stored in the cloud with strict access controls. Furthermore, vehicle geometry parameters received from the database can be encrypted and made visible only to a local cache for short-term tracking purposes, without actually storing the received vehicle geometry parameters.

[0022] An estimate can be provided for a first vehicle, wherein surrounding vehicles are within the first vehicle's field of vision. Therefore, the surrounding vehicles whose geometry is estimated by the method according to this disclosure can be located around or in the environment of the first vehicle, which can also be designated as a self-driving vehicle. The self-driving vehicle can drive autonomously or at least be able to assist a human driver. This means that the method according to this disclosure can be performed on a self-driving vehicle. Of course, one or more of these methods can be performed simultaneously on the self-driving vehicle, for example, one method can be performed for each detected surrounding vehicle.

[0023] In this disclosure, vehicle geometry parameters may include at least one of a vehicle type identifier, a vehicle model identifier, and a vehicle size parameter. The vehicle type identifier should be understood as information about the vehicle type (e.g., truck, bus, motorcycle), such as a code. The vehicle model identifier should be understood as information relating to the vehicle manufacturer and a specific model within the manufacturer's program. Vehicle size parameters may be, for example, the length, width, or height of surrounding vehicles. More complex vehicle size parameters may also be used, such as a bounding box containing the vehicle. It should be noted that vehicle size parameters may be actual measurements or a simplified version thereof.

[0024] Generally, vehicle geometry parameters are defined as alphanumeric strings that contain information about the vehicle's geometry or information about the vehicle's geometry that can be derived from them.

[0025] It should be noted that, for the purposes of this disclosure, vehicle geometry parameters may also include a set of vehicle geometry parameters. For example, such a set may include the length, height, and width of surrounding vehicles.

[0026] Alphanumeric identification codes can be used as identifiers for vehicle tracking. This means that the alphanumeric code of the license plate is used to associate surrounding vehicles detected in subsequent image frames with the corresponding previous image frames, thereby also transforming the corresponding estimate of the vehicle geometry (e.g., bounding box) to the new frame. The alphanumeric code of the license plate is unique compared to known identifiers, at least within a jurisdiction. It is also important to note that the license plate does not undergo modification. Therefore, no ID switching issues occur. This improves detection performance.

[0027] By performing object detection on the received image, surrounding vehicles can be detected in the received image as well. This further enhances detection performance. The object detection method can be a deep learning method executed by a trained learning algorithm configured to determine objects based on labeled data from supervised training.

[0028] Furthermore, additional vehicle geometry parameters can be extracted from detected surrounding vehicles in the received image. Therefore, this method has at least two or more vehicle geometry parameters available. Thus, this method can provide a relatively accurate estimate of the vehicle geometry. If a connection to a database is unavailable, the additional vehicle geometry parameters extracted from the detected image can also be used as an alternative solution. The additional vehicle geometry parameters can also be vehicle size parameters. Therefore, the additional vehicle geometry parameters can be related to the length, width, or height of surrounding vehicles. The additional vehicle geometry parameters can also include data describing the bounding boxes of the detected surrounding vehicles.

[0029] Similarly, additional vehicle geometry parameters should be understood to mean that it can also include a set of vehicle geometry parameters.

[0030] Furthermore, vehicle geometry parameters received from the database can be combined with additional vehicle geometry parameters. This enhances the detection performance of the method. For example, if one of the parameters is incorrect, it does not necessarily lead to an incorrect estimation of the vehicle geometry, as the error can be compensated for by other appropriate parameters.

[0031] The combination of vehicle geometry parameters received from the database and additional vehicle geometry parameters can also be specified as parameter fusion.

[0032] This method can determine whether the vehicle geometry parameters associated with the extracted alphanumeric identifier have already been retrieved from the database. If so, it is not necessary to retrieve the same vehicle geometry parameters again. In other words, the method can be executed only when the alphanumeric identifier is detected for the first time. Therefore, the method can only be triggered when a new license plate is detected. This makes the method computationally efficient.

[0033] According to a second aspect, a system is provided for estimating the geometry of surrounding vehicles. The system includes a data processing device capable of connecting to a vision sensor unit to receive images from the vision sensor unit. The data processing device has a license plate detection unit configured to detect a license plate in the image received from the vision sensor unit. Furthermore, the data processing device includes a character recognition unit configured to extract an alphanumeric identification code of the license plate detected by the license plate detection unit in the received image. Additionally, a vehicle geometry estimation unit is provided, configured to provide an estimate of the vehicle geometry. The system also has a communication unit connected to the data processing device and configured to exchange data with a database via a wireless data connection. Therefore, the vehicle geometry can be estimated with high accuracy and reliability. This also applies to cases where surrounding vehicles are at least partially obscured, as explained in conjunction with the method according to the invention.

[0034] Visual sensor units may include optical cameras, radar devices, ultrasonic units, lidar units, etc.

[0035] At least some units of the system can be software units.

[0036] The data processing device may also include a license plate tracking unit configured to track license plates. Therefore, license plates can be used to track vehicles with fixed license plates. This provides excellent tracking reliability because license plates are unique, at least within a jurisdiction. Thus, tracking inaccuracies related to ID switching are eliminated.

[0037] The data processing device may further include: a vehicle detection unit configured to detect surrounding vehicles from images received by a vision sensor unit; a vehicle tracking unit configured to track vehicles; and an additional vehicle geometry estimation unit configured to extract additional vehicle geometry estimates from the surrounding vehicles detected in the received images. Therefore, surrounding vehicles can be tracked additionally without using license plates. Furthermore, vehicle geometry can be additionally estimated based on the received images. Thus, the data processing device provides two separate possibilities for detecting surrounding vehicles and estimating the corresponding vehicle geometry. In other words, the data processing device is redundant in these respects and therefore operationally robust.

[0038] The data processing device may further include a fusion unit configured to combine vehicle geometric parameters received from a database via a communication unit with additional vehicle geometric parameters extracted from the received images by an additional vehicle geometry estimation unit. This can thus provide enhanced detection performance.

[0039] Furthermore, the system may include a vision sensor unit configured to capture images of portions of the vehicle. The vision sensor unit is connected to a data processing device to transmit the captured images. Therefore, images including surrounding vehicles can be captured in a simple and reliable manner.

[0040] The system may additionally include a database containing at least one vehicle geometry parameter linked to the alphanumeric identification code of the license plate, the database being linked to the communication unit via a wireless data connection. Therefore, high-quality vehicle geometry parameters can be obtained.

[0041] According to a third aspect, a vehicle including the system according to this disclosure is provided. The vehicle including the system can be designated as a self-driving vehicle. Therefore, the self-driving vehicle is able to estimate the geometry of surrounding vehicles in its environment with high accuracy.

[0042] According to a fourth aspect, a computer-readable medium including instructions, which, when executed by a computer, cause the computer to perform the method according to the present disclosure. This computer-readable medium provides the same effects and advantages as described in relation to the method of the present disclosure.

[0043] The method disclosed herein can be implemented at least in part by a computer, and can be implemented in software or hardware, or both. Furthermore, the method can be executed by computer program instructions running on a device providing data processing functionality. The data processing device can be a suitable computing device, such as an electronic control module, and can be a distributed computer system. The data processing device or computer can each include one or more of a processor, memory, data interface, etc.

[0044] It should be noted that the above examples can be combined with each other, regardless of the aspects involved. Therefore, this method can be combined with structural features, and similarly, the system can be combined with the features described above regarding this method.

[0045] These and other aspects of the invention will become clear and illustrated by reference to the examples described below. Attached Figure Description

[0046] Examples of the present invention will now be described with reference to the following figures.

[0047] Figure 1 A vehicle according to the present disclosure is shown, including a system according to the present disclosure and a computer-readable medium according to the present disclosure, and is capable of performing a method according to the present disclosure for providing an estimation of the vehicle geometry of surrounding vehicles.

[0048] Figure 2 It shows the operation during the process. Figure 1 vehicles, and

[0049] Figure 3 A method for estimating the geometry of a vehicle in relation to surrounding vehicles, according to this disclosure, is illustrated schematically.

[0050] The accompanying drawings are merely schematic representations and are intended to illustrate the invention only. In principle, identical or equivalent elements have the same reference numerals. Detailed Implementation

[0051] Figure 1 A vehicle 10 is shown, including a system 12 for providing information to surrounding vehicles 40 (only when...). Figure 2 The estimated vehicle geometry (shown in the figure).

[0052] System 12 includes a vision sensor unit 14 configured to capture images I of surrounding vehicles 40. In this example, vision sensor unit 14 is an optical camera unit.

[0053] Furthermore, system 12 includes a data processing device 16 connected to the vision sensor unit 14. The connection between the data processing device 16 and the vision sensor unit 14 is configured such that the image I captured by the vision sensor unit 14 can be transmitted to the data processing device 16.

[0054] Therefore, the data processing device 16 is configured to receive image I from the vision sensor unit 14.

[0055] In the following text, it is assumed that the image I captured by the vision sensor unit 14 includes a portion of the surrounding vehicles 40, wherein the portion includes license plates 42 fixed to the surrounding vehicles 40 (see [link to image]). Figure 2 ).

[0056] In addition, the data processing device 16 includes a license plate detection unit 18, which is configured to detect license plate 42 in image I received from the vision sensor unit 14.

[0057] In addition, the data processing device 16 has a character recognition unit 20, which is configured to extract the alphanumeric identification code of the license plate 42.

[0058] In addition, the data processing device 16 has a vehicle geometry estimation unit 22, which is configured to provide an estimate of the geometry of the surrounding vehicles 40.

[0059] The data processing device 16 also includes a license plate tracking unit 24 for tracking the license plate 42 within a set of image frames.

[0060] The data processing device 16 is also equipped with a vehicle detection unit 26. The vehicle detection unit 26 is configured to detect surrounding vehicles 40 from the image I received from the vision sensor unit 14.

[0061] In addition, the data processing device 16 has a vehicle tracking unit 28, which is configured to track surrounding vehicles 40.

[0062] The data processing device 16 also has an additional vehicle geometry estimation unit 30, which is configured to extract additional estimates of the vehicle geometry of the surrounding vehicles 40 detected by the vehicle detection unit 26 in the received image I.

[0063] The data processing device 16 further includes a fusion unit 32, which is configured to combine vehicle geometry parameters provided by the vehicle geometry estimation unit 22 with additional vehicle geometry parameters extracted from the received image I by the additional vehicle geometry estimation unit 30.

[0064] In addition, system 12 includes a communication unit 34, which is connected to data processing device 16 and configured to communicate via wireless data connection 38 (see [link]). Figure 2 It exchanges data with database 36.

[0065] System 12 is configured to perform a method for providing an estimate of the vehicle geometry of the surrounding vehicles 40. This will be further referenced. Figure 2 and Figure 3 Let me explain.

[0066] It should be noted that, for this purpose, a computer-readable medium is provided as part of the data processing apparatus 16. The computer-readable medium includes instructions that, when executed by a computer, cause the computer to perform the method.

[0067] In addition, for this purpose, system 12 also includes a database 36, which includes at least one vehicle geometry parameter linked to the alphanumeric identification code of license plate 42.

[0068] Database 36 does not form part of vehicle 10.

[0069] However, the database 36 is linked to the communication unit 34 of the vehicle 10 via a wireless data connection 38.

[0070] Image I is captured by vision sensor unit 14 as input to a method for providing an estimate of the vehicle geometry of surrounding vehicles 40. Image I includes at least a portion of the surrounding vehicles 40, including their license plates 42.

[0071] Image I is received by data processing device 16. Figure 3 Step S1 in the process.

[0072] Then, the license plate detection unit 18 detects the license plate 42 in the received image I, and the character recognition unit 20 extracts the corresponding alphanumeric identification code. Figure 3 Step S2 in the process.

[0073] More specifically, for surrounding vehicle 40, the alphanumeric identification code DEF23B is extracted (see...). Figure 2 ).

[0074] Next, it is evaluated whether a database request has already been executed in conjunction with that specific alphanumeric identifier. If it has, this approach can be abandoned.

[0075] If not executed, the method continues and requests the vehicle geometry parameters P (linked to the alphanumeric identifier) ​​from the database 36 via wireless data connection 38. Figure 3 Step S3 in the process.

[0076] The communication unit 34 is used to perform this step.

[0077] Suppose that the vehicle geometry parameter P for a specific alphanumeric identification code is available in database 36, and database 36 responds to the request by providing the vehicle geometry parameter P.

[0078] For illustrative purposes only, the vehicle geometry parameter P can be a set of parameters including the length, width, and height of the surrounding vehicles 40.

[0079] Therefore, data processing device 16 receives vehicle geometric parameters P from database 36 via wireless data connection 38. Figure 3 Step S4 in the process.

[0080] Similarly, the communication unit 34 is used to transmit the vehicle geometry parameters P from the database 36 to the data processing device 16.

[0081] More specifically, the vehicle geometric parameters P are transmitted to the geometric estimation unit 22.

[0082] Within the geometry estimation unit 22, the vehicle geometry parameter P is either directly used as an estimate of the vehicle geometry or used as an input parameter for generating such an estimate of the vehicle geometry.

[0083] In summary, an estimate of the vehicle geometry of the surrounding vehicles 40 is provided by requesting vehicle geometry parameters P from database 36 using the alphanumeric identification code of license plate 42 as an index.

[0084] Furthermore, an object detection method can be performed on image I captured by visual sensor unit 14, which is received in data processing device 16 (more precisely, in vehicle detection unit 26). Therefore, in image I, at least surrounding vehicles 40 are detected. Figure 3 Step S6 in the process.

[0085] Based on this, additional vehicle geometry parameters P' are obtained from image I through an additional vehicle geometry estimation unit 30. Figure 3 Step S7 in the process.

[0086] For illustrative purposes, we can again assume that the length, width, and height values ​​of the surrounding vehicles 40 are extracted from image I.

[0087] The vehicle geometry parameters P (i.e., the height, width, and length of surrounding vehicles already received from database 36) and the additional vehicle geometry parameters P' (i.e., the height, width, and length of surrounding vehicles already extracted from image I) can then be combined or fused in fusion unit 32. Figure 3 Step S8 in the process.

[0088] In this case, for example, the corresponding values ​​of length, height, and width are compared, and the larger value is always used to estimate the geometry of the surrounding vehicles 40.

[0089] Furthermore, alphanumeric identification codes are used as identifiers for surrounding vehicles 40 detected in the received image I, and based on this, surrounding vehicles 40 are tracked within the set of image frames.

[0090] Those skilled in the art, in practicing the claimed invention, can understand and implement other variations of the disclosed examples from a study of the drawings, disclosure, and appended claims. In the claims, the word "comprising" does not exclude other elements or steps, and the indefinite articles "a" or "an" do not exclude a plural. A single processor or other unit can perform the functions of several items or steps recited in the claims. The fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used advantageously. Computer programs can be stored / distributed on suitable media, such as optical storage media or solid-state media provided with or as part of other hardware, but can also be distributed in other forms, such as via the Internet or other wired or wireless telecommunications systems. Any reference signs in the claims should not be construed as limiting the scope of the claims.

[0091] List of reference numerals

[0092] 10 vehicles

[0093] 12. Systems for providing estimates of the geometry of surrounding vehicles.

[0094] 14 Visual sensor units

[0095] 16 Data processing equipment

[0096] 18 License Plate Detection Units

[0097] 20 character recognition units

[0098] 22 Vehicle geometry estimation unit

[0099] 24 License Plate Tracking Unit

[0100] 26 Vehicle Inspection Unit

[0101] 28 Vehicle tracking units

[0102] 30 additional vehicle geometry estimation units

[0103] 32 fusion units

[0104] 34 Communication Units

[0105] 36 Databases

[0106] 38 Wireless data connection

[0107] 40 surrounding vehicles

[0108] 42 license plates

[0109] P Vehicle geometry parameters

[0110] P' Additional vehicle geometry parameters

[0111] I Image

Claims

1. A method for providing an estimate of the vehicle geometry of surrounding vehicles (40), wherein, The surrounding vehicles (40) are located around the autonomous vehicle (10) capable of autonomous driving or capable of assisting a human driver, including: a. Receive at least a portion of an image (I) of the surrounding vehicles (40) at the vehicle (10), the portion including a license plate (42) attached to the surrounding vehicles (40). b. Detect the license plate (42) in the received image (I) and extract the alphanumeric identification code of the license plate (42), wherein the alphanumeric identification code is used as an identifier for tracking the surrounding vehicles (40). c. Request vehicle geometry parameters (P) linked to the alphanumeric identifier from the database (36) via a wireless data connection (38), wherein the vehicle geometry parameters (P) are alphanumeric strings that include information about the vehicle geometry, or information about the vehicle geometry that can be derived from them. d. Receive the vehicle geometry parameters (P) from the database (36) at the vehicle (10) via the wireless data connection (38), and e. Estimate the vehicle geometry based on the received vehicle geometry parameters (P) for use in the autonomous driving of the vehicle (10) or in assisting the human driver via the vehicle (10).

2. The method according to claim 1, wherein, The estimate is provided for the vehicle (10), wherein the surrounding vehicles (40) are within the field of vision of the vehicle (10).

3. The method according to claim 1 or 2, wherein, The vehicle geometry parameters (P) include at least one of the vehicle type identifier, vehicle model identifier, and vehicle size parameters.

4. The method according to claim 1 or 2, wherein, By performing an object detection method on the received image (I), the surrounding vehicles (40) are detected in the received image (I).

5. The method according to claim 4, wherein, Additional vehicle geometry parameters (P') are extracted from the detected surrounding vehicles (40) in the received image (I).

6. The method according to claim 5, wherein, The vehicle geometry parameters (P) received from the database (36) and the additional vehicle geometry parameters (P') are combined (S8).

7. The method according to claim 1 or 2, comprising determining whether the vehicle geometry parameter (P) linked to the extracted alphanumeric identifier has been previously retrieved from the database (36).

8. A system (12) for providing an estimate of the vehicle geometry of surrounding vehicles (40), wherein, The surrounding vehicles (40) are located around the autonomous vehicle (10) capable of autonomous driving or the autonomous vehicle (10) capable of assisting a human driver. The system (12) includes: a data processing device (16) capable of connecting to a vision sensor unit (14) to receive images (I) from the vision sensor unit (14), the data processing device (16) having: a. A license plate detection unit (18) is configured to detect the license plates (42) of the surrounding vehicles (40) in an image (I) received from the vision sensor unit (14). b. A character recognition unit (20) is configured to extract an alphanumeric code of the license plate (42) detected by the license plate detection unit (18) in the received image (I), wherein the alphanumeric code is used as an identifier for tracking the surrounding vehicles (40). c. A vehicle geometry estimation unit (22) is configured to provide an estimate of the vehicle geometry of the surrounding vehicles (40) for use in the autonomous driving of the vehicle (10) or in assisting the human driver via the vehicle (10), and The communication unit (34) is connected to the data processing device (16) and is configured to exchange data with the database (36) via a wireless data connection (38).

9. The system (12) according to claim 8, wherein, The data processing device (16) includes a license plate tracking unit (24) configured to track the license plate (42).

10. The system (12) according to claim 8 or 9, wherein, The data processing device (16) includes a. A vehicle detection unit (26) is configured to detect the surrounding vehicles (40) in an image (I) received from the vision sensor unit (14). b. A vehicle tracking unit (28) is configured to track the surrounding vehicles (40), and c. An additional vehicle geometry estimation unit (30) is configured to extract an estimate of the vehicle geometry from the surrounding vehicles (40) detected in the received image (I).

11. The system (12) according to claim 10, wherein, The data processing device (16) includes a fusion unit (32) configured to combine vehicle geometry parameters (P) received from the database (36) via the communication unit (34) and additional vehicle geometry parameters (P') extracted from the received image (I) by the additional vehicle geometry estimation unit (30).

12. The system (12) according to any one of claims 8 or 9, comprising the visual sensor unit (14) configured to capture an image (I) of a portion of the surrounding vehicle (40), the visual sensor unit (14) being connected to the data processing device (16) to transmit the captured image (I) to the data processing device (16).

13. The system (12) according to any one of claims 8 or 9 further includes a database (36) comprising at least one vehicle geometry parameter (P) linked to the alphanumeric identification code of the license plate (42), the database (36) being linked to the communication unit (34) via a wireless data connection (38).

14. A vehicle (10) comprising a system (12) according to any one of claims 8 to 12.

15. A computer-readable medium comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 7.

Images