Computer-implemented method for interpreting an object subregion

Abstract

The invention relates to a computer-implemented method for interpreting an object subregion (11) as a vehicle similar to a reference vehicle or not a vehicle.

Description

Computer-implemented method for interpreting a sub-area of ​​an object

[0001] The invention relates to a method according to the preamble of claim 1.

[0002] The method according to the invention relates to a computer-implemented method for interpreting a sub-area of ​​an object as either a vehicle similar to a reference vehicle or not a vehicle at all, where a portion of the object is recorded with a video camera and / or a lidar and / or a radar. The video camera creates video images as a series of pictures. The lidar or radar also creates a series of pictures. The images are created sequentially, possibly as a series over time.

[0003] Video cameras, lidar, and radar are mentioned as examples. The specialist can also use other imaging methods.

[0004] The documents DE102013005882, DE102016226204 Al, DE102008036219 Al, DE102017123982A1, DE102015200436A1, DE102017204347A1, US 2018101176A1, DE102012000949A1, US2015242708A1 do not disclose methods with a definition of a no-entry zone of a road in an image and the output of an alarm on a future trajectory of the approaching vehicle in the no-entry zone.

[0005] The invention disclosed herein aims to interpret, in a computer-implemented method, an object, of which only a part of the object is visible in an image, as a part of a vehicle or as not being a part of a vehicle.

[0006] According to the invention, this is achieved by claim 1.

[0007] The computing unit compares the recorded object sub-area, detected using an artificial intelligence method, with a multitude of single or multiple related reference vehicle sub-areas of a reference vehicle, determining probability values ​​of similarity between the recorded object sub-area and at least one reference vehicle sub-area.

[0008] The concept of at least one reference vehicle sub-area can, on the other hand, be comprised of several interconnected reference vehicle sub-areas, as explained below with reference to Figure 3. The relationship between the reference vehicle sub-areas can be spatial (adjacent to each other, at a distance from each other) or functional.

[0009] On the other hand, the term "at least one reference vehicle sub-area" can encompass a single reference vehicle sub-area, as explained below with reference to Figure 4.

[0010] The method according to the invention can comprise a mathematical routine for describing the area of ​​the object sub-area to the greatest possible extent with one or more reference vehicle sub-areas, wherein each reference vehicle sub-area describes a sub-area of ​​the object sub-area. The at least one reference vehicle sub-area or sub-areas should exhibit the highest possible probability of similarity to the described sub-area of ​​the object sub-area.

[0011] The individual reference vehicle sub-area and the related reference vehicle sub-areas are stored in a reference database. The procedure described here, in particular the comparison of the object sub-area with one or more reference vehicle sub-areas, can be implemented using trigonometric methods. Following established best practices, the aforementioned database can be created by training the neural network.

[0012] The individual reference vehicle sub-area or contiguous reference vehicle sub-areas represent a sub-area of ​​the recorded object sub-area. The individual reference vehicle sub-area thus has a smaller or equal area in the image to the object sub-area. The definition of the area, in particular a minimum area of ​​a reference vehicle sub-area, therefore determines the area at which an object sub-area is interpreted as a vehicle sub-area or not.

[0013] This prevents, for example, an object part such as a side rearview mirror from being interpreted as a vehicle part, which would render the subsequent process steps, in particular the detection of a hazardous situation described below, inefficient.

[0014] The method according to the invention can, on the one hand, deliver the result that, with a probability value greater than a probability limit, the computing unit interprets the recorded object sub-area as a vehicle sub-area corresponding to the reference vehicle sub-area and assigns a reference property to the recorded vehicle sub-area, which is linked to the reference vehicle sub-area in the reference database.

[0015] On the other hand, the method according to the invention can deliver the result that, with a probability value smaller than the probability limit, the computing unit interprets the recorded object sub-area as not being a vehicle sub-area of ​​a vehicle with a vehicle sub-area corresponding to a reference vehicle sub-area.

[0016] The method according to the invention can be characterized by the fact that the calculation unit extracts the probability value a determined similarity value between the recorded object sub-area and at least one reference vehicle sub-area and / or a number of related reference vehicle sub-areas determined, which reference vehicle sub-area exhibits a similarity value exceeding a similarity limit to a sub-area of ​​the recorded object sub-area.

[0017] The similarity between the reference vehicle sub-area and the object sub-area and / or the number of reference vehicle sub-areas used to describe the object sub-areas can be taken into account in determining the probability value.

[0018] The method according to the invention can be characterized by the fact that the calculation unit extracts the probability value a determined similarity value between the recorded object sub-area and at least one reference vehicle sub-area and / or a determined area fraction of at least one reference vehicle sub-area to the area of ​​the recorded object sub-area and / or to a reference vehicle, which reference vehicle sub-area exhibits a similarity value exceeding a similarity limit to a sub-area of ​​the recorded object sub-area.

[0019] Preferably, this process step is carried out using a video image from a video camera and / or an image created with a lidar.

[0020] The similarity between the reference vehicle sub-area and the object sub-area, as well as the area coverage of the object sub-area by at least one reference vehicle sub-area, can be taken into account in determining the probability value.

[0021] The method according to the invention can be characterized by the fact that The computing unit determines an initial distance between the video camera or lidar or radar and the first truck. The processing unit determines an image reference size of the first truck in the image from the first distance and a reference size of a reference vehicle. The processing unit determines an initial difference value between the image reference size and the image size of the first truck in the image. either the computing unit interprets the first truck as the reference vehicle if the first difference value is less than or equal to a first difference limit value. or the computing unit interprets the first truck as not being a reference vehicle if the first difference value is greater than the first difference limit.

[0022] The method according to the invention can be characterized by the fact that The computing unit determines a second distance between the video camera or lidar or radar and the second truck. The processing unit determines an image reference size of the second truck in the image from the second distance and a reference size of a reference vehicle. The processing unit determines a second difference value between the image reference size and the image size of the second truck in the image. either the computing unit interprets the second truck as the reference vehicle if the first difference value is less than or equal to a first difference limit value or the computing unit interprets the first truck as not being a reference vehicle if the second difference value is greater than the second difference limit.

[0023] The aforementioned first or second distance is the distance between the front surface of the respective truck and the video camera, lidar, or radar. The front surface is intersected with the road surface in the image. The respective distance is then determined from the position of this intersection line between the front surface and the road surface in the image.

[0024] The method according to the invention can be characterized by the fact that The object sub-area and the reference vehicle sub-area are a lateral sub-area of ​​a view of a vehicle.

[0025] The method according to the invention can also be a method for classifying a vehicle as a collision vehicle with a collision zone around the video camera, lidar, or radar. A vehicle is classified as a collision vehicle if it can be assumed that the vehicle is traveling within a collision zone around the video camera, lidar, or radar, particularly in the direction of travel behind the video camera. A vehicle traveling within this collision zone can cause damage to persons, especially emergency personnel, and / or property within this collision zone.

[0026] The method according to the invention can also be characterized by the fact that a road is recorded with a video camera, lidar, or radar, creating an image or series of images showing which lane the vehicle is moving on, in the computing unit at least one section of the road where driving restrictions are defined in at least one image, either the processing unit outputs an alarm signal in at least one image when a vehicle section is positively detected in a driving ban section, or the processing unit in the event of a negative detection of a vehicle sub-area in a driving ban section in which at least one image does not issue an alarm signal.

[0027] The restricted traffic zone can be defined as a real two-dimensional area within the road. This can be useful if the road is defined as a plane. The restricted traffic zone, as a real two-dimensional area, can be defined as a two-dimensional image area.

[0028] The restricted driving zone can also be defined as a real three-dimensional space. The restricted driving zone, as a real three-dimensional area, can be defined in the image as a two-dimensional image area.

[0029] Analogous to the no-entry zone, a traffic lane can also be defined. The no-entry zone and, if applicable, the traffic lane are defined in at least one image.

[0030] The method according to the invention also allows the vehicle to be classified as a collision vehicle.

[0031] The method according to the invention is characterized by the fact that a road is recorded with a video camera or lidar or radar, showing which lane the vehicle is moving on, In the computing unit, at least one section of the road where driving is prohibited must be defined in at least one image. The computing unit determines a completed trajectory of the vehicle, compares this completed trajectory with a large number of reference trajectories stored in the database, determining a similarity value for each, and selects at least one reference trajectory with a similarity value exceeding a similarity threshold. which selected reference trajectory is linked to a future reference trajectory, which future reference trajectory is assigned to the vehicle, either the processing unit outputs an alarm signal in at least one image when a future reference trajectory is positively determined in a driving ban section, or the processing unit does not issue an alarm signal in at least one image when a future reference trajectory is negative in a driving ban section.

[0032] Preferably, this process step is carried out using a video image from a video camera and / or an image created with a lidar.

[0033] The image of the road can be compared to a large number of reference images stored in a database, with a similarity value determined for each. The reference image with the highest similarity value is selected, and a reference no-entry zone linked to this image in the database is then transferred to the image as the no-entry zone. The corresponding section of the image is defined as the no-entry zone. These steps can be performed using standard methods based on artificial intelligence principles.

[0034] The method according to the invention can be characterized by the fact that the computing unit in at least two images the speed of the vehicle moving on the restricted driving area or the vehicle moving on the restricted driving area in the future and the distance of the vehicle to the video camera or lidar or radar are determined, The processing unit calculates a collision time interval until the vehicle collides with the camera from the determined speed and distance. either the processing unit only issues an alarm signal if the collision time span is equal to or below a collision time span limit value. or the processing unit does not issue an alarm signal if the collision time exceeds a collision time limit.

[0035] The time until the collision vehicle reaches the collision zone, defined by the collision time span, can therefore be used as a criterion for issuing the alarm signal. The alarm signal is only issued if the collision time span is below a certain threshold.

[0036] The method according to the invention can also include the method required for transmitting the alarm signal.

[0037] The method according to the invention can be characterized by the fact that the computing unit includes a radio unit the computing unit couples the radio unit with a selected peripheral device using a coupling signal, Furthermore, either the processing unit sends a control signal to the peripheral device, or the peripheral device indicates receipt with a reception indicator. or the processing unit sends the alarm signal to the peripheral device.

[0038] The invention is further explained with reference to the following embodiments shown in the figures:

[0039] Fig. 1 shows a top view of a possible application of the method according to the invention,

[0040] Fig. 2 shows a camera image of a possible application of the method according to the invention,

[0041] Figures 3 and 4 illustrate methods for interpreting the object sub-area.

[0042] The embodiments shown in the figures merely illustrate possible embodiments. It should be noted that the invention is not limited to these specifically depicted embodiments, but also encompasses combinations of the individual embodiments with one another and combinations of an embodiment with the general description given above. These further possible combinations need not be explicitly mentioned, as they are within the knowledge of a person skilled in the art in this technical field, given the teaching provided by the present invention.

[0043] The scope of protection is defined by the claims. However, the description and drawings must be consulted for the interpretation of the claims. Individual features or combinations of features from the different embodiments shown and described can, in themselves, constitute independent inventive solutions. The problem underlying these independent inventive solutions can be found in the description.

[0044] The following discussion focuses on the method using a video camera. A person skilled in the art could also use a lidar or radar instead of a video camera and accordingly process an image or images instead of a video image. Regarding Figure 1:

[0045] Figure 1 shows a top view of a possible application of the method according to the invention. Figure 1 shows a first truck 1 and a second truck 2, which are driving on a road 5. The trucks 1 and 2 are moving in a first direction of travel 3 and a second direction of travel 4, respectively.

[0046] Figure 1 further shows an emergency vehicle 7, which is parked on the shoulder of road 5. Personnel acting as emergency responders (not shown in Figure 1) may be located in front of and / or behind the emergency vehicle 7 (viewed in the direction of travel 3, 4). In the application example presented here, the task may be to protect these persons from an accident caused by a truck 1, 2. The method may be designed to issue an alarm signal when a truck 1, 2 is highly likely to enter a collision zone 17 extending around (or behind) the video camera 6. The alarm signal is intended to warn the persons located in the collision zone 17 so that they can move away from it.

[0047] A person skilled in the art can also devise other embodiments and other applications. The invention is not limited to the application shown in Figure 1.

[0048] The method according to the invention is preferably carried out as a computer-implemented method.

[0049] A video camera 6 records a section 11 of the second truck 2. Figure 2 shows a video image from the video camera 6. The first truck 1 partially obscures the second truck 2. Only a section 11 of the second truck 2, and therefore not the entire view of the second truck 2, can be recorded.

[0050] The video camera 6 creates video images. Each video image can be time-stamped.

[0051] Video camera 6 captures image data encompassing trucks 1 and 2, and, if applicable, road 5. Video camera 6 forwards the image data to a processing unit for further processing. Using state-of-the-art technology, the processing unit extracts object sub-area 11 from the image data. Furthermore, the processing unit can extract the first truck, 1, from the image data.

[0052] The object of the method according to the invention is to interpret the image data concerning the object sub-area 11 as either a sub-area of ​​a vehicle, corresponding to a reference vehicle, or as not being a sub-area of ​​a vehicle. This step of interpreting the object sub-area 11 as either a vehicle sub-area or not is determined by whether the subsequent method steps are executed or not. Efficient application of the method according to the invention to the analysis of a convoy of vehicles passing by the video camera 6 is advantageous with regard to the amount of data to be processed.

[0053] The processing unit compares the recorded object sub-area 11 with a plurality of single or multiple contiguous reference vehicle sub-areas 15, 16 of a reference vehicle, determining probability values ​​of similarity between the recorded object sub-area 11 and at least one reference vehicle sub-area 15, 16. A possible implementation of this process step of comparing the object sub-area 11 with a reference vehicle sub-area 15, 16 is discussed with reference to Figure 3.

[0054] The individual and related reference vehicle sub-areas 15, 16 are stored in a reference database.

[0055] The size of the individual reference vehicle sub-area 15, 16, or the size of the contiguous reference vehicle sub-areas 15, 16, as discernible in the video image, constitutes a sub-area of ​​the recorded object sub-area 11. The object sub-area 11 is equal to or larger than the individual reference vehicle sub-area 15, 16 or the contiguous reference vehicle sub-areas 15, 16. By exclusively storing reference vehicle sub-areas 15, 16 with a minimum size, the minimum size is defined from which an object sub-area 11 is considered a relevant vehicle sub-area. This prevents, for example, the interpretation of a side rearview mirror 18 of the second truck 2 in step 20.

[0056] The processing unit can select the single reference vehicle sub-area 15, 16 or the related reference vehicle sub-areas 15, 16 with the highest probability value. It is conceivable that several individual or related reference vehicle sub-areas 15, 16 have the same probability value for the object sub-area 11.

[0057] A similarity threshold is stored in the processing unit. This similarity threshold can be an input value. Alternatively, the similarity threshold can be a dynamic value, defined by the processing unit depending on the video image.

[0058] If the probability value exceeds a certain probability threshold, the processing unit interprets the recorded object sub-area 11 as a vehicle sub-area of ​​a vehicle with a vehicle sub-area corresponding to the reference vehicle sub-area 15, 16. The properties of the reference vehicle sub-area 15, 16 are assigned to the vehicle or vehicle sub-area.

[0059] Alternatively, if the probability value is less than the probability limit, the processing unit interprets the recorded object sub-area 11 as not being present. Vehicle component area.

[0060] The interpretation of the recorded object area as a vehicle sub-area can also include the possibility that object sub-area 11 is not interpreted as a further vehicle sub-area belonging to other vehicles, such as emergency vehicles 7. The procedure described above has the effect of classifying the recorded object sub-area 11 as a vehicle sub-area relevant for the further procedure, as an object sub-area 11 that is not relevant for the further procedure, as image interference, or as a further vehicle sub-area belonging to another vehicle. Another vehicle could, for example, be an emergency vehicle 7.

[0061] An interpretation of object sub-area 11 as another vehicle sub-area can also be made by having further reference vehicle sub-areas 15, 16 of further reference vehicles such as emergency vehicle 7 stored in the database and by finding a high probability of similarity between object sub-area 11 and the further vehicle sub-area, so that this object sub-area 11 is regarded as another reference sub-area.

[0062] In the application example shown in Figures 1 and 2, the object section and the reference vehicle section 15, 16 are a lateral section of a front view of the vehicle. Other sections and views are also conceivable.

[0063] As mentioned above, the object of the method according to the invention can be to warn persons present in collision area 17, such as emergency services, of an approaching truck 2 if it is likely that the truck 2 will enter the collision area 17.

[0064] The method according to the invention is characterized in that a road 5, on which the vehicle is moving, is recorded with the video camera 6. In the processing unit, at least one restricted traffic section 10 of the road 5 is defined in the video image, on which vehicles such as the aforementioned trucks 1 and 2 are normally not permitted to drive. The processing unit can also define a lane 9 on which vehicles such as the aforementioned trucks 1 and 2 are normally permitted to drive.

[0065] This definition of the driving ban section 10 and also of lane 9 can be entered via an input.

[0066] This definition of the restricted driving section 10 and also of lane 9 can be carried out by analyzing the video image(s), either as a supplement to or alternative to the aforementioned input. The processing unit detects the lane edges 12 and 13 in the video images and defines the area between the lane edges 12 and 13 as lane 9 and the area outside lane 9 as restricted driving section 10. Such methods for defining restricted driving section 10 and lane 9 are known according to the state of the art.

[0067] In the method according to the invention, a position of the object sub-area 11, classified as a vehicle sub-area, that can be detected in a video image can be a criterion. Figure 2 illustrates the object sub-area 11 of the second truck 2 projecting into the no-driving zone 10.

[0068] In step 21, the processing unit determines whether the object sub-area 11, identified as a vehicle sub-area, is located either within the restricted driving zone 10 or outside of the restricted driving zone 10 and thus in lane 9. The condition recorded in the video image can serve as a criterion for an alarm signal.

[0069] The processing unit outputs an alarm signal in the video image if a positive detection of an object sub-area 11, identified as a vehicle sub-area, is detected in step 21 within the restricted driving section 10. In the embodiment shown in Figures 1 and 2, the second truck 2 triggers an alarm with object sub-area 11 because a part of the second truck 2 is located within the restricted driving section 10.

[0070] Alternatively, if a negative detection of a vehicle segment within a restricted driving zone 10 is identified in step 21, the processing unit does not issue an alarm signal in the video image. In the embodiment shown in Figures 1 and 2, the first truck 1 does not trigger an alarm signal because no part of the first truck 1 is within restricted driving zone 10.

[0071] A fictitious condition, derived from video images and with a high probability of occurrence, can serve as a criterion for triggering the alarm signal. This criterion is checked in step 21.

[0072] Similar to the criterion described above regarding the output of the alarm signal, the video camera 6 records the road 5, specifically the lane on which the vehicle is traveling. The processing unit defines at least one restricted traffic zone 10 of road 5 within the video images. A lane 9 can also be defined.

[0073] The processing unit determines a completed trajectory 19 of the object sub-area 11 of the detected vehicle, here the completed trajectory of the second truck 2. The processing unit compares the completed trajectory 19 with a large number of reference trajectories stored in the database, determining a similarity value for each, and selects at least one reference trajectory with a similarity value exceeding a similarity threshold.

[0074] The selected reference trajectory in the database is linked to a future reference trajectory of a defined length. The processing unit assigns the future reference trajectory to the object sub-area 11, which is recognized as a vehicle sub-area. By defining the length of the future trajectory, only an area relevant to the process is included.

[0075] In Figure 1, the future reference trajectory assigned to the object sub-area 11 or vehicle sub-area of ​​the second truck 2 is shown as the second direction of travel 4.

[0076] The processing unit determines, using established principles, whether the future trajectory 4 of the second truck 2 is either within or outside the restricted driving area 10. A property, in particular a location of the future trajectory 4 within the future reference trajectory assigned to the object area, can serve as a criterion for issuing an alarm signal.

[0077] The processing unit either outputs an alarm signal in a video image if a future reference trajectory in a driving ban section 10 is positively determined, or it does not output an alarm signal in a video image if a future reference trajectory in a driving ban section 10 is negatively determined.

[0078] It is mentioned above that an object area 11 cannot be interpreted as a vehicle sub-area of ​​a vehicle relevant to the procedure. This also implies that the object area can be interpreted as another vehicle sub-area of ​​another vehicle, such as an emergency vehicle 7. Consequently, no alarm signal can be issued for another vehicle that is or will be located in the restricted driving zone 10.

[0079] The procedure described above, including the decision criteria mentioned, is illustrated in Figure 5. Figure 5 demonstrates that even a positive determination of the object area or the future trajectory within the restricted driving section 10 triggers an alarm. A positive determination of the object area or the future trajectory within the restricted driving section 10 can also trigger an alarm.

[0080] According to established theory, the processing unit can determine the speed of the vehicle moving on restricted section 10, or the vehicle that will move on restricted section 10 in the future, and / or the distance of the vehicle to the video camera 6 from the video images. From this, the processing unit can calculate a collision time interval 22 until the vehicle collides with the camera. The calculated collision time interval 22 can serve as a criterion for issuing a warning signal.

[0081] The processing unit either issues an alarm signal if the collision time interval 22 is equal to or below a collision time interval limit, or it does not issue an alarm signal if the collision time interval 22 is greater than a collision time interval limit.

[0082] The invention disclosed herein relates to the transmission of the alarm signal. The video camera 6 can be arranged on an emergency vehicle 7, which is parked on the shoulder of road 5. The video images recorded by the video camera 6 are forwarded to the processing unit for further processing. The processing unit is preferably arranged in the emergency vehicle 7. An alarm signal generated by the processing unit is preferably forwarded to at least one peripheral device 8 via the radio communication described below.

[0083] The peripheral device 8 can be worn by a person who, for example, is located in front of or behind the emergency vehicle 7, viewed from the direction of travel 3, 4. The object of the invention can be to warn this person of an approaching vehicle, such as the second truck 2, with an object section 11 in the restricted driving zone 10. The method of transmitting the radio signals between the processing unit and the peripheral device 8 is crucial for solving this problem.

[0084] The processing unit sends a coupling signal to the radio unit of peripheral device 8. The coupling signal is transmitted exclusively by the processing unit or by peripheral device 8 during the period required to establish a coupling between the processing unit and peripheral device 8.

[0085] The execution of the procedure described above, in particular the issuance of a warning signal to the aforementioned persons, takes place after the computing unit and the peripheral device 8 have been coupled. During this process of verifying the necessity of issuing warning signals, either a control signal is sent from the computing unit to the peripheral device 8, or the computing unit sends the alarm signal to the peripheral device 8.

[0086] The alarm signal has a higher priority than the control signal.

[0087] The peripheral device 8 can indicate the reception of the control signal with a reception indicator and / or report the reception of the control signal back to the processing unit.

[0088] The peripheral device 8 can indicate the receipt of an alarm signal by means of a vibration and / or an acoustic indication and / or via a display device.

[0089] As described above, an alarm signal is triggered if a future trajectory 4 lies within driving restriction zone 10 and / or if a vehicle segment is located within driving restriction zone 10. Conversely, no alarm signal is triggered if the future trajectory 4 does not lie within driving restriction zone 10.

[0090] As can be seen in Figure 1, the restricted driving zone 10 includes the collision zone 17. Furthermore, the collision zone 17, and thus also the restricted driving zone 10, includes a parking area for the vehicle 7. Therefore, the restricted driving zone 10 is advantageously subdivided into several sub-zones, such as the collision zone 18 and the parking area for the vehicle 7. The method according to the invention is characterized in that an alarm signal with a first alarm priority is issued for a future trajectory 4 in the restricted driving zone 10 and / or for a vehicle sub-zone within the restricted driving zone 10. Furthermore, an alarm signal with a second alarm priority can be issued for a future trajectory 4 in the restricted driving zone 10 and / or for a vehicle sub-zone within the restricted driving zone 10, wherein the second alarm priority is higher than the first alarm priority.Furthermore, an alarm signal with a third alarm priority can be issued for a future trajectory 4 in the driving ban area 10 and / or for a vehicle sub-area in the driving ban area 10, whereby the third alarm priority is higher than the first alarm priority and the second. Alarm priority.

[0091] The alarm signal with the first priority can take the form of an initial sound, such as an audible signal. The alarm signal with the second priority can take the form of a second sound, such as a second sound combined with a second vibration. The alarm signal with the third priority can take the form of an audible signal combined with a third vibration. The volume or pitch of the audible signal, or the intensity of the vibration, can increase from the first to the third priority. Regarding Figure 2:

[0092] Figure 2 is described above together with Figure 1.

[0093] Figures 3 and 4 deal with the technical problem of interpreting object sub-area 11. The technical problem of interpreting object sub-area 11 may be preceded by the problem of detecting the first truck 1 and the second truck 2 as individual objects in the image.

[0094] The processing unit can determine a first distance and / or a second distance 22 between the video camera 6 or the lidar or radar and the first truck 1 or the second truck 2, respectively. An intersection line between the front surface of the respective truck 1, 2 and the surface of the road 5 can be determined, and the respective distance can be deduced from the position of the intersection line.

[0095] Furthermore, the processing unit determines an image reference size of the first truck (1) and / or the second truck (2) in the image based on their respective distances and a reference size of a reference vehicle stored in the database. The image reference size indicates the size of each truck (1, 2) as a function of its distance in the image.

[0096] The processing unit determines the difference between the image reference size and the image size of the respective truck 1, 2 in the image. The processing unit interprets either the respective truck 1, 2 as the reference vehicle if the difference value is less than or equal to a first difference threshold, or it interprets the respective truck 1, 2 as not being a reference vehicle if the difference value is greater than the respective difference threshold.

[0097] The aforementioned procedural steps prevent the first truck (1) and the second truck (2) from being incorrectly identified as a single object. Such a false detection would lead to inaccurate size relationships. Regarding Figure 3:

[0098] Figure 3 illustrates a possible embodiment of the method for interpreting an object sub-area 11 contained in the video image. As mentioned above, the object sub-area 11 shown by way of example in Figure 3 can be extracted from one or more video images using prior art methods. The object sub-area 11, which is available in the form of image data, represents the input data for step 20 (see Figure 5).

[0099] The technical task is therefore to classify the extracted object sub-area 11 either as a vehicle part or as not a vehicle part, using a procedure carried out with the computing unit.

[0100] The solution presented in the invention provides that the processing unit determines a probability value as a criterion for the aforementioned classification. A probability value exceeding a certain probability threshold leads to the object sub-area 11 being interpreted as a vehicle sub-area. Otherwise, the processing unit interprets the object sub-area 11 as not being a vehicle sub-area.

[0101] The object sub-area 11 is compared with a determined number of related reference vehicle sub-areas 15, 16.

[0102] Figure 3 shows two reference vehicle sections 15 and 16 as examples. These sections are spatially related. Figure 3 illustrates the special case where the reference vehicle sections 15 and 16 are arranged adjacent to one another. A relationship between the reference vehicle sections 15 and 16 can also exist if they are spaced apart or if they belong to the same vehicle, in this example, the second truck 2.

[0103] A similarity can be determined between the exemplary reference vehicle sub-area 15 and the sub-area of ​​the object sub-area 11 that overlaps with reference vehicle sub-area 15. Furthermore, a similarity can be determined between the exemplary reference vehicle sub-area 16 and the sub-area of ​​the object sub-area 11 that overlaps with reference vehicle sub-area 16. In the example shown in Figure 3, reference vehicle sub-area 15 exhibits a similarity of 88.0% and reference vehicle sub-area 16 exhibits a similarity of 92.0% to the respective overlapping sub-area of ​​object sub-area 11.

[0104] The number of reference vehicle sub-areas 15, 16 can be determined which reference vehicle sub-areas 15, 16 exhibit a similarity to the object sub-area 11 that exceeds a similarity threshold, wherein the similarity is described by a similarity value in accordance with the prior art. In the embodiment according to Figure 3, the two reference vehicle sub-areas 15, 16 (thus two) are sufficiently similar to the respective overlapping sub-areas of the object sub-area 11.

[0105] The aforementioned probability value can be determined for at least one reference vehicle sub-area 15, 16 using a mathematical procedure, whereby a higher similarity value and / or a higher number of sufficiently similar reference vehicle sub-areas 15, 16 lead to a higher probability value.

[0106] The described comparison of object sub-area 11 with individual, related reference vehicle sub-areas 15 and 16 has the technical advantage that the reference vehicle sub-areas 15 and 16 can be adapted to object sub-area 11. It is not necessary to store reference vehicle sub-areas 15 and 16 in the database for every possible size of object sub-area 11.

[0107] The described method can further be characterized by the fact that the processing unit determines the probability value from a calculated area fraction of the contiguous reference vehicle sub-areas 15, 16 to the recorded object sub-area 11 or to the reference vehicle. In the example shown in Figure 3, for instance, the reference vehicle sub-areas 15, 16 cover 75% of the area of ​​the object sub-area 11.

[0108] Determining the area proportion of the reference vehicle sub-areas 15, 16 to the area of ​​the object sub-area 11 incorporates a reliability analysis into the calculation of the probability value. For example, an interpretation of an object sub-area 11 whose area can only be represented to a very small extent by sufficiently similar reference vehicle sub-areas 15, 16 may result in a very low probability value.

[0109] Determining the area ratio of the reference vehicle sub-area 15, 16 to the area ratio of the reference vehicle itself allows for a reliability analysis to be incorporated into the process. Furthermore, this enables the selection of whether the object sub-area 11 is sufficiently large to be relevant for subsequent process steps.

[0110] The aforementioned mathematical procedure can be extended such that the probability value increases with a greater coverage of the area of ​​object sub-area 11 by the sufficiently similar reference vehicle sub-areas 15, 16. Conversely, the probability value decreases with a smaller coverage of the area of ​​object sub-area 11 by the sufficiently similar reference vehicle sub-areas 15, 16.

[0111] The reference ranges, and thus the aforementioned number of reference sub-ranges, are determined in such a way that the mentioned probability value reaches a maximum.

[0112] The method according to the invention can include considering the object sub-area 11 as being similar to the reference vehicle sub-areas with the highest probability value. This highest probability value is compared with the probability limit value, thereby performing an interpretation of the object sub-area 11.

[0113] The computing unit can be used to determine an initial distance between the video camera 6 or the lidar or radar and the first truck 1, as described above.

[0114] The processing unit determines an image reference size of the first truck (1) in the image based on its initial distance and a reference size of the reference vehicle. This image reference size defines how large the vehicle or vehicle portion should appear in the image when the vehicle corresponds to the reference vehicle. The image reference size can be defined by an image height, width, or area.

[0115] The image size defines the actual size of the vehicle or vehicle part shown in the image. The image size is defined in relation to the image reference size.

[0116] The processing unit determines an initial difference value between the image reference size and the image size of the first truck (1) in the image. The processing unit interprets the first truck (1) as the reference vehicle if the initial difference value is less than or equal to a first difference threshold. Conversely, the processing unit interprets the first truck (1) as not being a reference vehicle if the initial difference value is greater than the first difference threshold.

[0117] This procedure step described above can also be applied to the second truck.

[0118] The processing unit determines a second distance 22 between the video camera 6, or the lidar or radar, and the second truck 2. Furthermore, the processing unit determines an image reference size of the second truck 2 within the image from the second distance 22 and a reference size of a reference vehicle. The above provisions regarding the first distance and the first truck 1 apply accordingly.

[0119] The processing unit determines a second difference value between the image reference size and the image size of the second truck (2) in the image. The processing unit interprets the second truck (2) as the reference vehicle if the first difference value is less than or equal to a first difference threshold. Conversely, the processing unit interprets the first truck (2) as not being a reference vehicle if the second difference value is greater than the first difference threshold.

[0120] The first truck (1) and the second truck (2) are detected using artificial intelligence methods. The neural networks must be sufficiently trained so that the first truck (1) and the second truck (2) are recognized as separate objects and not as a single object. The aforementioned procedural steps can be advantageous for the detection of trucks 1 and 2. Regarding Figure 4:

[0121] The embodiment shown in Figure 3 is based on the combination of several reference vehicle sub-areas 15, 16, which makes the advantages described above possible.

[0122] Figure 4 illustrates an equally possible embodiment in which the sub-area of ​​the object sub-area 11 to be interpreted by the processing unit is compared with a single reference vehicle sub-area 15 to determine a probability value. It is mentioned above that the object sub-area 11 shown by way of example in Figure 3 can be extracted from one or more video images using prior art methods. The object sub-area 11, available in the form of image data, represents the input data for step 20 (see Figure 5).

[0123] In analogy to the description of Figure 3, the technical task is therefore to classify the extracted object part area 11 either as a vehicle part or as not a vehicle part, using a procedure carried out with the computing unit.

[0124] The solution presented in the invention provides that the processing unit determines a probability value as a criterion for the aforementioned classification. A probability value exceeding a certain probability threshold leads to the object sub-area 11 being interpreted as a vehicle sub-area. Otherwise, the processing unit interprets the object sub-area 11 as not being a vehicle sub-area.

[0125] The object section 11 is compared with a reference vehicle section 15. A reference vehicle section 15 is shown in Figure 4. The processing unit determines the similarity value between the recorded object section 11 and a reference vehicle section 15.

[0126] The computing unit can further determine an area fraction of one reference vehicle sub-area 15 to the recorded object sub-area 11, which reference vehicle sub-area 15 has a similarity value exceeding a similarity limit to a sub-area of ​​the recorded object sub-area 11.

[0127] The processing unit further determines a probability value using a mathematical procedure, whereby a higher similarity value and / or a higher area share lead to a higher probability value. A lower similarity value and / or a lower area share leads to a higher probability value.

[0128] The method according to the invention can include considering the object sub-area 11 as similar to the at least one reference vehicle sub-area 15 with the highest probability value. This highest probability value is compared with the probability limit value, thereby performing an interpretation of the object sub-area 11.

[0129] The computing unit can be used to determine an initial distance between the video camera 6 or the lidar or radar and the first truck 1, as described above.

[0130] The processing unit determines an image reference size of the first truck (1) in the image based on its initial distance and a reference size of the reference vehicle. This image reference size defines how large the vehicle or vehicle portion should appear in the image when the vehicle corresponds to the reference vehicle. The image reference size can be defined by an image height, width, or area.

[0131] The image size defines the actual size of the vehicle or vehicle part shown in the image. The image size is defined in relation to the image reference size.

[0132] The processing unit determines an initial difference value between the image reference size and the image size of the first truck (1) in the image. The processing unit interprets the first truck (1) as the reference vehicle if the initial difference value is less than or equal to a first difference threshold. Conversely, the processing unit interprets the first truck (1) as not being a reference vehicle if the initial difference value is greater than the first difference threshold.

[0133] This procedure step described above can also be applied to the second truck.

[0134] The processing unit determines a second distance 22 between the video camera 6, or the lidar or radar, and the second truck 2. Furthermore, the processing unit determines an image reference size of the second truck 2 within the image from the second distance 22 and a reference size of a reference vehicle. The above provisions regarding the first distance and the first truck 1 apply accordingly.

[0135] The processing unit determines a second difference value between the image reference size and the image size of the second truck (2) in the image. The processing unit interprets the second truck (2) as the reference vehicle if the first difference value is less than or equal to a first difference threshold. Conversely, the processing unit interprets the first truck (2) as not being a reference vehicle if the second difference value is greater than the second difference threshold.

[0136] The first truck (1) and the second truck (2) are detected using artificial intelligence methods. The neural networks must be sufficiently trained so that the first truck (1) and the second truck (2) are recognized as separate objects and not as a single object. The aforementioned procedural steps can be advantageous for the detection of trucks 1 and 2. Regarding Figure 5:

[0137] Figure 5 shows a flowchart of the method according to the invention. Regarding Figure 6:

[0138] Figure 6 illustrates another application of the method according to the invention. Figure 6 shows a road 5 with lanes 9. At the left edge of Figure 6, the road 5 comprises two lanes 9. At the right edge of Figure 6, the road 5 comprises one lane 9. A person skilled in the art can estimate the number of lanes 9 from the distance between the lane edges 12, 13. Thus, a narrowing of the lanes occurs.

[0139] The object of the method according to the invention can be to indicate when a vehicle, here the second truck 2, does not follow the narrowing of the lanes 9 and poses a danger to persons working in a collision area 17.

[0140] Advantageously, the driving ban section 10 shown in Figure 6 extends into lane 9. This allows a second truck 2 entering driving ban section 10 to be detected before the actual narrowing of lane 9.

[0141] It is also conceivable that the left end of the driving ban section 10 follows the course of the lane edge 12. Regarding Figure 7:

[0142] An advantageous method for defining the driving ban section 10 is presented with reference to Figure 7. Figure 7 shows an image 23 recorded with the video camera 6 (see Figure 1 or Figure 6). The image 23 includes a marker 24 to indicate the orientation of the video camera 6 in space. In the simplest case, and the one discussed here, the marker 24 is the center of the image 23.

[0143] Using methods based on established principles, lane markings or lane boundaries 9, such as lane edge 12, can be detected in Figure 23 by the processing unit. Furthermore, the processing unit detects the image area encompassing the marking 24 in the image and the image area extending laterally to lane 12 as a boundary of lane 9, designated as the no-entry section 10. The upper edge of the no-entry section 10 is defined, for example, by the horizon 25 and / or by a horizontal line at a vertical distance from the marking and / or the lower edge of Figure 23. Regarding Figure 8 and Figure 9:

[0144] An advantageous embodiment of the method according to the invention is presented with reference to Figures 8 and 9. Figures 8 and 9 show a top view. The method explained below with reference to Figures 8 and 9 can also be carried out using a camera image 23, as shown, for example, in Figure 7.

[0145] Figure 8 shows a first truck 1 driving on a road 5. The road 5 includes lane edges 12 and 13, which define lane 9 and a no-entry section 10. The definition of road 5, lane 9, and the no-entry section 10 is carried out using state-of-the-art methods and the description above.

[0146] As described above, the first truck 1 is recognized as a vehicle. Furthermore, as described above, it is recognized that the future trajectory 3 of the first truck 1, with the length shown in Figure 8, is not within the restricted driving zone 10. The length of the future trajectory 3 of the first truck 1 determines the movement of the first truck 1 at the following times in the inventive method.

[0147] In particular, and therefore not restrictively, in the case of a narrow lane 9 and an adjacent driving ban section 10, if the future trajectory 3 is of sufficient length to adequately consider the future movement of the first truck 1, the future trajectory 3 would always be in the driving ban section 10.

[0148] The solution according to the invention provides that only a directional component 28 of the future trajectory, determined as follows, is entered into the method according to the invention as the future trajectory. This solution is applicable to the above description.

[0149] The method according to the invention can be characterized in that a vehicle position 26 of the vehicle in lane 9 is determined. The vehicle position 26 can be the center point of the edge of the frontal view that contacts the road 5.

[0150] A point 27 closest to vehicle position 26 on the lane edge 12, 13 closest to vehicle position 26 is determined. The tangent 29 to lane edge 13 is determined, and a directional component 28 is further determined as a vector which is oriented at a right angle to a tangent of the nearest point 27.

[0151] In the example shown in Figure 8, the directional component 28 points away from the no-entry section 10. Therefore, no alarm signal is issued due to a negative finding.

[0152] Figure 9 shows the case where an alarm signal is issued due to a positive detection. It indicates the direction component 28 of the future trajectory 4 of the second truck 2 to the restricted driving section 10. The direction component 28 of the trajectory 4 is determined using the description for Figure 8. Regarding Figure 10:

[0153] Figure 10 shows an example of how a three-dimensionally represented car 1 is captured in an image according to the prior art. The image area 30 occupied by the car 1 as an object, in the form of a rectangle with sides parallel to Figure 23, is included in the calculations, here in the method according to the invention. Such a consideration of an object is disadvantageous, for example, in the assessment described above of whether a part of the car 1 protrudes into the driving ban section 10. Regarding Figure 11:

[0154] It is proposed that the computing unit be used to detect the perimeter edges of the vehicle section.

[0155] the vehicle section is further subdivided into sections 31, 32,

[0156] which sub-areas have a shape adapted to the perimeter edges.

[0157] Figure 11 shows an example of a partial area 31 of a side surface of the car 1. The side edges of the partial area 31 are parallel to circumferential edges of the car 1.

[0158] Figure 11 shows an example of a partial area 32 of a front surface of the car 1. The side edges of the partial area 32 are parallel to circumferential edges of the car 1. The side edges of the partial area 32 are, for example, parallel to the edges of the car's license plate 33.

[0159] Thus, a shape adapted to the shape of the object, in this case the car, is incorporated into the method according to the invention.

[0160] The sub-areas can be compared with reference sub-areas, which reference sub-areas comprise a view of a reference view. The comparison can include comparing a rectified view of sub-area 31, 32 with an undistorted view of the reference sub-area. If the similarity exceeds a similarity threshold, the reference view is assigned to the sub-area. This preferably includes assigning a reference property to sub-area 31, 32.

[0161] A reference property could, for example, be a direction of travel 34. In this way, the direction of travel of car 1 can be defined with sufficient accuracy if, for example, only the sub-area 31 as a side view or only the sub-area 32 as a front view is detectable in the image. Reference symbol: 1 first truck 2 second truck 3 first direction of travel first truck 4 second direction second truck 5th Street 6 video camera 7 emergency vehicles 8 Peripheral device 9 lanes 10 Driving ban section 11 Object sub-area 12 lanes edge 13 lanes edge 14 (free) 15 Reference vehicle sub-area 16 Reference vehicle sub-area 17 Collision area 18 side rearview mirrors 19 expired trajectories 20 Steps Interpreting Object Sub-area Step 21: Determining the object sub-area and / or future trajectory in the no-driving zone 22 Collision time span Image 23 24 Marking 25 Horizon 26 Vehicle position 27 nearest point 28 Directional component 29 Tangent 30 Image area sub-area sub-area license plate direction of travel

Claims

29 Patent claims:

1. Computer-implemented method for interpreting a sub-area of ​​an object (11) as a vehicle similar to a reference vehicle or not a vehicle, wherein the object sub-area (11) is recorded with a video camera (6) and / or a lidar and / or a radar, which video camera (6) or lidar or radar creates a series of images, A computing unit compares the recorded object sub-area (11) with a multitude of individual reference sub-areas (15, 16) and several related reference vehicle sub-areas (15, 16) of a reference vehicle by determining probability values ​​of a similarity between the recorded object sub-area (11) and at least one reference vehicle sub-area (15, 16), which individual and related reference vehicle sub-areas (15, 16) are stored in a reference database. which individual or related reference sub-areas represent a sub-area of ​​the recorded object sub-area (11), wherein either the computing unit interprets the recorded object sub-area (11) as a vehicle sub-area corresponding to the reference vehicle sub-area (15, 16) when the probability value is greater than a probability limit, and assigns a reference property to the recorded vehicle sub-area that is linked to the reference vehicle sub-area (15, 16) in the reference database. or the computing unit interprets the recorded object sub-area (11) as not being a vehicle sub-area of ​​a vehicle with a vehicle sub-area corresponding to a reference vehicle sub-area (15, 16) when the probability value is less than the probability limit. a road (5) is recorded with the video camera (6) and / or a lidar and / or a radar, on which road the vehicle is moving, in the computing unit at least one driving ban section (10) of the road (5) is defined in at least one image, The computing unit determines a completed trajectory (19) of the vehicle section of a vehicle, which completed trajectory (19) is compared with a multitude of reference trajectories stored in the database, determining one 30 in each case. compares similarity values ​​and selects at least one reference trajectory with a similarity value exceeding a similarity threshold, which selected reference trajectory is linked to a future reference trajectory, which future reference trajectory is assigned to the vehicle sub-area, either the computing unit outputs an alarm signal in at least one image upon a positive determination of a future reference trajectory in a driving ban section (10), or the computing unit does not issue an alarm signal in at least one image when a negative determination of a future reference trajectory is made in a restricted driving section (10).

2. The method according to claim 1, characterized in that the calculation unit extracts the probability value a determined similarity value between the recorded object sub-area (11) and at least one reference vehicle sub-area (15, 16) and a number of related reference vehicle sub-areas (15, 16) determine which reference vehicle sub-areas (15, 16) have a similarity value to a sub-area of ​​the recorded object sub-area (11) that exceeds a similarity limit.

3. Method according to one of claims 1 to 2, characterized in that the computing unit calculates the probability value from a determined similarity value between the recorded object sub-area (11) and at least one reference vehicle sub-area (15, 16) and a determined area fraction of at least one reference vehicle sub-area (15, 16) to the area of ​​the recorded object sub-area (11) and / or to a reference vehicle, which reference vehicle sub-area (15, 16) exhibit a similarity value exceeding a similarity limit to a sub-area of ​​the recorded object sub-area (11).

4. Method according to one of claims 1 to 3, characterized in that the computing unit determines a first distance between the video camera (6) or the lidar or radar and a first truck (1), with the computing unit an image reference size of the first truck (1) in the image is determined from the first distance and a reference size of a reference vehicle, The processing unit determines a first difference value between the image reference size and the image size of the first truck (1) in the image, either the computing unit interprets the first truck (1) as the reference vehicle if the first difference value is less than or equal to a first difference limit, or the computing unit interprets the first truck (1) as not being a reference vehicle if the first difference value is greater than the first difference limit.

5. Method according to one of claims 1 to 4, characterized in that the computing unit determines a second distance (22) between the video camera (6) or the lidar or radar and a second truck (2), with the computing unit an image reference size of the second truck (2) in the image is determined from the second distance (22) and a reference size of a reference vehicle, with the computing unit a second difference value between the image reference size and the image size of the second truck (2) in the image is determined, either the computing unit interprets the second truck (2) as the reference vehicle if the first difference value is less than or equal to a first difference limit, or the computing unit interprets the first truck (2) as not being a reference vehicle if the first difference value is greater than the second difference limit.

6. Method according to one of claims 1 to 5, characterized in that the object part area and the reference vehicle part area (15, 16) is a lateral part area of ​​a view of a vehicle.

7. Method for classifying the vehicle as a collision vehicle according to one of claims 1 to 6, characterized in that a road 5 is recorded with the video camera 6 and / or a lidar and / or a radar, on which road the vehicle is moving, In the computing unit, at least one driving ban section (10) of road 5 must be defined in at least one image, either the processing unit outputs an alarm signal in at least one image upon a positive detection of a vehicle section within a restricted driving area (10), or the processing unit does not output an alarm signal in at least one image upon a negative detection of a vehicle section within a restricted driving area (10).

8. Method according to claim 1 or claim 7, characterized in that the processing unit in at least two images the speed of the vehicle moving on the driving ban section (10) or of the vehicle moving on the driving ban section (10) in the future and the distance of the vehicle to the video camera (6) or the lidar or the radar is determined, The computing unit calculates a collision time interval (22) from the determined speed and distance until the collision of the vehicle with the camera; the computing unit either issues an alarm signal only if the collision time interval (22) is equal to or below a collision time interval limit. or the computing unit does not issue an alarm signal if the collision time interval (22) exceeds a collision time interval limit.

9. Method according to any one of claims 1 to 8, characterized in that the computing unit includes a radio unit the computing unit couples the radio unit with a selected peripheral device (8) using a coupling signal, Furthermore, either the computing unit sends a control signal to the peripheral device (8), or the peripheral device (8) indicates the reception with a reception indicator, or the computing unit sends the alarm signal to the peripheral device (8).

10. Method according to any one of claims 1 to 10, characterized in that the computing unit a vehicle position (26) of the vehicle in the lane (9) is determined, a point (27) nearest to the vehicle position (26) of the lane edge (12, 13) nearest to the vehicle position (26) is determined, a directional component (28) is determined as a vector, which vector is oriented at a right angle to a tangent of the nearest point (27), either the processing unit outputs an alarm signal in at least one image when a positive determination of the orientation of the direction component (28) to the driving ban section (10) is made, or the computing unit outputs an alarm signal in at least one image if a negative determination of the orientation of the direction component (28) to the driving ban section (10) is obtained.

11. Method according to one of claims 1 to 10, characterized in that the computing unit detects the circumferential edges of the vehicle part area, and the vehicle part area is subdivided into further sub-areas. which sub-areas have a shape adapted to the perimeter edges.

12. Method according to claim 11, characterized in that the sub-areas are compared with reference sub-areas, which reference sub-areas comprise a view of a reference view, wherein, in the case of a similarity exceeding a similarity threshold, the reference view is assigned to the sub-area.

Images