Vehicle object detection
By using video streams from different vehicles to update the verified data of the object detection system, the difficulty of recognition when there is low-quality video stream or obstructed view is solved, and more efficient object detection training and recognition are achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- NINGBO GEELY AUTOMOBILE RES & DEV CO LTD
- Filing Date
- 2020-10-27
- Publication Date
- 2026-07-10
Smart Images
Figure CN114651285B_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the field of detection systems for vehicles. More specifically, it relates to a method, computer program, and system for object detection training. Background Technology
[0002] Object detection by vehicles, such as pedestrian detection, vehicle detection, animal detection, and obstacle detection, is usually based on an algorithm that is adjusted using real-world data to obtain the best possible detection performance.
[0003] Typically, these algorithms involve manual analysis and labeling of real-world data, and therefore adjusting the algorithms can be tedious and costly.
[0004] US Patent 9,158,971 B2 describes a system and method for enabling the generation of object detectors for a category of interest. The method includes identifying seed objects in frames of a video sequence using a pre-trained general detector for that category. Appearance models of individual seed objects are iteratively learned using other frames that have identified the seed objects.
[0005] However, when the video stream quality is poor or the view is partially obstructed, the algorithm may have difficulty determining whether the object contained in the video stream is actually the object to be detected.
[0006] Therefore, there is a need for an improved method and system for object detection by vehicles. Summary of the Invention
[0007] The purpose of this disclosure is to provide a method, system, and computer program product that avoids or at least mitigates the problems mentioned above. This purpose is achieved, at least in part, by the features of the independent claims.
[0008] It should be emphasized that, when used in this specification, the term "comprising / including…" (which may be replaced by "containing / including…") is used to specify the presence of a declared feature, integral, step, or component, but does not exclude the presence or addition of one or more other features, integrals, steps, components, or combinations thereof. As used herein, the singular forms "a," "an," and "the" are also intended to include the plural forms, unless the context clearly indicates otherwise.
[0009] Generally, the devices or systems mentioned herein should be understood as physical products, such as equipment. A physical product may include one or more components, such as control circuitry in the form of one or more controllers, one or more processors, etc.
[0010] The first aspect is a method for training an object detection system for vehicles. The object detection system includes verified object detection data (VODD) for one or more objects. The method includes the following steps:
[0011] • Acquire the first video stream containing video images of the region with object detection data.
[0012] • Objects in a region are identified by recognizing one or more first object detection data points in a first video stream that correspond to verified object detection data.
[0013] • Determine the location of the object within the area.
[0014] • Acquire a second video stream containing video images of the region with object detection data.
[0015] • Identify second object detection data from a second video stream at the object's location and determine whether the second object detection data is unverified object detection data.
[0016] If the second object detection data for the identified object is unverified object detection data, then the verified object detection data of the object detection system is updated, thereby training the object detection system to identify patterns in the unverified object detection data.
[0017] The advantage of the above is that, by using video streams from different vehicles covering the same area for object detection training, the training of detection algorithms for, for example, vehicle-based object detection systems becomes more reliable in terms of object detection. Therefore, if a vehicle has confirmed that it has detected, for example, a pedestrian, video streams from other vehicles covering the same area but potentially unable to verify the object can be used to update the object detection system, training it to better identify / detect objects.
[0018] Another advantage of the above implementation is that object data collected (aggregated) from multiple vehicles and covering the same area and location where the object has been verified can preferably be used to update the object detection system. That is, if there is verified object detection data from one vehicle that clearly identifies the object at the same area and location, but data from another vehicle does not clearly show the object (i.e., unverified object detection data), this data can preferably be used to update the object detection system. By using object detection data that includes unverified object detection data (i.e., data that does not confirm actually depicts the object), the system can be trained to identify objects even if the image data is incomplete or the line of sight is obstructed. This is based on the consideration that the system knows the confirmed object is at the location based on a first video stream including video images with verified object detection data. Therefore, by updating the verified object detection data of the object detection system if it is determined that the second object detection data is unverified object detection data, a better training algorithm can be developed compared to training using only verified images of the object, because the system can be trained to recognize patterns in unverified data.
[0019] In some implementations, the step of acquiring a first video stream containing video images of a region with object detection data includes receiving the first video stream from a first vehicle.
[0020] The advantage of the above implementation method is that it can easily collect real-time data of the area.
[0021] In some implementations, the step of acquiring a second video stream containing video images of a region with object detection data includes receiving the second video stream from a second vehicle.
[0022] The advantage of the above implementation is that data can be easily collected over a region. By collecting data from a second vehicle in the same region, the object detection system can receive diverse data in the same area, which can be used to update the system. Compared to receiving data from only one vehicle, diverse data provides better training and higher granularity.
[0023] In some embodiments, the method may further include storing a second video stream at a second vehicle, the second video stream comprising video images with unverified object detection data.
[0024] The advantage of the above implementation is that it can reduce data storage, because it is not necessary to store the entire video stream. Instead, the video stream that can be used as learning material (i.e., including video images with unverified object detection data) can be stored locally and used at a later time.
[0025] In some implementations, the method may further include determining timestamps of the first video stream and the second video stream, as well as the vehicle location, wherein the timestamps indicate when the respective video streams were acquired, and wherein the vehicle location indicates the geographical location from which the first video stream and the second video stream were acquired.
[0026] The advantage of the above implementation is that it can consider only video streams that have been acquired / recorded at valid time points. That is, in some implementations, vehicles with video streams that cover object areas but were recorded at different time points than video streams with identified valid object detection data can be disregarded.
[0027] Another advantage of the above implementation is that the geographic location and / or orientation of the vehicle recording the video stream is determined and taken into account, so as to more easily determine the location of the object, and in some implementations, it is further determined which video streams should be considered for use in training the system.
[0028] In some implementations, determining whether the second object detection data is unverified object detection data includes associating the second object detection data with verified object detection data and determining a confidence value for the second object detection data based on the correlation, wherein if the confidence value is determined to be lower than a confidence threshold, the second object detection data is determined to be unverified object detection data.
[0029] The advantage of the above implementation is that it determines the correlation (degree of correlation) between the verified object detection data from the system or the first video stream and the object detection data from the second video stream, such as the confidence value, which makes it possible to quickly determine whether the object detection data from the second video stream is valid or invalid.
[0030] In some implementations, the verified object detection data associated with the second object detection data is the verified object detection data of the first video stream.
[0031] The advantage of the above implementation is that this correlation operation is performed between verified data covering the same area, thus making it possible to verify the same object. Therefore, the object detection system can be updated based on the content of the first and second video streams.
[0032] It should be noted that the phrase “update the object detection system” can mean updating / training the detection algorithm of the object detection system so that it can improve object detection through self-learning.
[0033] In some implementations, the step of determining the location of an object includes determining the distance and angle of the object relative to a first vehicle configured to acquire a first video stream containing video images of the area with object detection data.
[0034] The advantage of the above implementation method is that the location of the detection object can be easily determined.
[0035] In some implementations, the step of identifying objects in the area further includes identifying the object type of the object as one or more of a person, vehicle, stationary object, moving object, or animal.
[0036] The advantage of the above-described embodiments is that they can detect several different types of objects, such as people, including pedestrians or cyclists, balance bike riders, scooter riders, children, and people in electric wheelchairs; or vehicles, such as trucks, other cars, trailers, motorcycles, and agricultural vehicles such as tractors and combine harvesters; fixed objects, such as houses, rocks, trees, walls, and signs; moving objects, such as strollers, trolleys, wheelchairs, skateboards, shopping carts, and trucks; or animals, such as dogs, cats, horses, reindeer, wild boars, and rabbits. Of course, these are merely examples, and other types of people, vehicles, fixed objects, moving objects, or animals can be detected using the embodiments disclosed herein.
[0037] In some implementations, the object type can be related to free space such as the background. That is, there is no object / object type to be detected.
[0038] One advantage of probing free space is that object detection systems can train themselves to determine when an object to be detected actually exists and when no object to be detected exists.
[0039] In some implementations, the step of acquiring a second video stream including a video image of the area with object detection data includes identifying a vehicle that records a corresponding video stream of a video image including the area with object detection data, and requesting to receive the corresponding video stream.
[0040] The advantage of the above implementation is that it can collect a larger amount of video data covering the area. Therefore, data covering the same location at different angles and distances can be used to train and update the object detection system, thereby achieving high-granularity and more reliable object detection.
[0041] In some implementations, a second video stream includes video images of a region with object detection data, including unverified object detection data at the location of the object.
[0042] The advantage of the above implementation is that if video streams from other vehicles covering the desired location and area include unverified detection data (i.e., invalid if the captured data actually includes verified objects), they are preferably requested. By utilizing unverified object data, the training algorithm of the object detection system can be improved. Unverified object data can be correlated (correlated) with verified object data included, for example, in the first video stream, thus determining that the unverified data is actually valid data that can be used to train the system. For example, correlation can be performed by pattern recognition or by determining confidence values indicating the degree of matching between data points, or the probability that the data includes the same object.
[0043] The second aspect is an object detection system, which includes a control unit and verified object detection data for one or more objects. The control unit is configured to perform the following steps:
[0044] • Acquire the first video stream containing video images of the region with object detection data.
[0045] • Objects in a region are identified by recognizing one or more first object detection data points in a first video stream that correspond to verified object detection data.
[0046] • Determine the location of the object within the area.
[0047] • Acquire a second video stream containing video images of the region with object detection data.
[0048] • Identify second object detection data from a second video stream at the object's location and determine whether the second object detection data is unverified object detection data.
[0049] If the second object detection data for the identified object is unverified object detection data, then the verified object detection data of the object detection system is updated, thereby training the object detection system to identify patterns in the unverified object detection data.
[0050] The advantage of the above is that, by using video streams from different vehicles covering the same area for object detection, training the vehicle-based object detection system becomes more reliable in terms of object detection. Therefore, if one vehicle has confirmed that it has recorded, for example, a pedestrian, video streams from other vehicles recording the same area can be used to update the object detection system, providing more and different video images of detected objects.
[0051] Another advantage of the above implementation is that by identifying and using object detection data, including unverified object detection data (i.e., data that has not been confirmed to actually depict the object), the system can be trained to recognize objects even if the video image data is incomplete or the line of sight is obstructed, because the system will know that the confirmed object is located at that position. Therefore, a better training algorithm can be developed compared to training using only confirmed images of the object.
[0052] In some implementations, the object detection system is included in the vehicle.
[0053] In some implementations, the object detection system is included in a remote server.
[0054] In some implementations, the object detection system includes a system comprising several units. These units may be included, for example, in a vehicle and a server. In some implementations, the system may be included only in a vehicle.
[0055] In some implementations, the control unit is configured to connect to at least first and second vehicles and receive video streams from the at least first and second vehicles, the video streams including video images with object detection data.
[0056] The advantage of the above implementation is that data can be easily collected over a region. By collecting data from more than one vehicle in the same region, the object detection system can receive diverse data covering the same area, which can be used to update the system. Compared to receiving data from only one vehicle, diverse data provides better training and higher granularity.
[0057] In some implementations, the control unit is configured to store a second video stream at a second vehicle, the second video stream comprising video images with unverified object detection data.
[0058] The advantage of the above implementation is that data storage can be reduced because it is not necessary to store the entire video stream. Instead, video streams that can be used as learning material (i.e., including unverified object detection data) can be stored locally and used at a later time.
[0059] In some implementations, the control unit is configured to determine the timestamps of the first and second video streams and the vehicle location, wherein the timestamps indicate when the respective video streams were acquired, and the vehicle location indicates the geographical location from which the first and second video streams were acquired.
[0060] The advantage of the above implementation is that it can consider only video streams that have been acquired / recorded at valid time points. That is, vehicles with video streams that cover object areas but were recorded at different time points than video streams with identified valid object detection data can be disregarded.
[0061] Another advantage of the above implementation is that the geographic location and / or orientation of the vehicle recording the video stream is determined and taken into account, so as to more easily determine the location of the object, and in some implementations, it is determined which video streams should be considered for use in training the system.
[0062] In some implementations, the control unit is configured to determine whether the second object detection data is unverified object detection data by associating the second object detection data with verified object detection data and determining a confidence value of the second object detection data based on the correlation, wherein if the confidence value is determined to be below a confidence threshold, the second object detection data is determined to be unverified object detection data.
[0063] The advantage of the above implementation is that by determining the correlation (degree of correlation) between the verified object detection data and the object detection data in the video stream, such as the confidence value, it is possible to quickly determine whether the object detection data in the video stream is valid or invalid.
[0064] In some implementations, the control unit is configured to recognize second object detection data, which includes unverified object detection data from a second video stream at the location of the object.
[0065] In some implementations, the step of identifying objects in the area further includes identifying the object type of the object as one or more of a person, vehicle, stationary object, moving object, or animal.
[0066] The advantage of the above-described embodiments is that they can detect several different types of objects, such as people, including pedestrians or cyclists, balance bike riders, scooter riders, children, and people driving motor vehicles; or vehicles, such as trucks, other cars, trailers, motorcycles, and agricultural vehicles such as tractors and combine harvesters; fixed objects, such as houses, rocks, trees, walls, and signs; moving objects, such as strollers, trolleys, wheelchairs, skateboards, shopping carts, and trucks; or animals, such as dogs, cats, horses, reindeer, wild boars, and rabbits. Of course, these are merely examples, and other types of people, vehicles, fixed objects, moving objects, or animals can be detected using the embodiments disclosed herein.
[0067] In some implementations, the control unit is configured to identify the vehicle recording the corresponding video stream and request to receive the corresponding video stream, which includes a video image with object detection data for the area.
[0068] The advantage of the above implementation is that it can collect a larger amount of video data covering the area. Therefore, data covering the same location at different angles and distances can be used to train and update the object detection system, thereby achieving high-granularity and more reliable object detection.
[0069] In some implementations, the requested video stream includes video images of the region with object detection data, which includes unverified object detection data associated with the location of the object.
[0070] The advantage of the above implementation is that object data associated with the location of an object (i.e., video streams that cover that location) but not yet verified as including verified object data (i.e., objects that should be detected, such as pedestrians) can be verified as including the object based on object data obtained from another video stream that has already verified the object. Therefore, this provides the training algorithm with better granularity and more reliable object detection.
[0071] The third aspect is a computer program that includes instructions that, when executed by a computer, cause the computer to perform the method according to the third aspect.
[0072] In some implementations, any of the above aspects may also have the same or corresponding features as any of the various features described above for any of the other aspects.
[0073] Other features and advantages of the invention will become apparent when examined in light of the appended claims and the following description. Those skilled in the art will recognize that different features of this disclosure can be combined to create embodiments other than those expressly described above and below, without departing from the scope of this disclosure. Attached Figure Description
[0074] The present disclosure will now be described in detail with reference to the accompanying drawings, in which...
[0075] Figure 1 A flowchart illustrating method steps according to some embodiments is shown.
[0076] Figure 2 An exemplary detection scenario according to some embodiments is illustrated schematically.
[0077] Figure 3 A block diagram of an exemplary system according to some embodiments is shown, and
[0078] Figure 4 A block diagram of an exemplary computer program according to some embodiments is shown. Detailed Implementation
[0079] Various aspects of this disclosure will now be described in conjunction with the accompanying drawings, in order to illustrate and not limit the disclosure, wherein similar reference numerals denote similar elements, and variations of the described aspects are not limited to the embodiments specifically shown, but are other variations that can be applied to the disclosure.
[0080] Figure 1 An exemplary method 1 according to some embodiments is shown. Method 1 is used to train an object detection system 10 for vehicles 100, 31, 32, 33, 34. The object detection system 10 includes verified object detection data (VODD) of one or more objects, i.e., object data confirming that the object data depicts one or more desired objects to be detected.
[0081] Method 1 begins in step S1 by acquiring a first video stream VS1, which includes video images of region 21 with object detection data ODD. The object detection data consists of video images captured in the video stream VS1 of the vehicle's direct environment. For example, the direct environment can be a radius of 5 meters, 10 meters, 50 meters, 100 meters, 400 meters, 600 meters, or larger extending from the vehicle. In step S2, the method continues to identify objects 22 in region 21 of the vehicle's direct environment by identifying one or more first object detection data ODD1 in the first video stream VS1 corresponding to the verified object detection data ODD.
[0082] Verified object detection data can be stored, for example, in the object detection system and include a database of verified objects. The verified object data can, for example, correspond to multiple images depicting different objects in different settings, which helps the object detection system learn and recognize (i.e., be trained for detection) and thus detect objects captured in the video stream.
[0083] Alternatively or additionally, in some embodiments, the verified object detection data is a detection algorithm that analyzes (e.g., by applying pattern recognition) the content of the object detection data of the video stream to determine whether the content includes, for example, pixels that form the object to be detected.
[0084] To determine whether object detection data from a first video stream is validated object detection data, the acquired first object detection data can be correlated with validated object detection data from an object detection system. This correlation may include, for example, comparing the video image content of the first video stream with the valid object detection data for similarities or differences, and determining a correlation result or confidence value indicating the probability that the first object detection data is valid. If the correlation value or confidence result indicates that the first object detection data includes, for example, a video image of a person, with a probability greater than 60%, then the first object detection data can be marked as or determined to be valid object detection data. It should be noted that 60% is just an example, and higher or lower values are also possible.
[0085] Then, in step S3, method 1 continues to determine the location 23 of object 22 within region 21. In some embodiments, determining the location 23 of object 22 may further include attaching a timestamp label to the location 23, i.e., determining the timestamp of the video stream. The timestamp enables object detection system 10 to collect data collected within a predetermined time range. For example, for moving objects such as pedestrians, cyclists, vehicles, etc., collecting video streams taken hours or days later in the same area may not be meaningful. However, in some scenarios where objects have been detected at a verification location for a longer period of time (e.g., if the object is stationary, or if motion patterns have been detected, such as commuters detected daily at approximately the same time and in the same area), collecting video streams covering that area for a longer period of time may also be beneficial.
[0086] In some embodiments, determining the location 23 of object 22 may alternatively or additionally include determining the vehicle location and / or vehicle orientation of the vehicle recording the corresponding video stream. By associating the video stream with vehicle location and orientation, more information can be considered when determining the object location, which can also help other vehicles in the area to pinpoint the object's location. Vehicle location is also important when determining whether video streams should be collected for training the object detection system.
[0087] Method 1 includes acquiring a second video stream VS2 in step S4. The second video stream VS2 includes video images of region 21 with object detection data (ODD). As described above, the object detection data consists of video images of the vehicle's immediate environment.
[0088] Therefore, at least two video streams covering the same area are obtained.
[0089] In some embodiments, timestamps are determined for both the first video stream and the second video stream.
[0090] Then, in step S5, the method includes identifying second object detection data ODD2 of the second video stream VS2 at location 23 of object 22, and determining whether the second object detection data ODD2 is unverified object detection data UODD. Therefore, method 1 may include analyzing the second video stream to consider video images including the object location.
[0091] In some embodiments, step S51 of determining whether the second object probe data ODD2 is unverified object probe data UODD may include associating the second object probe data ODD2 with verified object probe data VODD, and determining a confidence value for the second object probe data ODD2 based on the correlation. If the confidence value is determined to be lower than a confidence threshold, then the second object probe data ODD2 is determined to be unverified object probe data UODD.
[0092] Therefore, method 1 may include identifying or determining that at least one vehicle 100, 31, 32, 33, 34 (e.g., a second vehicle) cannot currently verify that what is seen at location 23 of object 22 in the object detection data ODD of the second video stream VS2 is actually object 22. Therefore, the second video stream VS2 includes unverified object detection data UODD associated with location 23 of object 22.
[0093] In some embodiments, the second object detection data ODD2 may be correlated (degree of correlation) with the verified object detection data VODD of the first video stream VS1.
[0094] Therefore, the verified object detection data VODD of the first stream VS1 can be used to determine whether the second object detection data ODD2 is valid or invalid. If the second object detection data ODD2 is determined to be invalid object detection data UODD based on its correlation with the first video stream VS1, then the object detection system 10 can use the invalid object detection data UODD of the second stream VS2 to train itself to find patterns and better identify objects. This may be effective because the first video stream VS1 includes verified object detection data VODD on the object 22 at a defined location. The second video stream VS2, covering the same region 21 and location 23, should therefore also see and be able to verify the object 22. However, for some reason, the second video stream VS2 may include poor quality or be partially occluded, and only a part of the object may be identifiable, but not enough to perform correlation or pattern detection to form a valid object detection. In this case, the second video stream VS2 may still show the object 22, but the algorithm of the object detection system 10 cannot verify it. Therefore, based on the fact that the first video stream VS1 includes verified object detection data VODD of object 22, the object detection system 10 can use unverified object detection data UODD to train itself to identify objects, and thus the same may be true for the second video stream VS2.
[0095] In some embodiments, unverified data may be identified simply because the object may have moved quickly off the road, walked behind a tree, or been obscured by a passing vehicle, while in the case of a second vehicle, the object is simply not visible. In such cases, the unverified data can still be used to train the system. For example, learning to identify background data (free space), i.e., scenes where the object to be detected does not exist, can be beneficial to the algorithm. Therefore, the object detection system can be trained to distinguish between objects and non-objects based on unverified object detection data (which may include both unverified objects and non-existent objects).
[0096] When at least two video streams have been captured and the object detection data associated with each corresponding video stream has been analyzed / identified, the method continues in step S6. If it is determined that the second object detection data ODD2 of object 22 is unverified object detection data UODD, the verified object detection data VODD of the object detection system 10 is updated. In some embodiments, the method may further include updating the verified object detection data VODD with unverified object detection data UODD based on the verified object detection data VODD of the first video stream VS1.
[0097] In some embodiments, step S1 of method 1, which includes acquiring a first video stream VS1, may optionally further include receiving the first video stream VS1 from the first vehicle in step S11. The first video stream VS1 includes a video image of the region having object detection data ODD.
[0098] The object detection system 10 may be located in one or more vehicles 100, 31, 32, 33, 34, but may also wirelessly communicate with and / or include a server 200 in, for example, a network cloud. The server 200 may collect video streams, for example, from the first vehicle 100, 31 and from other vehicles 100, 31, 32, 33, 34, and perform training of the object detection system 10 based on the received video streams and the object detection data ODD included therein. The server 200 may then update the validated object detection data ODD (e.g., an object detection algorithm) and push the update to the respective object detection systems 10 of the vehicles 100, 31, 32, 33, 34 via the network cloud.
[0099] In some embodiments, step S4 of method 1, which includes acquiring the first video stream VS1, may optionally further include receiving a second video stream VS2 from the second vehicles 100, 32, 33, 34 in step S41. The second video stream VS2 includes a video image having object detection data ODD of the region.
[0100] Therefore, in some embodiments, the server described above can obtain video streams from the second vehicle. In some embodiments, the first vehicles 100, 31 can be configured to obtain a second video VS2 stream from the second vehicles 100, 32, 33, 34. In some embodiments, the first vehicle can send the first video stream, the first video stream and the second video stream, or only the second video stream to the server for updating the object detection system.
[0101] Similarly, in some embodiments, the second vehicle can obtain the first video stream VS1 from the first vehicle and can transmit the first video stream, the second video stream, or both the first and second video streams to the server. In some embodiments, the object detection systems of the corresponding first and second vehicles can perform object detection data updates locally in each vehicle without involving the external server 200.
[0102] In some embodiments, the second vehicle may also store a second video stream VS2 including unverified object detection data (UODD). By locally storing the video stream UODD including video images with unverified object detection data at the second vehicle (or at the vehicle capturing the video stream including video images with unverified object detection data), data storage and storage space can be improved. The vehicle and / or server may prioritize storing the video stream including unverified object detection data to facilitate the storage of verified object detection data. The stored stream can then be used to train the algorithm / object detection system at a later stage.
[0103] In some embodiments, the step of determining the object location in method 1 (S3) may optionally include step S31, which includes determining distances D1, D2, D3 and angles α1, α2, α3 relative to the object with respect to a first vehicle configured to acquire a first video stream VS1, which includes video images having object detection data ODD for the region. Therefore, method 1 enables the vehicle to determine the location 23 of the detected object 22, which can then be used when collecting (summarizing) and analyzing other video streams of region 21 to detect or not detect the object 22 in other video streams, and thus trains the object detection system to identify unverified object detection data UODD as verified object detection data VODD.
[0104] In some embodiments, step S2 in method 1 for identifying objects in the region (step S21) may optionally further include step S22 of identifying the object type of object 22 as one or more of a person, vehicle, stationary object, mobile object, or animal.
[0105] By identifying the object type, one can limit how much object detection data should be collected and for how long. If the object type is identified as a person, vehicle, moving object, or animal, collecting video streams within a shorter period may be more advantageous compared to collecting data for a stationary object. Furthermore, for stationary objects, it is more interesting to prioritize collecting unverified object detection data to train the object detection system to better identify objects. For example, if a first vehicle determines it has detected a stationary object, its video stream, including video images with the first object detection data, may not be of much value for training the system. If a second vehicle arrives at an area where a stationary object is assumed to be present, but fails to detect it in the second video stream even though the video stream covers the area and the assumed location of the object, then the second video stream, including video images with unverified object detection data, may be more interesting for training the system.
[0106] In some embodiments, step S4 of method 1, which includes acquiring a second video stream VS2 comprising a video image having object detection data ODD of the region, may optionally include step S42 of identifying vehicles 100, 31, 32, 33, 34 recording the corresponding video stream VS and requesting to receive the corresponding video stream VS, which includes a video image having object detection data ODD of the region. Optional step S7 of method 1 may also include recording, and in some embodiments includes requesting. In some embodiments, recording and requesting may include two different method steps, for example, step S7 of recording and step S8 of requesting. Figure 1 (Not shown in the image).
[0107] In some embodiments, the second video stream VS2, which includes a video image of region 21 with object detection data ODD, includes unverified object detection data UODD at location 23 of object 22. As described above, the object detection data of the second video stream may, for example, include a blurred video image or a partial video image, which makes it impossible to confirm whether the object detection data associated with the object's location actually shows the object, and therefore the object detection data includes unverified object detection data at the object's location.
[0108] Method 1, as described above, is executed in a series of method steps in a verification order. It should be noted that in some embodiments, the order of the steps may differ from the order described above. For example, in some embodiments, steps S4 and S5 may be interchanged with steps S1 and S2. Method 1, as described above, defines a scenario where a first vehicle has detected an object, and the object detection system of the first vehicle has verified it as verified object detection data. That is, the object detection system of the first vehicle has verified that it has detected, for example, a person, vehicle, stationary object, animal, etc., and can query other vehicles in the area whether they have seen the same object. If the other vehicles in the area determine that they have not seen it, i.e., the object detection data associated with the object location and included in their respective video streams by these other vehicles is unverified object detection data, then the video image streams of these other vehicles, including the unverified object data, can be used to update the verified object detection data and thus train the system.
[0109] However, in some embodiments, the vehicle may acquire a video stream of the area, which includes video images with object detection data. The vehicle's object detection system may react to the presence of an object in the object detection data but cannot verify what it is. The video stream may be of poor quality, for example due to weather conditions (rain can, for example, result in blurry or poorly rendered video images that are difficult to resolve), or the video images may be partially obscured, blurry, or for any other reason may not provide object detection data that can be matched with verified object detection data. For example, when determining whether the video stream includes video images with verified object detection data, the system may determine that the confidence value of the object detection data indicates a 42% probability that the data shows a person. This probability is not high enough to safely assume that the object is a person, but high enough to determine that there may be a person. Therefore, verification is required. The vehicle (or the object detection system included in the vehicle) can then query other vehicles in the area whether their video streams have captured verified object detection data associated with the location of the unverified object detection data. The vehicle can receive video streams from other vehicles, including verified object detection data, and then update the object detection system based on the acquired / received unverified and verified object detection data.
[0110] In some embodiments, vehicles may send their respective video streams to an external server to update the object detection system.
[0111] Figure 2 Example scenarios in which the above methods and embodiments can be applied are shown.
[0112] exist Figure 2 There are four motor vehicles 100 on the road. Specifically, they are vehicle 31, vehicle 32, vehicle 33, and vehicle 34. Vehicle 31 may be, for example, a combination of... Figure 1 The first vehicle is described. Similarly, the second vehicle 32, the third vehicle 33, and the fourth vehicle 34 can be combined as follows. Figure 1 The second vehicle mentioned above. Figure 2 In the text, vehicles 31, 32, 33 and 34 are exemplified as cars, but this should only be considered as an example, as other types of vehicles are also possible, such as trucks, motorcycles, recreational vehicles, buses, etc.
[0113] The first vehicle 31, the second vehicle 32, the third vehicle 33, and the fourth vehicle 34 are all equipped with corresponding object detection systems 10. Each corresponding vehicle's object detection system 10 provides corresponding video streams VS1, VS2, VS3, and VS4 for acquiring area 21 (VS4 is not included in the following explanation). Figure 2 (As shown in the diagram). In region 21, object 22 exists at position 23. Although object 22 is in... Figure 2 The image shown is of a person riding a bicycle, but this is just an example (for simplicity, although bicycles are also considered vehicles, but...). Figure 2 The cyclists were not labeled as vehicles, but rather as objects within this disclosure. (For example, in conjunction with...) Figure 1 The object 22 can also be any other type of object, such as a pedestrian / person, vehicle, stationary object, or animal. Furthermore, the object 22 can be located in a location other than the middle of the road, such as on a sidewalk or similar location.
[0114] In some embodiments, the vehicle object detection system 10 can be configured to send the recorded video streams VS1-VS4 to... Figure 2 The example in the text is an external server 200 in the cloud.
[0115] It should be noted that Figure 2 The video stream illustrated is exemplary and can have other ranges. For example, the video stream could cover a semicircle spanning 180 degrees from the object detection system. The range and coverage of the video stream can be specified by the type of unit recording the stream. Some cameras can record, for example, a full 360-degree circle, while others can record portions of the circle within angular ranges such as 270 degrees, 180 degrees, 90 degrees, 60 degrees, etc. It should also be noted that the circular range is an example, and ranges of other shapes are conceivable. Furthermore, the length of this range can also be varied depending on the limitations of the recording camera / unit.
[0116] Therefore, the object detection system 10 of the first vehicle 31 can acquire a first video stream VS1, which includes video images of region 21 with object detection data. Then, the object detection system 10 of the first vehicle 31 can identify the object 22 in region 21 by identifying one or more first object detection data ODD1 in the first video stream VS1 that correspond to the verified object detection data VODD.
[0117] For example, the one or more first object detection data ODD1 can clearly show a person on a bicycle (i.e., a cyclist) on the road. The object detection system 10 of the first vehicle 31 may include a database of verified object detection data, and when the one or more first object detection data ODD1 are compared with the verified object detection data, a clear match is found, and the object detection system of the first vehicle can then determine that it has seen / detected object 22. In some embodiments, the verified object detection data of the object detection system may alternatively or additionally be an algorithm that instructs the object detection system what to look for in the object detection data in order to determine whether the object detection data is valid. The algorithm may, for example, include a series of patterns that should be satisfied when analyzing the pixels of the video stream to determine valid or invalid object detection data.
[0118] Then, the object detection system 10 of the first vehicle can determine the position 23 of the object 22 in the region 21. The first detection system can, for example, determine the distance D1 and angle α1 to the object 22 relative to the first vehicle 31. In some embodiments, the object detection system 10 can also add timestamp tags to the first video stream VS1 of the region 21 and the object 22.
[0119] In some embodiments, the object detection system 10 may be configured to determine timestamps of a first video stream and a second video stream, the timestamps indicating when the respective video streams were acquired.
[0120] In some embodiments, the object detection system 10 can be configured to determine the vehicle location and / or orientation of a vehicle recording a corresponding video stream. (Additionally or alternatively, based on timestamps) vehicle location tags can be added to the video stream. The vehicle location can be a geographic location representing the vehicle's physical location and can be determined using, for example, GPS.
[0121] When the object detection system 10 of the first vehicle has determined that it has seen an object, it can query other vehicles in the area, such as the second vehicle 32, the third vehicle 33, and the fourth vehicle 34, to inquire whether they have a video stream in that area and whether they have seen the object 22. The object detection system of the first vehicle 31 can, for example, identify other vehicles recording video streams that include video images of the area with object detection data. In some embodiments, the second vehicle 32 and the third vehicle 33 may respond (answer or reply), while the fourth vehicle 34 may not respond because it does not capture an image of area 21 (and its video stream VS4 is therefore not in the area). Figure 2(As shown in the diagram). Whether the fourth vehicle should respond can be indicated by a timestamp. For example, if only the live video stream is of interest, then the video stream of the fourth vehicle is not of interest because it did not take a picture (capture an image) of area 21 at the requested time.
[0122] In some embodiments, the second vehicle 32 and the third vehicle 33 may only respond if they determine that the object detection data of their respective respective video streams VS2 and VS3 includes unverified object detection data UODD associated with the location 23 of the alleged object 22 in region 21. The second and third vehicles may attempt to identify the object 22 by, for example, also determining the distances D2, D3 and angles α2, α3 to the alleged object 22 relative to the second and third vehicles respectively, and determine whether the object detection data associated with the location is unverified object detection data UODD or verified object detection data Vodd. If the second object detection data ODD2 and / or the third object detection data ODD3 of the second video stream VS2 and / or the third video stream VS3 is determined to be unverified object detection data UODD, then the second and / or third video streams including video images with unverified object detection data UODD can be used to update and thereby train the object detection system. In some embodiments, the object detection system 10 of the first vehicle may acquire all video streams of region 21 (in Figure 2 The object detection system 10 for the first vehicle can also update the verified object detection data using object detection data acquired from the second video stream and / or the third video stream. This is done by attempting to identify object 22 at position 23 within region 21 of video streams VS2 and VS3.
[0123] In some embodiments, video stream VS1, video stream VS2, and video stream VS3 may be provided by an external server 200 (e.g., a server in the cloud, such as...). Figure 2 (As shown) is obtained via a network connection such as the Internet. If it is determined that the obtained unverified object detection data UODD may be related to the verified object detection data VODD of that area, for example, from the first video stream, then the external server can perform an update of the verified object detection data (and thus train the algorithm of the object detection system, for example, that performs the detection), and then push the update to the object detection systems 10 of the first vehicle, the second vehicle, the third vehicle, and the fourth vehicle, so that each object detection system 10 can be trained to identify objects in various settings.
[0124] The above describes updating the object detection system based on verified object detection data detected by a first vehicle (i.e., the first vehicle knows what object it is looking at), and then collecting additional unverified object detection data from vehicles that cannot verify that they see the object at the same location. This collected data is then used to update / train the system. For example, blurry video images, low-resolution video images, or partial video images of the object (i.e., unverified object detection data) can be associated with verified object detection data. For instance, the first vehicle might detect a pedestrian at a distance of 20 meters. The detection result is reliable and based on a pedestrian video image with, for example, a size of 200*50 pixels. The second vehicle might be further away and might see the same area, but at a distance of 250 meters. In this case, the pedestrian might be captured (image captured) at 10*3 pixels, which gives a resolution lower than that the first vehicle can acquire, and therefore the second vehicle might have more difficulty knowing what it is looking at; thus, the video stream from the second vehicle is valuable for training the system.
[0125] Video images can be compared and matched for details to confirm that unverified object detection data from a second vehicle is actually verified object detection data, and to identify patterns in the unverified object detection data that can be used in the future to determine whether object detection data is verified or unverified. The next time a similar blurry or incomplete video image of an object is captured in the video stream, the updated / trained object detection system can determine that the video image is verified object detection data based on the updated training algorithm of the object detection system.
[0126] It should be noted that the order of the vehicles is exemplary. The first vehicle can be, for example, the second, third, or fourth vehicle, and vice versa.
[0127] Furthermore, in some embodiments, the method may begin when a vehicle determines that it cannot verify that what it actually sees in the video stream of its region is a verified object. For example, a third vehicle 33 may probe unverified object detection data (UODD) in its video stream VS3 at location 23 of region 21. Video stream VS3 may, for example, include a partially occluded video image of object 22. Figure 2 In this scenario, the line of sight from the third vehicle to object 22 is partially obstructed, for example, by the second vehicle 32. If an "association" operation is performed, the confidence value of the object detection data in the video stream might be, for example, 40%. This confidence value might not be high enough to pass the threshold of the validated object detection data, but it is still high enough for the object detection system to determine that there may be an object in the video stream that should be detected.
[0128] Then, after determining that the third video stream VS3 includes unverified object detection data (UODD) associated with / at location 23 of region 21, the third vehicle 33 can identify other vehicles that have recorded corresponding video streams covering region 21. For example, the third vehicle 33 can send an inquiry to other current vehicles asking whether they have detected verified object detection data associated with that location. In some embodiments, the second vehicle 32 and the first vehicle 31 can respond by sending their respective video streams to the third vehicle 33, which include video images with verified object detection data for object 22. The third vehicle can then update the object detection system based on the determined unverified object detection data and possible verified object detection data.
[0129] In some embodiments, the third vehicle 33 may locally update its object detection system 10 and may send the update to other vehicles, thereby updating their respective object detection systems 10 as well. In some embodiments, the third vehicle may transmit acquired unverified object detection data and possibly verified object detection data to an external server, which includes an object detection system and a database and / or algorithm for identifying verified object detection data. The external server can then use the acquired data to train / update the object detection system and may push the update to all object detection systems connected to the server and associated with the vehicles (e.g., the respective object detection systems 10 associated with vehicles 31, 32, 33, and 34).
[0130] Figure 3 A block diagram illustrates an object detection system 10 for a vehicle 100 according to some embodiments. The object detection system 10 may be, for example, a combination of the foregoing... Figures 1 to 2 The object detection system described in any of the above. The first vehicle 100 may be, for example, a combination of... Figures 1 to 2 Any of the vehicles described.
[0131] according to Figure 3 The object detection system 10 includes a control unit 11 (CNTR) and an object detection data module 112 (ODD), which includes verified object detection data 113 (VODD) and unverified object detection data 114 (UODD) for one or more objects.
[0132] In some embodiments, the control unit 11 may include control circuitry. The control unit / control circuitry may include an object detection data module 112 for storing verified object data 113 and unverified object detection data 114, representing object detection data / algorithms. In some embodiments, the control unit may also include a video unit 111 (VID) and a determination unit 115 (DET). In some embodiments, the object detection system 10 may also include an antenna circuit 12 (RX / TX).
[0133] Control unit 11 is configured to execute a first video stream (e.g., a video image containing object detection data of a region) that includes a region. Figure 2 The acquisition of VS1 (similar to step S1 of method 1). The control unit 11 may be configured, for example, to enable the video unit 111 to record and relay the first video stream VS1, and to enable the object detection data module 112 to store the object detection data ODD of the first video stream VS1.
[0134] The control unit can also be configured to identify an object 22 in region 21 by recognizing one or more first object detection data ODD1s of an object 22 in a first video stream VS1 corresponding to the verified object detection data VODD (similar to step S2 of method 1). The control unit 11 can be configured, for example, to enable the ODD module 112 to determine whether the acquired object detection data corresponds to verified object data or unverified object detection data, for example by using an algorithm for object detection and storing the object detection data as verified object detection data 113 or unverified object detection data 114.
[0135] The control unit 11 can be configured to determine the position 23 of the object 22 in the region 21 (similar to step S3 in method 1). The control unit 11 can, for example, be configured to enable the determining module 115 to determine the position 23.
[0136] The control unit 11 can be configured to cause the acquisition of a second video stream VS2, which includes a video image of the region with object detection data (similar to step S4 in method 1). The control unit 11 can, for example, be configured to enable the antenna circuit 12 to receive the second video stream VS2.
[0137] Control unit 11 can be configured to cause the identification of second object detection data ODD2 of the second video stream VS2 at position 23 of object 22 (similar to step S5 of method 1), and to determine whether the second object detection data ODD2 is unverified object detection data UODD. Control unit 11 can, for example, cause object data detection module 112 to analyze the second object detection data ODD2 by means of a stored algorithm for verified object detection data VODD, and / or match the second object detection data ODD2 with stored verified object detection data VODD or with verified object detection data VODD of the first video stream VS1. Control unit 11 can, for example, be configured to cause object detection module 112 to determine whether the second object detection data ODD2 matches verified detection data VODD stored in modules 112, 113. When a mismatch is determined, the second object detection data ODD2 at position 23 of object 22 can be seen and identified as unverified detection data UODD.
[0138] In some embodiments, the control unit 11 can be configured to determine whether the second object detection data ODD2 is unverified object detection data VODD by associating the second object detection data ODD2 with the verified object detection data VODD and determining a confidence value of the second object detection data based on the correlation (degree of correlation or amount of correlation). If the confidence value is determined to be below a confidence threshold, the second object detection data is determined to be unverified object detection data (similar to step S51 of method 1).
[0139] The control unit 11 can be configured to update the verified object detection data 113 of the object detection system 10 if it is determined that the second object detection data of object 22 is unverified object detection data (UODD). The control unit 11 can, for example, cause the object detection module 112 to store the second object detection data in the verified object detection database 113 as verified object detection data and / or update the stored detection algorithm.
[0140] In some embodiments, the control unit 11 is configured to be able to connect to at least the first vehicle and the second vehicle 100, 31, 32, 33, 34 (similar to method 1 and ...). Figure 2 The control unit 11 may be configured, for example, to enable the video module 111 to record the video stream and / or enable the antenna circuit 12 to receive one or more video streams from at least the first vehicle and the second vehicle.
[0141] In some embodiments, the control unit 11 is also configured to store a second video stream VS2, including unverified object detection data UODD, at the second vehicles 100, 32, 33, 34.
[0142] In some embodiments, the control unit 11 is further configured to determine the timestamps of the first video stream VS1 and the second video stream VS2, the timestamps indicating when the respective video streams were acquired.
[0143] In some embodiments, the control unit 11 is configured to identify the object type of an object as one or more of a person, vehicle, stationary object, or animal (similar to step S22 of method 1). The control unit 11 may, for example, enable the ODD module 112 to cooperate with the determination module 115 to determine, and thereby identify, the object type of the object (based on object detection data).
[0144] In some embodiments, the control unit 11 is configured to identify vehicles recording video streams that include video images of a region with object detection data (similar to steps S42 and S7 in method 1). The control unit 11 may, for example, be configured to enable the antenna circuitry 12 to search for and identify other vehicles in the region.
[0145] In some embodiments, the control unit 11 is configured to identify a vehicle recording a video stream that includes an object detection data ODD in region 21, the object detection data ODD including unverified object detection data UODD at location 23 of object 22 (similar to steps S42 and S7 of method 1). The control unit 11 may, for example, be configured to enable the object detection module 112 to determine that the object detection data ODD is unverified object detection data UODD.
[0146] In some embodiments, such as Figure 3 The object detection system 10 may be included in the external server 200. When included in the external server, the object detection system 10 may be configured to communicate with other object detection systems 10 included in vehicles 100, 31, 32, 33, 34 and (wirelessly) connected to the external server 200.
[0147] Figure 4 A computer program is shown that includes instructions, which, when executed by a computer, cause the computer to perform actions in conjunction with the preceding instructions. Figures 1 to 3 The method described by any of them.
[0148] More specifically, Figure 4 In some embodiments, a computer program product on a non-transitory computer-readable medium 400 is shown. Figure 4An example non-transitory computer-readable medium 400 in the form of an optical disc (CD) ROM 400 is shown. A computer program including program instructions is stored on the non-transitory computer-readable medium. The computer program can be loaded into a data processor (PROC, e.g., a data processing circuit or data processing unit) 420, which may, for example, be included in a control unit 410. When loaded into the data processor, the computer program can be stored in a memory (MEM) 430 associated with or included in the data processing unit. According to some embodiments, when the computer program is loaded into the data processor and executed by the data processor, the computer program can cause execution according to, for example... Figures 1 to 3 Method steps of any of the methods exemplified herein or other methods described herein.
[0149] Those skilled in the art will understand that the steps, services, and functions explained herein can be implemented using separate hardware circuitry, software that functions in conjunction with a programmable microprocessor or general-purpose computer, one or more application-specific integrated circuits (ASICs), and / or one or more digital signal processors (DSPs). It will also be understood that, when described from a methodological perspective, this disclosure can also be implemented in one or more processors and one or more memories coupled to said one or more processors, wherein said one or more memories store one or more programs that, when executed by said one or more processors, perform the steps, services, and functions disclosed herein.
[0150] The present disclosure has been presented above with reference to specific embodiments. However, other embodiments besides those described above are possible and are within the scope of this disclosure. Method steps, performed by hardware or software, different from the steps described above, may be provided within the scope of this disclosure. Therefore, according to an exemplary embodiment, a non-transitory computer-readable storage medium is provided storing one or more programs configured to be executable by one or more processors of a system for object detection, the one or more programs including instructions for performing the method according to any of the above embodiments. Alternatively, according to another exemplary embodiment, a cloud computing system may be configured to perform any aspect of the method presented herein. The cloud computing system may include distributed cloud computing resources that collectively perform the presented method aspects under the control of one or more computer program products. Furthermore, the processor may be connected to one or more communication interfaces and / or sensor interfaces for receiving and / or sending data with external entities, such as sensors deployed on the surface of a vehicle, an off-site server, or a cloud-based server.
[0151] The processors (associated with the object detection system) may be any number of hardware components, or include them, for performing data or signal processing or for executing computer code stored in memory. The system may have associated memory, and the memory may be one or more means for storing data and / or computer code to perform or facilitate the various methods described herein. The memory may include volatile or non-volatile memory. The memory may include database components, object code components, script components, or any other type of information structure for supporting the various activities described herein. According to exemplary embodiments, any distributed or local memory device may be used with the systems and methods described herein. According to exemplary embodiments, the memory may be communicatively connected to the processor (e.g., via circuitry or any other wired, wireless, or network connection) and includes computer code for performing one or more processes described herein.
[0152] It should be understood that the above description is exemplary in nature only and is not intended to limit the scope of this disclosure, its application, or use. Although specific examples have been described in the specification and shown in the drawings, those skilled in the art will understand that various changes can be made and equivalents can be substituted for elements therein without departing from the scope of this disclosure as defined in the claims. Furthermore, modifications can be made to adapt a particular situation or material to the teachings of this disclosure without departing from its essential scope. Therefore, this disclosure is not intended to be limited to the specific examples disclosed as the best mode contemplated for implementing the teachings of this disclosure, which are shown in the drawings and described in the specification. Rather, the scope of this disclosure will include any embodiments falling within the scope of the foregoing description and the appended claims. Reference numerals used in the claims should not be construed as limiting the scope of the subject matter protected by the claims; their sole purpose is to facilitate the understanding of the claims.
[0153] Figure Labels
[0154] 1: Method
[0155] 100: Vehicles
[0156] 31: First vehicle
[0157] 32: Second vehicle
[0158] 33: Third vehicle
[0159] 34: The fourth vehicle
[0160] 10: Object Detection System
[0161] 200: External Server
[0162] VS1: First Video Stream
[0163] VS2: Second Video Stream
[0164] VS3: Third Video Stream
[0165] VS4: Fourth Video Stream
[0166] ODD1: First Object Probe Data
[0167] ODD2: Second Object Probe Data
[0168] ODD3: Third Object Detection Data
[0169] VODD: Verified object probe data
[0170] UODD: Unverified Object Probe Data
[0171] D1-D3: Distance from vehicle to object
[0172] α1-α3: Angles of the vehicle relative to the object
[0173] 21: Region
[0174] 22: Object
[0175] 23: Location
[0176] 11: Control Unit, CNTR
[0177] 12: Antenna circuit, RX / TX
[0178] 111: Video Unit, VID
[0179] 112: Object Detection Data Module, ODD
[0180] 113: Verified object detection data
[0181] 114: Unverified object detection data
[0182] 115: Determine module, DET
[0183] 400: Computer Program Products
[0184] 410: Data Processing Unit
[0185] 420: Processor
[0186] 430: Memory
Claims
1. A method for training an object detection system for vehicles, wherein, The object detection system includes verified object detection data (VODD) for one or more objects, and the method includes the following steps: • Step S1: Acquire the first video stream (VS1) containing video images of the region with object detection data (ODD); • Step S2: Identify the object in the region by recognizing one or more first object detection data (ODD1) of the object in the first video stream (VS1) that correspond to the verified object detection data (VODD); • Step S3: Determine the location of the object in the area; • Step S4: Acquire a second video stream (VS2) including a video image of the region with object detection data (ODD); • Step S5: Identify the second object detection data (ODD2) of the second video stream (VS2) at the location of the object, and determine whether the second object detection data (ODD2) is unverified object detection data (UODD); • Step S6: If it is determined that the second object probe data (ODD2) of the object is unverified object probe data (UODD), then update the verified object probe data (VODD) of the object detection system, thereby training the object detection system to identify patterns in the unverified object probe data.
2. The method according to claim 1, wherein, Step S1 of acquiring the first video stream (VS1) containing video images of the region with object detection data (ODD) includes receiving the first video stream (VS1) from the first vehicle.
3. The method according to claim 1 or 2, wherein, Step S4 of acquiring the second video stream (VS2) containing video images of the region with object detection data (ODD) includes receiving the second video stream (VS2) from the second vehicle.
4. The method according to claim 3, further comprising: The second video stream (VS2) including unverified object detection data (UODD) is stored at the second vehicle.
5. The method according to any one of claims 1, 2, and 4, further comprising: The timestamps of the first video stream and the second video stream, as well as the vehicle location, are determined, wherein the timestamps indicate when the corresponding video streams were acquired, and the vehicle location indicates the geographical location at which the first video stream and the second video stream were acquired.
6. The method according to any one of claims 1, 2, and 4, wherein, The steps to determine whether the second object probe data (ODD2) is unverified object probe data (UODD) include: The second object detection data (ODD2) is correlated with the verified object detection data (VODD), and a confidence value of the second object detection data (ODD2) is determined based on the correlation. If the confidence value is determined to be lower than a confidence threshold, the second object detection data (ODD2) is determined to be unverified object detection data (UODD).
7. The method according to any one of claims 1, 2, and 4, wherein, Step S3, which determines the location of the object, includes determining the distance and angle of the object relative to the first vehicle, the first vehicle being configured to acquire the first video stream (VS1) containing video images of the region with object detection data (ODD).
8. The method according to any one of claims 1, 2, and 4, wherein, Step S2, which identifies objects in the region, further includes identifying the object type of the object as one or more of a person, vehicle, stationary object, moving object, or animal.
9. The method according to any one of claims 1, 2, and 4, wherein, Step S4, which involves acquiring the second video stream (VS2) containing video images of the region with object detection data (ODD), includes: The system identifies vehicles whose video images of the area contain object detection data (ODD) and their corresponding video streams (VS), and requests to receive the corresponding video streams (VS).
10. The method according to claim 9, wherein, The second video stream (VS2) including video images of the region with object detection data (ODD) includes unverified object detection data (UODD) associated with the location of the object.
11. An object detection system for a vehicle, wherein, The object detection system includes a control unit and verified object detection data (VODD) for one or more objects. The control unit is configured to perform the following steps: • Step S1: Acquire the first video stream (VS1) containing video images of the region with object detection data (ODD); • Step S2: Identify the object in the region by recognizing one or more first object detection data (ODD1) of the object in the first video stream (VS1) corresponding to the verified object detection data (VODD); • Step S3: Determine the location of the object in the area; • Step S4: Acquire a second video stream (VS2) including a video image of the region with object detection data (ODD); • Step S5: Identify the second object detection data (ODD2) of the second video stream (VS2) at the location of the object, and determine whether the second object detection data (ODD2) is unverified object detection data (UODD); • Step S6: If it is determined that the second object probe data (ODD2) of the object is unverified object probe data (UODD), then update the verified object probe data (VODD) of the object probe system, thereby training the object probe system to identify patterns in the unverified object probe data.
12. The object detection system according to claim 11, wherein, The control unit is configured to connect to and receive video streams (VS) including video images with object detection data (ODD) from at least a first vehicle and a second vehicle.
13. The object detection system according to claim 12, wherein, The control unit is configured to store the second video stream (VS2) including unverified object detection data (UODD) at the second vehicle.
14. The object detection system according to any one of claims 11-13, wherein, The control unit is also configured to determine the timestamps of the first video stream and the second video stream, as well as the vehicle location, wherein the timestamps indicate when the corresponding video stream was acquired, and wherein the vehicle location indicates the geographical location at which the first video stream and the second video stream were acquired.
15. The object detection system according to any one of claims 11-13, wherein, The control unit is configured to determine whether the second object detection data (ODD2) is unverified object detection data (UODD) by correlating the second object detection data (ODD2) with verified object detection data (VODD) and determining a confidence value of the second object detection data (ODD2) based on the correlation, wherein if the confidence value is determined to be lower than a confidence threshold, the second object detection data (ODD2) is determined to be unverified object detection data (UODD).
16. The object detection system according to any one of claims 11-13, wherein, The control unit is configured to: identify the object type of the object as one or more of a person, vehicle, stationary object, mobile object, or animal.
17. The object detection system according to any one of claims 11-13, wherein, The control unit is configured to identify vehicles that record video streams (VS) containing video images of the area with object detection data (ODD).
18. The object detection system according to any one of claims 11-13, wherein, The control unit is configured to: identify a vehicle that records a video stream (VS) containing video images of the area with object detection data (ODD), the object detection data (ODD) including unverified object detection data (UODD) at the location of the object.
19. A computer program comprising instructions that, when executed by a computer, cause the computer to perform the method according to any one of claims 1-10.