Detection method, device, equipment, system and storage medium

Abstract

Disclosed are a detection method, device, system, equipment and storage medium. A first timestamp is added to first data detected by a first type of detection device, the first timestamp is used to characterize the detection time of the first data, and the first data includes object data in a road; a second timestamp is added to second data detected by a second type of detection device, the second timestamp is used to characterize the detection time of the second data, and the second data includes object data in a road; when the difference between the first timestamp and the second timestamp is less than a predetermined threshold, the detection result of the same object is determined based on the first data and the second data. Thus, the data matching problem is solved based on the timestamp, so that the final detection result can characterize the multi-dimensional information of the object.

Description

Technical Field

[0001] The present disclosure relates to the field of data processing, and in particular to a detection method, device, equipment, system and storage medium. Background Art

[0002] In order to obtain multi-dimensional or more accurate data, it is usually necessary to fuse the data collected by multiple types of sensors. For example, when accurately positioning a vehicle or detecting a vehicle collision, camera data and radar data are needed. Camera data is used to detect the size and color of the vehicle, while radar data is used to detect the location of the vehicle in rainy weather. It is usually necessary to fuse the data of these two sensors to more accurately determine the spatiotemporal information of the vehicle.

[0003] When processing the raw data collected by the sensor, a certain amount of processing time is required, and in general, the processing time for the raw data collected by different types of sensors is different. Therefore, when data fusion is performed after processing different types of sensor data for different lengths of time, how to achieve accurate matching to obtain multi-dimensional detection data corresponding to the same detected object is a problem that needs to be solved urgently. Summary of the invention

[0004] An object of the present disclosure is to provide a detection solution to solve the above-mentioned problem.

[0005] According to one aspect of the present disclosure, a detection method is provided, including: adding a first timestamp to first data detected by a first type of detection device, the first timestamp being used to characterize the detection time of the first data, the first data including object data in a road; adding a second timestamp to second data detected by a second type of detection device, the second timestamp being used to characterize the detection time of the second data, the second data including object data in a road; and determining a detection result of the same object based on the first data and the second data when a difference between the first timestamp and the second timestamp is less than a predetermined threshold.

[0006] Optionally, the step of determining the detection result of the same object includes: processing the first data to obtain a first processing result; processing the second data to obtain a second processing result; and determining the detection result of the same object based on the first processing result and the second processing result.

[0007] Optionally, the objects include: vehicles; and / or pedestrians; and / or road facilities.

[0008] Optionally, the first type of detection device is an imaging device, the first data is image data, and the second type of detection device is a radar device, and the second data is radar data.

[0009] Optionally, the step of determining the detection result of the same object includes: processing the image data to determine the appearance information of the object; processing the radar data to determine the position information of the object; and determining the appearance and position information of the same object based on the appearance information and the position information.

[0010] Optionally, the step of adding a first timestamp to first data detected by the first type detection device includes: saving the image data collected by the imaging device as a first data structure, the first data structure includes a time field, and the value of the time field is used to represent the detection time of the image data.

[0011] Optionally, the method further includes: sorting the multiple image data collected by the imaging device according to the time field.

[0012] Optionally, the first data structure is an AVPacket structure, and the time field is a pts field.

[0013] Optionally, the first type detection device performs detection at a first detection frequency, and the step of adding a first timestamp to the first data detected by the first type detection device includes: adding a first timestamp to each frame of first data detected by the first type detection device, and / or the second type detection device performs detection at a second detection frequency, and the step of adding a second timestamp to the second data detected by the second type detection device includes: adding a second timestamp to each frame of second data detected by the second type detection device.

[0014] Optionally, the method further includes: before using the first type of detection device and the second type of detection device to collect data, performing clock synchronization on the first type of detection device and the second type of detection device.

[0015] According to the second aspect of the present disclosure, a detection method is also provided, including: receiving first data detected by a first type of detection device, the first data including a first timestamp, the first timestamp being used to characterize the detection time of the first data, and the first data including object data in a road; receiving second data detected by a second type of detection device, the second data including a second timestamp, the second timestamp being used to characterize the detection time of the second data, and the second data including object data in a road; when the difference between the first timestamp and the second timestamp is less than a predetermined threshold, determining the detection result of the same object based on the first data and the second data.

[0016] According to the third aspect of the present disclosure, a detection method is also provided, including: adding a first timestamp to first data detected by a first type of detection device, the first timestamp being used to characterize the detection time of the first data; adding a second timestamp to second data detected by a second type of detection device, the second timestamp being used to characterize the detection time of the second data; and determining the detection result of the same detected object based on the first data and the second data when the difference between the first timestamp and the second timestamp is less than a predetermined threshold.

[0017] According to the fourth aspect of the present disclosure, a detection method is also provided, including: adding a first timestamp to first data detected by a first type of detection device, the first timestamp being used to characterize the detection time of the first data; adding a second timestamp to second data detected by a second type of detection device, the second timestamp being used to characterize the detection time of the second data; and determining that the first data and the second data match when the difference between the first timestamp and the second timestamp is less than a predetermined threshold.

[0018] According to the fifth aspect of the present disclosure, a detection system is also provided, including: a first type of detection device, used to detect objects in a road based on a first detection mechanism, wherein the first type of detection device adds a first timestamp to the detected first data, and the first timestamp is used to characterize the detection time of the first data; a second type of detection device, used to detect objects in a road based on a second detection mechanism, wherein the second type of detection device adds a second timestamp to the detected second data, and the second timestamp is used to characterize the detection time of the second data; and a communication module, used to upload the first data and the second data to a server, so that the server determines the detection result of the same object based on the first data and the second data when the difference between the first timestamp and the second timestamp is less than a predetermined threshold.

[0019] Optionally, the detection system is a vehicle-mounted detection system, or the detection system is arranged on the roadside.

[0020] According to the sixth aspect of the present disclosure, a detection system is also provided, including: a first type of detection device, for detecting objects in a road based on a first detection mechanism, wherein the first type of detection device also adds a first timestamp to the detected first data, and the first timestamp is used to characterize the detection time of the first data; a second type of detection device, for detecting objects in a road based on a second detection mechanism, wherein the second type of detection device also adds a second timestamp to the detected second data, and the second timestamp is used to characterize the detection time of the second data; and a processor, for determining the detection result of the same object based on the first data and the second data when the difference between the first timestamp and the second timestamp is less than a predetermined threshold.

[0021] Optionally, the detection system is a vehicle-mounted detection device, or the detection system is arranged on the roadside.

[0022] According to the seventh aspect of the present disclosure, a detection device is also provided, including: a first adding module, used to add a first timestamp to first data detected by a first type of detection device, the first timestamp is used to characterize the detection time of the first data, and the first data includes object data in the road; a second adding module, used to add a second timestamp to second data detected by a second type of detection device, the second timestamp is used to characterize the detection time of the second data, and the second data includes object data in the road; and a determination module, used to determine the detection result of the same object based on the first data and the second data when the difference between the first timestamp and the second timestamp is less than a predetermined threshold.

[0023] According to the eighth aspect of the present disclosure, a data processing device is also provided, including: a first receiving module, used to receive first data detected by a first type of detection device, the first data including a first timestamp, the first timestamp is used to characterize the detection time of the first data, and the first data also includes object data in the road; a second receiving module, used to receive second data detected by a second type of detection device, the second data including a second timestamp, the second timestamp is used to characterize the detection time of the second data, and the second data also includes object data in the road; and a determination module, used to determine the detection result of the same object based on the first data and the second data when the difference between the first timestamp and the second timestamp is less than a predetermined threshold.

[0024] According to the ninth aspect of the present disclosure, a detection device is also provided, including: a first adding module, used to add a first timestamp to first data detected by a first type of detection device, the first timestamp is used to characterize the detection time of the first data; a second adding module, used to add a second timestamp to second data detected by a second type of detection device, the second timestamp is used to characterize the detection time of the second data; and a determination module, used to determine the detection result corresponding to the same detected object based on the first data and the second data when the difference between the first timestamp and the second timestamp is less than a predetermined threshold.

[0025] According to the tenth aspect of the present disclosure, a detection device is also provided, including: a first adding module, used to add a first timestamp to first data detected by a first type of detection device, the first timestamp is used to characterize the detection time of the first data; a second adding module, used to add a second timestamp to second data detected by a second type of detection device, the second timestamp is used to characterize the detection time of the second data; a judgment module, used to determine that the first data and the second data match when the difference between the first timestamp and the second timestamp is less than a predetermined threshold.

[0026] According to the eleventh aspect of the present disclosure, a computing device is also provided, including: a processor; and a memory on which executable code is stored, and when the executable code is executed by the processor, the processor executes the method described in any one of the first to fourth aspects of the present disclosure.

[0027] According to the twelfth aspect of the present disclosure, a non-temporary machine-readable storage medium is also provided, on which executable code is stored. When the executable code is executed by a processor of an electronic device, the processor executes the method described in any one of the first to fourth aspects of the present disclosure.

[0028] The present invention adds a timestamp that can represent the detection time to the original data collected by the detection device, so that the data collected by different types of detection devices can be accurately matched according to the timestamp previously added after being processed for different lengths of time. The matched detection data can be regarded as the detection data for the same detected object. Therefore, the detection results of the same detected object can be determined based on the matched detection data, and the detection results obtained in this way can represent the multi-dimensional information of the detected object. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The above and other objects, features and advantages of the present disclosure will become more apparent through a more detailed description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, wherein like reference numerals generally represent like components in the exemplary embodiments of the present disclosure.

[0030] Figure 1 A schematic flow chart of a detection method according to an embodiment of the present disclosure is shown.

[0031] Figure 2 A schematic flowchart of a detection method according to another embodiment of the present disclosure is shown.

[0032] Figure 3 A schematic structural diagram of a detection system according to an embodiment of the present disclosure is shown.

[0033] Figure 4 A schematic structural diagram of a detection system according to another embodiment of the present disclosure is shown.

[0034] Figure 5 A schematic structural diagram of a detection system according to another embodiment of the present disclosure is shown.

[0035] Figure 6 A schematic diagram of a data fusion process according to an embodiment of the present disclosure is shown.

[0036] Figure 7 A schematic structural diagram of a detection device according to an embodiment of the present disclosure is shown.

[0037] Figure 8 A schematic structural diagram of a detection device according to another embodiment of the present disclosure is shown.

[0038] Fig. 9 A schematic structural diagram of a detection device according to another embodiment of the present disclosure is shown.

[0039] Fig.10 A schematic diagram of the structure of a computing device according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0040] The preferred embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the preferred embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to make the present disclosure more thorough and complete, and to fully convey the scope of the present disclosure to those skilled in the art.

[0041] In order to ensure that data collected by different types of detection devices (such as different types of sensors) can still be accurately matched after being processed for different lengths of time, the present disclosure proposes that a timestamp representing the detection time can be added to the original data collected by the detection device. In this way, after the data collected by different types of detection devices have been processed for different lengths of time, they can be matched according to the previously added timestamp, so as to facilitate subsequent processing (such as fusion processing) based on the matching results.

[0042] Figure 1 A schematic flow chart of a detection method according to an embodiment of the present disclosure is shown.

[0043] See also Figure 1 In step S110, a first timestamp is added to first data detected by a first type detection device.

[0044] In step S120, a second timestamp is added to the second data detected by the second type detection device.

[0045] The present invention does not limit the execution order of step S110 and step S120. Before using the first type detection device and the second type detection device to collect data, the first type detection device and the second type detection device may be firstly clock synchronized. For example, the first type detection device and the second type detection device may be clock synchronized with the NTP server respectively.

[0046] In the process of collecting data using the first type of detection device, a first timestamp may be added to the first data detected by the first type of detection device, and the first timestamp is used to characterize the detection time of the first data. Correspondingly, in the process of collecting data using the second type of detection device, a second timestamp may be added to the second data detected by the second type of detection device, and the second timestamp is used to characterize the detection time of the second data. The first timestamp and the second timestamp may refer to the time in the same time coordinate system, such as the time after synchronization, for example, may indicate the absolute time.

[0047] The first type of detection device is different from the second type of detection device, and the first type of detection device and the second type of detection device may refer to detection devices with different detection principles (or detection mechanisms). For example, the first type of detection device may be an imaging device, such as an image sensor (e.g., a camera), in which case the first data is image data, and the second type of detection device may be a radar device, such as a radar sensor, in which case the second data is radar data.

[0048] The first type detection device and the second type detection device have a certain operating frequency (i.e., detection frequency) respectively. For example, the first type detection device can detect with a first detection frequency, and the second type detection device can detect with a second detection frequency. The first frequency can be the same as the first frequency, or it can be different from the second frequency. In the process of using the first type detection device and the second type detection device to detect data, multiple frames of data can be obtained respectively. As an example, a first timestamp can be added to each frame of first data detected by the first type detection device, and / or a second timestamp can also be added to each frame of second data detected by the second type detection device.

[0049] The first data refers to the original data detected by the first type detection device, that is, the data before data processing. The second data refers to the original data detected by the second type detection device, that is, the data before data processing.

[0050] Step S110 and step S120 may be performed without affecting the existing data processing process (such as the encoding and decoding process). For example, the software and hardware may be modified to functionally support inserting a timestamp (i.e., the first timestamp or the second timestamp) that can represent the detection time into the data (i.e., the first data or the second data) detected by the detection device (i.e., the first type detection device or the second type detection device mentioned above).

[0051] Taking the first type of detection device as an imaging device and the second type of detection device as a radar device as an example, it is relatively simple to add a second timestamp to the radar data. For example, the radar device can be set to directly add an absolute timestamp to the detected data during operation. The specific implementation process of adding a second timestamp to the radar data will not be repeated here. For image data (such as video data frames), the image data collected by the imaging device can be saved as a first data structure, and the first data structure includes a time field, and the value of the time field is used to characterize the detection time of the image data. Among them, the time field can also be used to sort the multiple image data collected by the imaging device.

[0052] For example, the first data structure may be an AVPacket structure, and the time field may be a pts field. That is to say, for image data, the pts field in AVPacket may be reused. The original meaning of the pts field is the display timestamp of the video frame, which is a relative time used for sorting video frames. In the present disclosure, the pts field may be changed to a shooting timestamp (i.e., the first timestamp), which is an absolute time (1970-1-1 may be used as the start time), which may also be used for sorting, and will not affect the existing encoding and decoding process. The specific structure of AVPacket and the pts field are all prior art and will not be described in detail here. In short, the pts field is defined in line 1423 of the standard AVPacket structure, and the present disclosure may modify the definition of this field to an absolute time.

[0053] In step S130, based on the first timestamp and the second timestamp, it is determined whether the first data and the second data match.

[0054] Since the operating frequencies of the first type detection device and the second type detection device may be different, a predetermined threshold may be set when performing data matching according to the first timestamp and the second timestamp. When determining whether the first data and the second data match, the difference between the first timestamp of the first data and the second timestamp of the second data may be compared to determine whether it is less than the predetermined threshold. If the difference is less than the predetermined threshold, it may be determined that the first data and the second data match.

[0055] The present disclosure may also process the first data to obtain a first processing result, and process the second data to obtain a second processing result. The specific process of processing the data may refer to the existing data processing mechanism, which will not be described in detail here.

[0056] The first processing result and the second processing result can represent processing results of different dimensions, and more accurate information can be obtained by synthesizing the matching first processing result and the second processing result. Therefore, the first processing result and the second processing result corresponding to the matching first data and the second data can be synthesized.

[0057] Taking the first type of detection device as an imaging device and the second type of detection device as a radar device as an example, the image data detected by the imaging device can be processed to determine the appearance information of the detected object, and the radar data detected by the radar device can be processed to determine the position information of the detected object, and the appearance information and position information corresponding to the matched image data and radar data are respectively fused to obtain the appearance and position information corresponding to the same detected object. Among them, the specific implementation process of processing image data and radar data is the existing technology in the field and will not be repeated here.

[0058] That is to say, when the detection directions of the first type detection device and the second type detection device are consistent, for the first timestamp and the second timestamp whose difference is less than the predetermined threshold, it can be considered that the first data and the second data corresponding to the two timestamps are detection data for the same detected object. Therefore, the processing results of the first data and the processing results of the second data corresponding to the first timestamp and the second timestamp whose difference is less than the predetermined threshold can be fused. For the processing results of mismatched data, other strategies can be adopted, such as discarding or using them separately.

[0059] Application Examples

[0060] The present disclosure can be applicable to the field of intelligent transportation, and can be used to detect road objects such as vehicles, pedestrians, road facilities, etc. on the road. The obtained detection results can be applied to various application scenarios such as vehicle collision warning detection, vehicle precise positioning, and traffic violation determination.

[0061] Figure 2 A schematic flowchart of a detection method according to another embodiment of the present disclosure is shown.

[0062] See also Figure 2 In step S210, a first timestamp is added to first data detected by a first type detection device.

[0063] In step S220, a second timestamp is added to the second data detected by the second type detection device.

[0064] In this embodiment, the first type of detection device and the second type of detection device are used to detect the same road scene. For example, the first type of detection device and the second type of detection device can be set at the same position to detect road conditions in the same range. The first data detected by the first type of detection device and the second data detected by the second type of detection device both include object data in the road. Among them, the first type of detection device and the second type of detection device can be set on the side of the road as roadside devices, or can be set on the vehicle as vehicle-mounted devices. Optionally, the first type of detection device can be an imaging device, and the second type of detection device can be a radar. Regarding the first type of detection device, the second type of detection device, and the implementation process of adding timestamps to the data, please refer to the relevant description above, which will not be repeated here.

[0065] In step S230, when the difference between the first timestamp and the second timestamp is less than a predetermined threshold, a detection result of the same object is determined according to the first data and the second data.

[0066] For the first timestamp and the second timestamp whose difference is less than a predetermined threshold, it can be considered that the first data and the second data corresponding to the two timestamps are detection data in the same spatiotemporal scene. Therefore, when the difference between the first timestamp and the second timestamp is less than a predetermined threshold, the corresponding first data and the second data can be fused to determine the detection result corresponding to the same object. The "fusion processing" mentioned here means that the data corresponding to the same object in the first data and the second data are fused to obtain the detection result of the object.

[0067] As an example, the first data may be processed to obtain a first processing result, and the second data may be processed to obtain a second processing result, and the detection result of the same object may be determined based on the first processing result and the second processing result.

[0068] Taking the first type of detection device as an imaging device and the second type of detection device as a radar device as an example, the image data formed by the imaging device at a certain moment may include appearance data (color, shape, etc.) of multiple objects, and the radar data detected by the radar device at a certain moment may also include position data for multiple objects. For image data and radar data whose difference between the first timestamp and the second timestamp is less than a predetermined threshold, the image data and radar data may be analyzed to fuse the appearance data and position data corresponding to the same object, thereby obtaining the appearance and position data of the same object.

[0069] Figure 2 The detection method shown can be implemented as a detection system.

[0070] Figure 3FIG. 1 shows a schematic diagram of the structure of a detection system according to an embodiment of the present disclosure. Figure 3 The detection system 300 shown can be implemented as a vehicle-mounted detection system to detect objects around the vehicle (such as other vehicles, pedestrians, road facilities, and other road objects around the vehicle). Figure 3 The detection system 300 shown may also be arranged on the roadside to detect road objects such as vehicles, pedestrians, and road facilities on the road.

[0071] See also Figure 3 The detection system 300 includes a first type detection device 310 , a second type detection device 320 , and a processor 330 .

[0072] The first type detection device 310 is used to detect objects on the road based on a first detection mechanism, wherein the first type detection device 310 also adds a first timestamp to the detected first data, and the first timestamp is used to represent the detection time of the first data.

[0073] The second type detection device 320 is used to detect objects on the road based on a second detection mechanism, wherein the second type detection device 320 further adds a second timestamp to the detected second data, and the second timestamp is used to represent the detection time of the second data.

[0074] The processor 330 is configured to determine a detection result of the same object according to the first data and the second data when the difference between the first timestamp and the second timestamp is less than a predetermined threshold.

[0075] For details about the first type detection device 310 , the second type detection device 320 , and the processor 330 , please refer to the above description, which will not be repeated here.

[0076] Figure 4 FIG. 2 shows a schematic diagram of the structure of a detection system according to another embodiment of the present disclosure. Figure 4 The detection system 400 shown can be implemented as a vehicle-mounted detection system to detect objects around the vehicle (such as other vehicles, pedestrians, road facilities, and other road objects around the vehicle). Figure 4 The detection system 400 shown may also be arranged on the roadside to detect road objects such as vehicles, pedestrians, and road facilities on the road.

[0077] See also Figure 4 The detection system 400 includes a first type detection device 410 , a second type detection device 420 and a communication module 430 .

[0078] The first type detection device 410 is used to detect objects on the road based on a first detection mechanism, wherein the first type detection device adds a first timestamp to the detected first data, and the first timestamp is used to represent the detection time of the first data.

[0079] The second type detection device 420 is used to detect objects on the road based on a second detection mechanism, wherein the second type detection device adds a second timestamp to the detected second data, and the second timestamp is used to represent the detection time of the second data.

[0080] The communication module 430 is used to upload the first data and the second data to the server, so that the server determines the detection result of the same object according to the first data and the second data when the difference between the first timestamp and the second timestamp is less than a predetermined threshold.

[0081] For details about the first type detection device 310 and the second type detection device 320 , please refer to the above description and will not be repeated here.

[0082] In this embodiment, the detection system 400 may upload the detected first data and the second data to a server, and the server may perform subsequent fusion processing according to the timestamps in the data.

[0083] The server may receive first data detected by a first type detection device, and receive second data detected by a second type detection device. When the difference between the first timestamp and the second timestamp is less than a predetermined threshold, the detection result of the same object is determined based on the first data and the second data. The data processing flow can be referred to the relevant description above, which will not be repeated here.

[0084] Figure 5 FIG. 2 shows a schematic diagram of the structure of a detection system according to another embodiment of the present disclosure. Optionally, Figure 5 The detection system shown can be implemented as a vehicle-mounted detection system to detect objects around the vehicle (such as other vehicles around the vehicle, pedestrians, road facilities and other road objects). Figure 5 The detection system shown can also be arranged on the roadside to detect road objects such as vehicles, pedestrians, road facilities, etc. on the road.

[0085] like Figure 5 As shown, the detection system may include an image sensor 210 , a radar sensor 220 , a first data processor 230 , a second data processor 240 , and a fusion module 250 .

[0086] The image sensor 210 and the radar sensor 220 may firstly perform clock synchronization, for example, they may perform clock synchronization with an NTP server respectively.

[0087] For each video frame detected by the image sensor 210, a timestamp that can represent the detection time (the first timestamp mentioned above, i.e., t1, t2 shown in the figure) can be added thereto. The first timestamp can be transmitted to the first data processor 230 together with the video frame as part of the video frame detected by the image sensor 210, and processed by the first data processor 230, for example, it can be encoded by the first data processor 230.

[0088] For each video frame detected by the radar sensor 220, a timestamp that can represent the detection time (the second timestamp mentioned above, i.e., t1′, t2′ shown in the figure) can be added thereto. The second timestamp can be transmitted to the second data processor 240 together with the radar frame as part of the radar frame detected by the radar sensor 220, and processed by the second data processor 240.

[0089] The first data processor 230 and the second data processor 240 may be DSP (Digital Signal Processing) chips located inside a SoC. SoC refers to a system-on-chip, which is an abbreviation of System on Chip.

[0090] The fusion module 250 may fuse the processed video data and the radar data by comparing the first timestamp and the second timestamp.

[0091] like Figure 6 As shown, the input data of the fusion module 250 may be a video data queue and a radar data queue. The video data queue may include a plurality of video data, and the radar data queue may include a plurality of radar data. And the data in the two data queues may be sorted according to the added timestamps (first timestamp / second timestamp).

[0092] The fusion module 250 can take data from the heads of the two queues respectively, and then take the timestamp contained in the original data, for example, for a video frame, the pts field can be taken. Then it is determined whether the two timestamps match. If the timestamps match (approximately equal, a certain error is allowed because the frequencies of the image sensor and the radar sensor may be different), it can be basically considered that the time and space of the detected object in the two sensors are consistent, and then the two data can be fused. If they do not match, other strategies are adopted (either publishing the data separately or discarding it).

[0093] Since the SoC hardware processes video data and radar data at different times, it is difficult to fuse the two data. The present invention records the absolute timestamps of the video data and radar data so that the data matching problem can be solved according to the original timestamps of the data in the fusion algorithm, which increases the accuracy of the fusion result.

[0094] Figure 7 FIG. 1 is a schematic diagram showing the structure of a detection device according to an embodiment of the present disclosure. The functional modules of the detection device can be implemented by hardware, software, or a combination of hardware and software that implements the principles of the present disclosure. It can be understood by those skilled in the art that Figure 7 The functional modules described can be combined or divided into sub-modules to implement the principles of the above invention. Therefore, the description herein can support any possible combination, division, or further limitation of the functional modules described herein.

[0095] The following is a brief description of the functional modules that the detection device may have and the operations that each functional module may perform. For the details involved, please refer to the relevant description above and will not be repeated here.

[0096] See also Figure 7 The detection device 700 includes a first adding module 710 , a second adding module 720 and a determining module 730 .

[0097] The first adding module 710 is used to add a first timestamp to the first data detected by the first type detection device, where the first timestamp is used to represent the detection time of the first data.

[0098] The second adding module 720 is used to add a second timestamp to the second data detected by the second type detection device, where the second timestamp is used to represent the detection time of the second data.

[0099] Optionally, the first data and the second data may include object data in the road. The objects mentioned here may be, but are not limited to, road objects such as vehicles, pedestrians, and road facilities.

[0100] The determination module 730 is used to determine the detection result corresponding to the same detected object according to the first data and the second data when the difference between the first timestamp and the second timestamp is less than a predetermined threshold.

[0101] As an example, the first type of detection device is an imaging device, and the first road data includes image data, and the second type of detection device is a radar device, and the second road data includes radar data.

[0102] Taking the first type of detection device as an imaging device and the second type of detection device as a radar device as an example, the detection device 700 may further include a first processing module and a second processing module. The first processing module is used to process the image data to determine the appearance information of the vehicle, and the second processing module is used to process the radar data to determine the location information of the vehicle. The vehicle detection result may include appearance and location information.

[0103] Figure 8FIG. 1 is a schematic diagram showing the structure of a detection device according to another embodiment of the present disclosure. The functional modules of the detection device can be implemented by hardware, software, or a combination of hardware and software that implements the principles of the present disclosure. It can be understood by those skilled in the art that Figure 8 The functional modules described can be combined or divided into sub-modules to implement the principles of the above invention. Therefore, the description herein can support any possible combination, division, or further limitation of the functional modules described herein.

[0104] The following is a brief description of the functional modules that the detection device may have and the operations that each functional module may perform. For the details involved, please refer to the relevant description above and will not be repeated here.

[0105] See also Figure 8 The detection device 800 includes a first receiving module 810, a second receiving module 820 and a determining module 830.

[0106] The first receiving module 810 is used to receive first data detected by a first type of detection device. The first data includes a first timestamp, which is used to represent the detection time of the first data. The first data also includes object data in the road.

[0107] The second receiving module 820 is used to receive second data detected by a second type of detection device, the second data includes a second timestamp, the second timestamp is used to represent the detection time of the second data, and the second data also includes object data in the road; and

[0108] The determination module 830 is used to determine the detection result of the same object according to the first data and the second data when the difference between the first timestamp and the second timestamp is less than a predetermined threshold.

[0109] Fig. 9 FIG. 1 is a schematic diagram showing the structure of a detection device according to another embodiment of the present disclosure. The functional modules of the detection device can be implemented by hardware, software, or a combination of hardware and software that implements the principles of the present disclosure. It can be understood by those skilled in the art that Fig. 9 The functional modules described can be combined or divided into sub-modules to implement the principles of the above invention. Therefore, the description herein can support any possible combination, division, or further limitation of the functional modules described herein.

[0110] The following is a brief description of the functional modules that the detection device may have and the operations that each functional module may perform. For the details involved, please refer to the relevant description above and will not be repeated here.

[0111] See also Fig. 9The detection device 900 includes a first adding module 910 , a second adding module 920 and a judging module 930 .

[0112] The first adding module 910 is used to add a first timestamp to the first data detected by the first type detection device, and the first timestamp is used to indicate the detection time of the first data. The second adding module 920 is used to add a second timestamp to the second data detected by the second type detection device, and the second timestamp is used to indicate the detection time of the second data.

[0113] As an example, the first type detection device can perform detection at a first detection frequency, and the first adding module 910 can add a first timestamp to each frame of first data detected by the first type detection device, and / or the second type detection device can perform detection at a second detection frequency, and the second adding module 420 can add a second timestamp to each frame of second data detected by the second type detection device.

[0114] The judgment module 930 is used to judge whether the first data and the second data match based on the first timestamp and the second timestamp. The judgment module 930 can compare whether the difference between the first timestamp and the second timestamp is less than a predetermined threshold, and if the difference is less than the predetermined threshold, it is determined that the first data and the second data match.

[0115] Optionally, the detection device 900 may further include a first processing module, a second processing module and a synthesis module. The first processing module is used to process the first data to obtain a first processing result, the second processing module is used to process the second data to obtain a second processing result, and the synthesis module is used to synthesize the first processing result and the second processing result corresponding to the matched first data and the second data respectively.

[0116] As an example, the first type of detection device is an imaging device, the first data is image data, and the second type of detection device is a radar device, the second data is radar data.

[0117] The first processing module can process the image data detected by the imaging device to determine the appearance information of the detected object, the second processing module is used to process the radar data detected by the radar device to determine the position information of the detected object, and the integration module can fuse the appearance information and position information corresponding to the matching image data and radar data respectively to obtain the appearance and position information corresponding to the same detected object.

[0118] Taking the first type of detection device as an imaging device as an example, the first adding module can save the image data collected by the imaging device as a first data structure, the first data structure includes a time field, and the value of the time field is used to characterize the detection time of the image data. Among them, the first data structure can be an AVPacket structure, and the time field is a pts field.

[0119] Optionally, the data processing device 400 may further include a sorting module, which is used to sort the multiple image data collected by the imaging device according to the time field.

[0120] Optionally, the data processing device 400 may further include a synchronization module, which is used to synchronize the clocks of the first type of detection device and the second type of detection device before the first type of detection device and the second type of detection device collect data.

[0121] The specific implementation of the data processing device 900 according to the exemplary embodiment of the present disclosure can be referred to in conjunction with Figure 1 The relevant specific implementation methods described are implemented and will not be repeated here.

[0122] Fig.10 A schematic diagram of the structure of a computing device that can be used to implement the above detection method according to an embodiment of the present invention is shown.

[0123] See also Fig.10 , the computing device 1000 includes a memory 1010 and a processor 1020 .

[0124] The processor 1020 may be a multi-core processor or may include multiple processors. In some embodiments, the processor 1020 may include a general-purpose main processor and one or more special coprocessors, such as a graphics processing unit (GPU), a digital signal processor (DSP), etc. In some embodiments, the processor 1020 may be implemented using a customized circuit, such as an application-specific integrated circuit (ASIC) or a field programmable gate array (FPGA).

[0125] The memory 1010 may include various types of storage units, such as system memory, read-only memory (ROM), and permanent storage. Among them, ROM can store static data or instructions required by the processor 1020 or other modules of the computer. The permanent storage device may be a readable and writable storage device. The permanent storage device may be a non-volatile storage device that does not lose the stored instructions and data even after the computer is powered off. In some embodiments, the permanent storage device uses a large-capacity storage device (such as a magnetic or optical disk, flash memory) as a permanent storage device. In some other embodiments, the permanent storage device may be a removable storage device (such as a floppy disk, optical drive). The system memory may be a readable and writable storage device or a volatile readable and writable storage device, such as a dynamic random access memory. The system memory may store some or all instructions and data required by the processor at run time. In addition, the memory 1010 may include any combination of computer-readable storage media, including various types of semiconductor memory chips (DRAM, SRAM, SDRAM, flash memory, programmable read-only memory), and disks and / or optical disks may also be used. In some embodiments, the memory 1010 may include a readable and / or writable removable storage device, such as a laser disc (CD), a read-only digital versatile disc (such as a DVD-ROM, a double-layer DVD-ROM), a read-only Blu-ray disc, an ultra-density optical disc, a flash memory card (such as an SD card, a mini SD card, a Micro-SD card, etc.), a magnetic floppy disk, etc. The computer-readable storage medium does not include carrier waves and transient electronic signals transmitted wirelessly or wired.

[0126] The memory 1010 stores executable codes, and when the executable codes are processed by the processor 1020 , the processor 1020 can execute the detection method mentioned above.

[0127] The detection method, apparatus, system and computing device according to the present invention have been described in detail above with reference to the accompanying drawings.

[0128] In addition, the method according to the present invention may also be implemented as a computer program or a computer program product, which includes computer program code instructions for executing the above steps defined in the above method of the present invention.

[0129] Alternatively, the present invention may also be implemented as a non-temporary machine-readable storage medium (or computer-readable storage medium, or machine-readable storage medium) on which executable code (or computer program, or computer instruction code) is stored. When the executable code (or computer program, or computer instruction code) is executed by a processor of an electronic device (or computing device, server, etc.), the processor executes the various steps of the above-mentioned method according to the present invention.

[0130] Those skilled in the art will further appreciate that the various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the disclosure herein may be implemented as electronic hardware, computer software, or a combination of both.

[0131] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system and method according to multiple embodiments of the present invention. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or code, and the part of the module, program segment or code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0132] The embodiments of the present invention have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terms used herein are selected to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A detection method, characterized in that: include: adding a first timestamp to first data detected by a first type detection device, wherein the first timestamp is used to characterize a detection time of the first data, the first data comprising object data in a road; adding a second timestamp to second data detected by the second type detection device, wherein the second timestamp is used to characterize a detection time of the second data, and the second data includes object data in the road; When the difference between the first timestamp and the second timestamp is less than a predetermined threshold, determining that the first data and the second data match, and determining a detection result of the same object based on the first data and the second data, wherein the detection result is used to characterize multi-dimensional information of the same object; The first type of detection device is an imaging device, and the first data is image data; The step of adding a first timestamp to the first data detected by the first type detection device comprises: saving the image data collected by the imaging device as a first data structure, wherein the first data structure comprises a time field, and the value of the time field is used to represent the detection time of the image data; wherein the first data structure is an AVPacket structure, and the time field is a pts field; The method further comprises: sorting the plurality of image data collected by the imaging device according to the time field; The step of determining the detection results of the same object comprises: Processing the first data to obtain a first processing result; Processing the second data to obtain a second processing result; A detection result of the same object is determined based on the first processing result and the second processing result.

2. The detection method according to claim 1, characterized in that: The objects include: vehicles; and / or pedestrians; and / or road facilities.

3. The detection method according to claim 1, characterized in that: The second type detection device is a radar device, and the second data is radar data.

4. The detection method according to claim 3, characterized in that: The step of determining the detection results of the same object comprises: Processing the image data to determine appearance information of the object; Processing the radar data to determine location information of an object; Based on the appearance information and the position information, appearance and position information of the same object are determined.

5. The detection method according to claim 1, characterized in that: The first type detection device performs detection at a first detection frequency, and the step of adding a first timestamp to the first data detected by the first type detection device comprises: adding a first timestamp to each frame of first data detected by the first type detection device, and / or The second type detection device performs detection at a second detection frequency, and the step of adding a second timestamp to the second data detected by the second type detection device includes: adding a second timestamp to each frame of second data detected by the second type detection device.

6. The detection method according to claim 1, characterized in that: Also includes: Before using the first type detection device and the second type detection device to collect data, clock synchronization is performed on the first type detection device and the second type detection device.

7. A detection method, characterized in that: include: receiving first data detected by a first type of detection device, the first data comprising a first timestamp, the first timestamp being used to characterize a detection time of the first data, the first data comprising object data in a road; the first type of detection device being an imaging device, the first data being image data; The image data collected by the imaging device is saved as a first data structure, wherein the first data structure includes a time field, and a value of the time field is used to represent a detection time of the image data; The first data structure is an AVPacket structure, and the time field is a pts field; The plurality of image data collected by the imaging device are sorted according to the time field; receiving second data detected by a second type of detection device, wherein the second data includes a second timestamp, the second timestamp is used to characterize a detection time of the second data, and the second data includes object data in a road; When the difference between the first timestamp and the second timestamp is less than a predetermined threshold, determining that the first data and the second data match, and determining a detection result of the same object based on the first data and the second data, wherein the detection result is used to characterize multi-dimensional information of the same object; The step of determining the detection results of the same object comprises: Processing the first data to obtain a first processing result; Processing the second data to obtain a second processing result; A detection result of the same object is determined based on the first processing result and the second processing result.

8. A detection method, characterized in that: include: Adding a first timestamp to first data detected by a first type detection device, where the first timestamp is used to represent a detection time of the first data; The first type of detection device is an imaging device, and the first data is image data; The step of adding a first timestamp to the first data detected by the first type detection device comprises: saving the image data collected by the imaging device as a first data structure, wherein the first data structure comprises a time field, and the value of the time field is used to represent the detection time of the image data; wherein the first data structure is an AVPacket structure, and the time field is a pts field; and the plurality of image data collected by the imaging device are sorted according to the time field; adding a second timestamp to second data detected by the second type detection device, where the second timestamp is used to represent a detection time of the second data; When the difference between the first timestamp and the second timestamp is less than a predetermined threshold, determine that the first data and the second data match, and determine a detection result of the same detected object based on the first data and the second data, wherein the detection result is used to characterize multi-dimensional information of the same detected object; The step of determining the detection result of the same detected object comprises: Processing the first data to obtain a first processing result; Processing the second data to obtain a second processing result; A detection result of the same object is determined based on the first processing result and the second processing result.

9. A detection system, characterized in that: include: A first type detection device is used to detect an object in a road based on a first detection mechanism, wherein the first type detection device adds a first timestamp to the detected first data, and the first timestamp is used to represent the detection time of the first data; the first type detection device is an imaging device, and the first data is image data; the step of adding a first timestamp to the detected first data includes: saving the image data collected by the imaging device as a first data structure, the first data structure includes a time field, and the value of the time field is used to represent the detection time of the image data; wherein the first data structure is an AVPacket structure, and the time field is a pts field; and the multiple image data collected by the imaging device are sorted according to the time field; A second type detection device, configured to detect an object in a road based on a second detection mechanism, wherein the second type detection device adds a second timestamp to the detected second data, and the second timestamp is used to represent a detection time of the second data; and a communication module, configured to upload the first data and the second data to a server, so that the server determines that the first data and the second data match when a difference between the first timestamp and the second timestamp is less than a predetermined threshold, and determines a detection result of the same object based on the first data and the second data, wherein the detection result is used to characterize multi-dimensional information of the same object; The step of determining the detection results of the same object comprises: Processing the first data to obtain a first processing result; Processing the second data to obtain a second processing result; A detection result of the same object is determined based on the first processing result and the second processing result.

10. The detection system according to claim 9, characterized in that: The detection system is a vehicle-mounted detection system, or the detection system is arranged on the roadside.

11. A detection system, characterized in that: include: A first type detection device is used to detect an object in a road based on a first detection mechanism, wherein the first type detection device also adds a first timestamp to the detected first data, and the first timestamp is used to represent the detection time of the first data; the first type detection device is an imaging device, and the first data is image data; the step of adding a first timestamp to the detected first data includes: saving the image data collected by the imaging device as a first data structure, the first data structure includes a time field, and the value of the time field is used to represent the detection time of the image data; wherein the first data structure is an AVPacket structure, and the time field is a pts field; and the multiple image data collected by the imaging device are sorted according to the time field; A second type detection device, configured to detect an object in a road based on a second detection mechanism, wherein the second type detection device further adds a second timestamp to the detected second data, the second timestamp being used to characterize a detection time of the second data; and a processor, configured to determine that the first data and the second data match when a difference between the first timestamp and the second timestamp is less than a predetermined threshold, and determine a detection result of the same object based on the first data and the second data, wherein the detection result is used to characterize multi-dimensional information of the same object; The step of determining the detection results of the same object comprises: Processing the first data to obtain a first processing result; Processing the second data to obtain a second processing result; A detection result of the same object is determined based on the first processing result and the second processing result.

12. The detection system according to claim 11, characterized in that: The detection system is a vehicle-mounted detection system, or the detection system is arranged on the roadside.

13. A detection device, characterized in that: include: A first adding module, configured to add a first timestamp to first data detected by a first type of detection device, wherein the first timestamp is used to characterize a detection time of the first data, and the first data includes object data in a road; The first type of detection device is an imaging device, and the first data is image data; The step of adding a first timestamp to the first data detected by the first type detection device comprises: saving the image data collected by the imaging device as a first data structure, wherein the first data structure comprises a time field, and the value of the time field is used to represent the detection time of the image data; wherein the first data structure is an AVPacket structure, and the time field is a pts field; and the plurality of image data collected by the imaging device are sorted according to the time field; A second adding module, configured to add a second timestamp to second data detected by a second type of detection device, wherein the second timestamp is used to characterize a detection time of the second data, and the second data includes object data in the road; and a determination module, configured to determine that the first data and the second data match when the difference between the first timestamp and the second timestamp is less than a predetermined threshold, and determine a detection result of the same object based on the first data and the second data, wherein the detection result is used to characterize multi-dimensional information of the same object; The step of determining the detection results of the same object comprises: Processing the first data to obtain a first processing result; Processing the second data to obtain a second processing result; A detection result of the same object is determined based on the first processing result and the second processing result.

14. A data processing device, characterized in that: include: A first receiving module is used to receive first data detected by a first type of detection device, the first data including a first timestamp, the first timestamp is used to represent the detection time of the first data, and the first data also includes object data in the road; the first type of detection device is an imaging device, and the first data is image data; the image data collected by the imaging device is saved as a first data structure, the first data structure includes a time field, and the value of the time field is used to represent the detection time of the image data; The first data structure is an AVPacket structure, and the time field is a pts field; A second receiving module, configured to receive second data detected by a second type of detection device, wherein the second data includes a second timestamp, the second timestamp is used to characterize a detection time of the second data, and the second data also includes object data in the road; as well as a determination module, configured to determine that the first data and the second data match when the difference between the first timestamp and the second timestamp is less than a predetermined threshold, and determine a detection result of the same object based on the first data and the second data, wherein the detection result is used to characterize multi-dimensional information of the same object; The step of determining the detection result of the same object includes: processing the first data to obtain a first processing result; processing the second data to obtain a second processing result; and determining the detection result of the same object based on the first processing result and the second processing result; A sorting module is used to sort the multiple image data collected by the imaging device according to the time field.

15. A detection device, characterized in that: include: A first adding module, configured to add a first timestamp to first data detected by a first type of detection device, wherein the first timestamp is used to represent a detection time of the first data; The first type of detection device is an imaging device, and the first data is image data; The step of adding a first timestamp to the first data detected by the first type detection device comprises: saving the image data collected by the imaging device as a first data structure, wherein the first data structure comprises a time field, and the value of the time field is used to represent the detection time of the image data; wherein the first data structure is an AVPacket structure, and the time field is a pts field; and the plurality of image data collected by the imaging device are sorted according to the time field; A second adding module, used for adding a second timestamp to second data detected by a second type of detection device, where the second timestamp is used to represent a detection time of the second data; a determination module, configured to determine that the first data and the second data match when the difference between the first timestamp and the second timestamp is less than a predetermined threshold, and determine a detection result corresponding to the same detected object based on the first data and the second data, wherein the detection result is used to characterize multi-dimensional information of the same detected object; A first processing module, used for processing the first data to obtain a first processing result; A second processing module, used for processing the second data to obtain a second processing result; The synthesis module is used to synthesize the first processing result and the second processing result respectively corresponding to the matched first data and the second data to determine the detection result of the same object.

16. A computing device, characterized in that include: processor; as well as A memory having executable codes stored thereon, which, when executed by the processor, causes the processor to execute the method according to any one of claims 1 to 8.

17. A non-transitory machine-readable storage medium, characterized in that: An executable code is stored thereon, and when the executable code is executed by a processor of an electronic device, the processor is caused to execute the method as claimed in any one of claims 1 to 8.

Images