Image detection method, device and equipment based on vehicle-road cooperation and storage medium
By having the image acquisition devices of the vehicle-mounted unit and the roadside unit in the vehicle-road cooperative system work together, and by using mapping relationships and feature data fusion, the problem of inaccurate target detection results in the Internet of Vehicles is solved, and target recognition with higher accuracy and security is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- CHINA MOBILE COMM LTD RES INST
- Filing Date
- 2021-07-13
- Publication Date
- 2026-07-24
AI Technical Summary
Existing target detection methods in vehicle-to-everything (V2X) systems are affected by environmental factors, leading to inaccurate detection results, especially in blurry video images where it is difficult to accurately identify targets.
By having the image acquisition devices of the vehicle-mounted unit and the roadside unit work together to acquire images of the same detection area, the target detection algorithm is used to determine the candidate region, and the detection accuracy is improved based on the mapping relationship and the non-maximum suppression algorithm. The classification accuracy is improved by combining feature data fusion.
It improves the location accuracy and classification accuracy of target detection, reduces the probability of false detection, provides timely warnings of potential dangers, and enhances traffic safety.
Smart Images
Figure CN115620244B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of intelligent transportation, and in particular to an image detection method, apparatus, device, and storage medium based on vehicle-road cooperation. Background Technology
[0002] With the rapid development of the automotive industry, urban traffic pressure is increasing. Vehicle-road cooperative systems (V2X) represent a crucial breakthrough in alleviating urban traffic congestion and a significant development in the Internet of Vehicles (IoV). V2X systems comprise onboard units and roadside units that collaborate to ensure traffic safety and efficiency. Road monitoring is a vital component of IoV technology, and accurately identifying targets in cameras is the cornerstone of subsequent technologies such as autonomous driving and vehicle anomaly detection.
[0003] The main task of object detection is to find targets in video images and determine their category. In related technologies, object detection in vehicle-to-everything (V2X) systems is mainly based on footage captured by a single camera, such as surveillance cameras on highways. However, the captured footage is often very blurry and easily affected by environmental factors such as lighting and weather, resulting in unsatisfactory image quality. Furthermore, since the pre-trained models are often trained on relatively high-definition datasets, they are not easily adapted to real-world scenarios, making it difficult to find targets in video images and leading to inaccurate detection results. Summary of the Invention
[0004] In view of this, embodiments of this application provide an image detection method, apparatus, device, and storage medium based on vehicle-road cooperation, aiming to improve the accuracy of detection results.
[0005] The technical solution of this application embodiment is implemented as follows:
[0006] This application provides an image detection method based on vehicle-road cooperation, including:
[0007] The first and second images are obtained by the first image acquisition device on the vehicle-mounted unit side and the second image acquisition device on the roadside unit side, based on the same detection area.
[0008] Based on the target detection algorithm, the first target candidate region of the first image and the second target candidate region of the second image are determined respectively;
[0009] Based on the mapping relationship between the first image and the second image, a first virtual target candidate region is determined to be mapped from the first target candidate region to the second image, and a second virtual target candidate region is determined to be mapped from the second target candidate region to the first image.
[0010] The location region of the target in the detection region is determined based on the first target candidate region, the second target candidate region, the first virtual target candidate region, and the second virtual target candidate region;
[0011] The first image corresponds to the first image acquisition device, and the second image corresponds to the second image acquisition device.
[0012] In the above scheme, determining the location region of the target in the detection region based on the first target candidate region, the second target candidate region, the first virtual target candidate region, and the second virtual target candidate region includes:
[0013] The nonmaximum suppression algorithm is used to traverse the first target candidate region, the second virtual target candidate region, and the second target candidate region and the first virtual target candidate region of the second image to determine the location region of the target in the detection region.
[0014] The image detection method described above also includes:
[0015] Target recognition is performed on the first image based on the location region of the target;
[0016] The target recognition result is compared with the static obstacles in the pre-constructed detection area;
[0017] If at least one of the static obstacles is found to be missing in the identification results, a warning message is generated.
[0018] The image detection method described above also includes:
[0019] Target recognition is performed on both the first image and the second image based on the location region of the target.
[0020] Determine the target location region where the same object exists in the first image and the second image;
[0021] Based on the target location region, extract the first feature data of the first image and the second feature data of the second image respectively;
[0022] The target classification result of the target location region is determined based on the first feature data and the second feature data.
[0023] In the above scheme, determining the target classification result of the target location region based on the first feature data and the second feature data includes:
[0024] The first feature data and the second feature data are fused and then input into an image classification network to obtain the target classification result of the target location region.
[0025] The image detection method described above also includes:
[0026] The mapping relationship between the alignment of the first image and the second image is determined based on the corresponding key points in the first image and the second image.
[0027] In the above scheme, the acquisition of the first image and the second image obtained by the first image acquisition device on the vehicle-mounted unit side and the second image acquisition device on the roadside unit side based on the same detection area includes:
[0028] Acquire a first image set acquired by the first image acquisition device and a second image set acquired by the second image acquisition device;
[0029] Based on the acquisition time, location, and angle of each image, the first image and the second image that match the same detection region are extracted from the first image set and the second image set.
[0030] This application also provides an image detection device based on vehicle-road cooperation, including:
[0031] The acquisition module is used to acquire a first image and a second image obtained by the first image acquisition device on the vehicle-mounted unit side and the second image acquisition device on the roadside unit side based on the same detection area;
[0032] The candidate region extraction module is used to determine the first target candidate region of the first image and the second target candidate region of the second image based on the target detection algorithm.
[0033] The mapping module is used to determine, based on the mapping relationship between the first image and the second image, a first virtual target candidate region mapped from the first target candidate region to the second image and a second virtual target candidate region mapped from the second target candidate region to the first image;
[0034] The target determination module is used to determine the location region of the target in the detection region based on the first target candidate region, the second target candidate region, the first virtual target candidate region, and the second virtual target candidate region.
[0035] The first image corresponds to the first image acquisition device, and the second image corresponds to the second image acquisition device.
[0036] This application embodiment further provides an image detection device based on vehicle-road cooperation, including: a processor and a memory for storing a computer program capable of running on the processor, wherein,
[0037] The processor is used to execute the steps of the method described in the embodiments of this application when running a computer program.
[0038] This application also provides a storage medium storing a computer program, which, when executed by a processor, implements the steps of the method described in this application.
[0039] The technical solution provided in this application involves acquiring a first image and a second image from a first image acquisition device on the vehicle-mounted unit side and a second image acquisition device on the roadside unit side, based on the same detection area. A first target candidate region in the first image and a second target candidate region in the second image are determined based on a target detection algorithm. A first virtual target candidate region is mapped from the first target candidate region to the second image, and a second virtual target candidate region is mapped from the second target candidate region to the first image, based on the mapping relationship between the first and second images. The location region of the target in the detection area is determined based on the first target candidate region, the second target candidate region, the first virtual target candidate region, and the second virtual target candidate region. This allows the use of existing first image acquisition devices on the vehicle-mounted unit side and second image acquisition devices on the roadside unit side to construct a vehicle-road cooperative system, improving the accuracy of the target location region in the detection area and thus enhancing the accuracy of the detection results. Attached Figure Description
[0040] Figure 1 This is a schematic flowchart of the image detection method based on vehicle-road cooperation according to an embodiment of this application;
[0041] Figure 2 This is a flowchart illustrating the image detection method based on vehicle-road cooperation in an application embodiment of this application;
[0042] Figure 3 This is a schematic diagram of the structure of the image detection device based on vehicle-road cooperation according to an embodiment of this application;
[0043] Figure 4 This is a schematic diagram of the structure of an image detection device based on vehicle-road cooperation according to an embodiment of this application. Detailed Implementation
[0044] The present application will now be described in further detail with reference to the accompanying drawings and embodiments.
[0045] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application.
[0046] This application provides an image detection method based on vehicle-road cooperation, such as... Figure 1 As shown, the method includes:
[0047] Step 101: Obtain the first image and the second image obtained by the first image acquisition device on the vehicle unit side and the second image acquisition device on the roadside unit side based on the same detection area.
[0048] Here, an on-board unit (OBU) refers to a monitoring device with communication functions installed on a vehicle, and a roadside unit (RSU) refers to a monitoring device with communication functions that is fixedly installed on the road where the vehicle travels. The first image acquisition device installed on the on-board unit and the second image acquisition device installed on the roadside unit can be a camera, a laser imaging radar, etc., and this application embodiment does not limit this.
[0049] It is understandable that the first image corresponds to the first image acquisition device, and the second image corresponds to the second image acquisition device.
[0050] For example, the image detection method based on vehicle-road cooperation can be applied to an image data processing platform, which can be a server or a server cluster. This application embodiment does not limit this.
[0051] In some embodiments, acquiring the first image and the second image obtained by the first image acquisition device on the vehicle-mounted unit side and the second image acquisition device on the roadside unit side based on the same detection area includes:
[0052] Acquire a first image set acquired by the first image acquisition device and a second image set acquired by the second image acquisition device;
[0053] Based on the acquisition time, location, and angle of each image, the first image and the second image that match the same detection region are extracted from the first image set and the second image set.
[0054] For example, the image data processing platform can receive a first image set acquired by a first image acquisition device on the vehicle-mounted unit side and a second image set acquired by a second image acquisition device on the roadside unit side. It should be noted that the first and second image acquisition devices also upload acquisition parameters such as the acquisition time, acquisition location, and acquisition angle of the images.
[0055] For example, the first image acquisition device on the vehicle-mounted unit side can report the acquired images and corresponding acquisition parameters at a set frequency. Since the first image acquisition device moves in real time along with the vehicle, the vehicle-mounted unit can upload images via video streaming, and each frame will have a corresponding timestamp. The acquisition angle on the vehicle-mounted unit side can be determined based on the vehicle's driving angle and the installation angle of the first image acquisition device. For example, the installation angle of the first image acquisition device can be input by the user, the vehicle's driving angle can be determined based on the vehicle's driving trajectory through calculation, and the acquisition angle can be determined based on the current driving angle and the initially set installation angle.
[0056] Understandably, the acquisition location and angle of the second image acquisition device in the roadside unit can be determined after initial installation. If adjustments to the installation location and / or acquisition angle occur, the corresponding parameters can be updated promptly. The acquisition time of the second image acquisition device can be generated in real time.
[0057] For example, the acquisition parameters may also include: the operating parameters of the image acquisition device, such as resolution, device model, etc.
[0058] Understandably, the image data processing platform matches the received images based on the acquisition time, acquisition location, and acquisition angle to obtain image pairs corresponding to the same detection area, the same detection time, and the same acquisition angle, namely, the first image and the second image.
[0059] Step 102: Determine the first target candidate region of the first image and the second target candidate region of the second image based on the target detection algorithm.
[0060] Here, the image data processing platform can input the first image from the first image acquisition device and the second image from the second image acquisition device into the target detection algorithm to extract the first target candidate region from the first image and the second target candidate region from the second image.
[0061] For example, the aforementioned first and second target candidate regions can be extracted based on the Mask R-CNN algorithm. The Mask R-CNN algorithm is a deep learning-based object detection method that inherits from Faster R-CNN. Mask R-CNN adds a Mask Prediction Branch to Faster R-CNN and proposes ROI Align based on ROI Pooling to obtain target candidate regions.
[0062] Step 103: Based on the mapping relationship between the first image and the second image alignment, determine the first virtual target candidate region mapped to the second image and the second virtual target candidate region mapped to the first image.
[0063] It is understood that in this embodiment, the first target candidate region of the first image is mapped onto the second image to obtain the first virtual target candidate region on the second image, and the second target candidate region of the second image is mapped onto the first image to obtain the second virtual target candidate region on the first image, so that the target candidate region of the detection area is richer, which is conducive to more accurately determining the location region of the target in the detection area.
[0064] Step 104: Determine the location region of the target in the detection region based on the first target candidate region, the second target candidate region, the first virtual target candidate region, and the second virtual target candidate region.
[0065] It is understood that the embodiments of this application can utilize the existing first image acquisition device on the vehicle unit side and the second image acquisition device on the roadside unit side to construct a vehicle-road cooperative system, and determine the location region of the target in the detection region based on the fusion result of the aforementioned candidate regions of the first image and the second image, which can improve the accuracy of the location region of the target in the detection region and help improve the accuracy of the detection results.
[0066] For example, determining the location region of the target in the detection region based on the first target candidate region, the second target candidate region, the first virtual target candidate region, and the second virtual target candidate region includes:
[0067] The first target candidate region, the second virtual target candidate region, and the second target candidate region and the first virtual target candidate region of the second image are traversed based on the non-maximum suppression (NMS) algorithm to determine the location region of the target in the detection region.
[0068] For example, the NMS algorithm can be used to traverse each of the aforementioned first target candidate region, second virtual target candidate region, second target candidate region and first virtual target candidate region, and filter them based on the intersection-union ratio between each candidate region to determine the location region of the target in the detection region.
[0069] It is understandable that, since step 103 above requires utilizing the mapping relationship between the first image and the second image alignment, in some embodiments, the image detection method further includes:
[0070] The mapping relationship between the alignment of the first image and the second image is determined based on the corresponding key points in the first image and the second image.
[0071] For example, key points in the first and second images can be extracted using the ORB (Oriented Fast and Rotated BRIEF) algorithm, and the mapping relationship between the first and second images can be determined based on the transformation relationship of the coordinate positions of the corresponding key points in the first and second images. The ORB algorithm is a fast algorithm for feature point extraction and description.
[0072] In practical applications, the first image acquisition device on the vehicle unit side is often affected by factors such as vehicle speed and driving angle, which may lead to missed target detection. It is necessary to provide early warning for this situation in order to better meet the needs of subsequent intelligent driving scenarios.
[0073] Based on this, in some embodiments, the image detection method further includes:
[0074] Target recognition is performed on the first image based on the location region of the target;
[0075] The target recognition result is compared with the static obstacles in the pre-constructed detection area;
[0076] If at least one of the static obstacles is found to be missing in the identification results, a warning message is generated.
[0077] It is understandable that the image data processing platform can perform target recognition on the first image acquired by the first image acquisition device based on the target location area determined in the aforementioned step 104, and perform static comparison based on the recognition results, thereby providing timely alarms for situations where the first image acquisition device on the vehicle unit side fails to detect a target.
[0078] Considering that the second image acquisition device on the roadside unit side is generally in a fixed state, the static obstacles in the detection area can be determined based on the images acquired by the second image acquisition device. For example, based on the video captured by the camera of the roadside unit, images of static obstacles are filtered out. To save labor costs, target detection algorithms (such as Mask R-CNN) can be used first for target recognition, followed by manual review. False or missed static obstacles are manually labeled, thus obtaining a dataset of static obstacles in the detection area. If the image data processing platform determines, for the first image, that at least one static obstacle in the dataset lacking static obstacles in the detection area, a warning message can be generated. Preferably, the current driving status data of the vehicle collected by the vehicle-mounted unit, such as driving speed and driving angle, can be acquired and stored for subsequent analysis and processing.
[0079] In practical applications, inaccurate classification results of targets in the detection area may occur due to factors such as the resolution of the first image acquisition device or the small size of the target. For example, the existing detection algorithm (YOLOv5) might identify a sheep on the road as a car, and this false detection could pose a threat to the vehicle's intelligent driving capabilities. If other living beings can be detected in a timely manner and an alarm is triggered in the vehicle-to-infrastructure (V2I) system, subsequent traffic accidents can be avoided to some extent, reducing casualties.
[0080] Based on this, in some embodiments, the image detection method further includes:
[0081] Target recognition is performed on both the first image and the second image based on the location region of the target.
[0082] Determine the target location region where the same object exists in the first image and the second image;
[0083] Based on the target location region, extract the first feature data of the first image and the second feature data of the second image respectively.
[0084] The target classification result of the target location region is determined based on the first feature data and the second feature data.
[0085] It is understandable that the image data processing platform can re-identify the target based on the target location region in the first image and the second image. For target location regions where the same object exists, the first feature data of the first image and the second feature data of the second image are fused based on the target location region to determine the target classification result of the target location region. Compared with the existing technology, this can effectively improve the accuracy of the target classification result and reduce the probability of the aforementioned false detection of the target.
[0086] For example, determining the target classification result of the target location region based on the first feature data and the second feature data includes:
[0087] The first feature data and the second feature data are fused and then input into an image classification network to obtain the target classification result of the target location region.
[0088] For example, the first feature data can be a first feature map of the target location region on the first image, and the second feature data can be a second feature map of the target location region on the second image. The first feature map and the second feature map can be concatenated and input into an image classification network to obtain the target classification result of the target location region.
[0089] The present application will be further described in detail below with reference to application examples.
[0090] like Figure 2 As shown, the image detection method based on vehicle-road cooperation in this application embodiment may include the following steps:
[0091] Step 201: Filter the first image set of the first image acquisition device and the second image set uploaded by the second image acquisition device to determine the paired images consisting of the first image and the second image of the same detection area.
[0092] For example, the image data processing platform can filter the received first image set and second image set based on the acquisition time, acquisition location and acquisition angle of each image to obtain image pairs corresponding to the same detection area, the same detection time and the same acquisition angle, that is, paired first images and second images.
[0093] Step 202: Input the paired first and second images into the Mask R-CNN algorithm.
[0094] Step 203: Determine the first target candidate region of the first image and the second target candidate region of the second image.
[0095] Here, the Mask R-CNN algorithm determines the candidate bounding boxes (i.e., the first target candidate regions) of the first image and the candidate bounding boxes (i.e., the second target candidate regions) of the second image based on the intermediate results.
[0096] Step 204: Determine the mapping relationship between the first image and the second image based on the camera alignment algorithm.
[0097] For example, key points of the first and second images can be extracted based on the ORB algorithm, and the mapping relationship f between the first and second images can be determined based on the transformation relationship of the coordinate positions of the corresponding key points in the first and second images.
[0098] Step 205: Map the first target candidate region and the second target candidate region.
[0099] Here, the candidate bounding boxes of the first image can be mapped to the second image based on the mapping relationship f, and the candidate bounding boxes of the second image can be mapped to the first image.
[0100] Step 206: Determine the location region of the target in the detection area based on the NMS algorithm.
[0101] Here, the candidate boxes from step 205 can be combined to determine the location region of the target in the detection region based on the NMS algorithm.
[0102] Step 207: Determine whether there is a target missed detection on the vehicle unit side based on static comparison.
[0103] The image data processing platform can perform target recognition on the first image acquired by the first image acquisition device based on the target location area determined in the aforementioned step 206, and perform static comparison based on the recognition result with a pre-built dataset of static obstacles, thereby providing timely alarms for situations where the first image acquisition device on the vehicle unit side fails to detect a target.
[0104] For example, the warning message may include the location area of the missed target and send the warning message to the corresponding driver to improve the safety of driving the vehicle.
[0105] Step 208: Update the target recognition results based on dynamic comparison.
[0106] The image data processing platform can re-identify targets in the first image and the second image based on the target's location region. For target location regions containing the same object, the platform fuses the first feature data of the first image with the second feature data of the second image based on the target location region to determine the target classification result of the target location region. Compared with existing technologies, this can effectively improve the accuracy of target classification results and reduce the probability of the aforementioned false detection of targets.
[0107] As described above, the method in this application embodiment integrates image acquisition information from both the roadside unit and the vehicle-mounted unit, updating the original detection results in terms of detection box accuracy and classification accuracy. Specifically, based on a camera alignment mapping algorithm, the detection box results from both image acquisition devices are integrated, improving the localization accuracy of the detection boxes. Furthermore, based on static comparison, timely alerts can be issued for missed target detections on the vehicle-mounted unit side. By using a re-identification algorithm to match the same target and increasing the feature dimensions of the same target, the classification accuracy of the target can be improved.
[0108] To implement the method of the embodiments of this application, the embodiments of this application also provide an image detection device based on vehicle-road cooperation. This image detection device based on vehicle-road cooperation corresponds to the above-mentioned image detection method based on vehicle-road cooperation. The steps in the embodiments of the above-mentioned image detection method based on vehicle-road cooperation are also fully applicable to the embodiments of this image detection device based on vehicle-road cooperation.
[0109] like Figure 3As shown, the vehicle-road cooperative image detection device includes: an acquisition module 301, a candidate region extraction module 302, a mapping module 303, and a target determination module 304. The acquisition module 301 acquires a first image and a second image obtained by a first image acquisition device on the vehicle-mounted unit side and a second image acquisition device on the roadside unit side, based on the same detection area. The candidate region extraction module 302 determines a first target candidate region in the first image and a second target candidate region in the second image based on a target detection algorithm. The mapping module 303 determines a first virtual target candidate region mapped from the first target candidate region to the second image and a second virtual target candidate region mapped from the second target candidate region to the first image, based on the mapping relationship between the first image and the second image. The target determination module 304 determines the location region of the target in the detection area based on the first target candidate region, the second target candidate region, the first virtual target candidate region, and the second virtual target candidate region. The first image corresponds to the first image acquisition device, and the second image corresponds to the second image acquisition device.
[0110] In some embodiments, the target determination module 304 is specifically used for:
[0111] The nonmaximum suppression algorithm is used to traverse the first target candidate region, the second virtual target candidate region, and the second target candidate region and the first virtual target candidate region of the second image to determine the location region of the target in the detection region.
[0112] In some embodiments, the vehicle-road cooperative image detection device further includes:
[0113] The static comparison module 305 is used to perform target recognition on the first image based on the location region of the target; compare the recognition result of the target recognition with the static obstacles in the pre-constructed detection region; and generate a warning message if it is determined that at least one of the static obstacles is missing in the recognition result.
[0114] In some embodiments, the vehicle-road cooperative image detection device further includes:
[0115] The dynamic comparison module 306 is used to perform target recognition on both the first image and the second image based on the location region of the target; determine the target location region where the same object exists in the first image and the second image; extract the first feature data of the first image and the second feature data of the second image based on the target location region; and determine the target classification result of the target location region based on the first feature data and the second feature data.
[0116] For example, the dynamic comparison module 306 determines the target classification result of the target location region based on the first feature data and the second feature data, including:
[0117] The first feature data and the second feature data are fused and then input into an image classification network to obtain the target classification result of the target location region.
[0118] In some embodiments, the mapping module 303 is further configured to: determine the alignment mapping relationship between the first image and the second image based on corresponding key points in the first image and the second image.
[0119] In some embodiments, the acquisition module 301 is specifically used for:
[0120] Acquire a first image set acquired by the first image acquisition device and a second image set acquired by the second image acquisition device;
[0121] Based on the acquisition time, location, and angle of each image, the first image and the second image that match the same detection region are extracted from the first image set and the second image set.
[0122] In practical applications, the acquisition module 301, candidate region extraction module 302, mapping module 303, target determination module 304, static comparison module 305, and dynamic comparison module 306 can be implemented by the processor in the vehicle-road cooperative image detection device. Of course, the processor needs to run the computer program in the memory to implement its functions.
[0123] It should be noted that the vehicle-road cooperative image detection device provided in the above embodiments is only illustrated by the division of the above-described program modules when performing vehicle-road cooperative image detection. In practical applications, the above processing can be assigned to different program modules as needed, that is, the internal structure of the device can be divided into different program modules to complete all or part of the processing described above. In addition, the vehicle-road cooperative image detection device and the vehicle-road cooperative image detection method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the method embodiments, which will not be repeated here.
[0124] Based on the hardware implementation of the above program modules, and in order to implement the method of the embodiments of this application, the embodiments of this application also provide an image detection device based on vehicle-road cooperation. Figure 4 This is only an exemplary structure of the vehicle-road cooperative image detection device, not the entire structure; it can be implemented as needed. Figure 4 The structure shown may be part or all of the structure.
[0125] like Figure 4As shown, the vehicle-to-infrastructure (V2I) based image detection device 400 provided in this embodiment includes: at least one processor 401, a memory 402, a user interface 403, and at least one network interface 404. The various components in the V2I based image detection device 400 are coupled together via a bus system 405. It can be understood that the bus system 405 is used to implement communication between these components. In addition to a data bus, the bus system 405 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in... Figure 4 The general designated all buses as Bus System 405.
[0126] The user interface 403 may include a monitor, keyboard, mouse, trackball, click wheel, buttons, touchpad, or touch screen.
[0127] The memory 402 in this embodiment is used to store various types of data to support the operation of the vehicle-to-infrastructure (V2I) based image detection device. Examples of such data include any computer program used to operate on the V2I based image detection device.
[0128] The vehicle-to-infrastructure (V2I) image detection method disclosed in this application can be applied to or implemented by processor 401. Processor 401 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the V2I-based image detection method can be completed by integrated logic circuits in the hardware or by software instructions in processor 401. Processor 401 can be a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Processor 401 can implement or execute the methods, steps, and logic block diagrams disclosed in this application. A general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this application can be directly represented as being executed by a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules can be located in a storage medium, specifically memory 402. Processor 401 reads information from memory 402 and, in conjunction with its hardware, completes the steps of the V2I-based image detection method provided in this application.
[0129] In an exemplary embodiment, the vehicle-road cooperative image detection device may be implemented by one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers (MCUs), microprocessors, or other electronic components to perform the aforementioned method.
[0130] It is understood that memory 402 can be volatile memory or non-volatile memory, or both. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), ferromagnetic random access memory (FRAM), flash memory, magnetic surface memory, optical disc, or compact disc read-only memory (CD-ROM); magnetic surface memory can be disk storage or magnetic tape storage. Volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Synchronous Static Random Access Memory (SSRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic Random Access Memory (SDRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDRSDRAM), Enhanced Synchronous Dynamic Random Access Memory (ESDRAM), SyncLink Dynamic Random Access Memory (SLDRAM), and Direct Rambus Random Access Memory (DRRAM).The memories described in the embodiments of this application are intended to include, but are not limited to, these and any other suitable types of memories.
[0131] In an exemplary embodiment, this application also provides a storage medium, namely a computer storage medium, specifically a computer-readable storage medium, such as a memory 402 that stores a computer program. This computer program can be executed by the processor 401 of a vehicle-to-everything (V2X) image detection device to complete the steps described in the method of this application embodiment. The computer-readable storage medium can be a ROM, PROM, EPROM, EEPROM, Flash Memory, magnetic surface memory, optical disc, or CD-ROM, etc.
[0132] It should be noted that terms such as "first" and "second" are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.
[0133] Furthermore, the technical solutions described in the embodiments of this application can be combined arbitrarily without conflict.
[0134] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.
Claims
1. An image detection method based on vehicle-road cooperation, characterized in that, include: The first and second images are obtained by the first image acquisition device on the vehicle-mounted unit side and the second image acquisition device on the roadside unit side, based on the same detection area. Based on the target detection algorithm, the first target candidate region of the first image and the second target candidate region of the second image are determined respectively; Based on the mapping relationship between the first image and the second image, a first virtual target candidate region is determined to be mapped from the first target candidate region to the second image, and a second virtual target candidate region is determined to be mapped from the second target candidate region to the first image. The location region of the target in the detection region is determined based on the first target candidate region, the second target candidate region, the first virtual target candidate region, and the second virtual target candidate region; Wherein, the first image corresponds to the first image acquisition device, and the second image corresponds to the second image acquisition device; The method further includes: Target recognition is performed on the first image based on the location region of the target; The target recognition result is compared with the static obstacles in the pre-constructed detection area; If at least one of the static obstacles is found to be missing in the identification results, a warning message is generated; the static obstacle is determined based on the image acquired by the second image acquisition device.
2. The method according to claim 1, characterized in that, Determining the location region of the target in the detection region based on the first target candidate region, the second target candidate region, the first virtual target candidate region, and the second virtual target candidate region includes: The nonmaximum suppression algorithm is used to traverse the first target candidate region, the second virtual target candidate region, and the second target candidate region and the first virtual target candidate region of the second image to determine the location region of the target in the detection region.
3. The method according to claim 1, characterized in that, The method further includes: Target recognition is performed on both the first image and the second image based on the location region of the target. Determine the target location region where the same object exists in the first image and the second image; Based on the target location region, extract the first feature data of the first image and the second feature data of the second image respectively; The target classification result of the target location region is determined based on the first feature data and the second feature data.
4. The method according to claim 3, characterized in that, The step of determining the target classification result of the target location region based on the first feature data and the second feature data includes: The first feature data and the second feature data are fused and then input into an image classification network to obtain the target classification result of the target location region.
5. The method according to claim 1, characterized in that, The method further includes: The mapping relationship between the alignment of the first image and the second image is determined based on the corresponding key points in the first image and the second image.
6. The method according to claim 1, characterized in that, The first image and the second image obtained by the first image acquisition device on the vehicle-mounted unit side and the second image acquisition device on the roadside unit side based on the same detection area include: Acquire a first image set acquired by the first image acquisition device and a second image set acquired by the second image acquisition device; Based on the acquisition time, location, and angle of each image, the first image and the second image that match the same detection region are extracted from the first image set and the second image set.
7. An image detection device based on vehicle-road cooperation, characterized in that, include: The acquisition module is used to acquire a first image and a second image obtained by the first image acquisition device on the vehicle-mounted unit side and the second image acquisition device on the roadside unit side based on the same detection area; The candidate region extraction module is used to determine the first target candidate region of the first image and the second target candidate region of the second image based on the target detection algorithm. The mapping module is used to determine, based on the mapping relationship between the first image and the second image, a first virtual target candidate region mapped from the first target candidate region to the second image and a second virtual target candidate region mapped from the second target candidate region to the first image; The target determination module is used to determine the location region of the target in the detection region based on the first target candidate region, the second target candidate region, the first virtual target candidate region, and the second virtual target candidate region. Wherein, the first image corresponds to the first image acquisition device, and the second image corresponds to the second image acquisition device; A static comparison module is used to perform target recognition on the first image based on the location region of the target; compare the recognition result of the target recognition with the static obstacles in the pre-constructed detection region; if it is determined that at least one of the static obstacles is missing in the recognition result, a warning message is generated; the static obstacles are determined based on the image acquired by the second image acquisition device.
8. An image detection device based on vehicle-road cooperation, characterized in that, include: A processor and memory for storing computer programs that can run on the processor, wherein, The processor, when running a computer program, performs the steps of the method according to any one of claims 1 to 6.
9. A storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.