A Rapid AVP Vehicle Detection Method Based on Dual Sensing of Lidar and Near-infrared Sensors

Abstract

A rapid AVP vehicle detection method based on radar-thermal infrared dual sensing, comprising: mainly using two types of sensors, i.e., a near infrared imager and a radar receiver, to collect near inf

Description

The present invention relates to the field of AVP vehicle target detection, multi-sensor synergy application, and specifically relates to a fast AVP vehicle detection method based on the joint decision of thermal infrared image data and radar wave signal sensing results. Background technology With the continuous development of China's economic level, people's material living conditions continue to improve, the car has become an indispensable travel tool for more and more families. While enjoying the convenience brought by the continuous growth of automobile ownership, the problems of traffic congestion, traffic accidents, environmental pollution, and resource wastage have also intensified. To solve the above problems, researchers around the world have made rich attempts and made great progress in the optimization of information control algorithms, upgrading of traffic safety hardware, automatic driving, and the application of new energy and new materials in many directions. In recent years, with the continuous development of the Internet of Things, artificial intelligence, robotics and automation technology, among the above directions, the research on automatic driving technology has made the most rapid progress. Many automobile manufacturers and technology companies have invested many resources and funds in the research and development of automatic driving technology. At present, the automatic driving technology has reached the requirements of the assisted driving stage (L2) and is in the transition stage of conditional automatic driving (L3) and highly automatic driving (L4), and in some scenarios, the L4 level of automatic driving function has been realized. Autonomous Valet Parking (AVP) refers to the use of automatic driving technology in parking lots with the goal of achieving L4-level vehicle self-driving parking. Based on the degree of intelligent transformation of vehicles and parking lots, the current industry has identified three technical paths to realizing AVP: single-vehicle intelligence, intelligent field ends, and vehicle-field integration. The first type, single-vehicle intelligence, focuses on the intelligence of individual vehicles. Single-vehicle intelligence involves transforming a vehicle so that it is capable of independently completing the AVP driving task. Technical terms will be explained on their first use. Upgrades to the vehicle hardware include integrating various sensors like LiDAR, fish-eye cameras, and ultrasonic radar, as well as software support for path planning, path memory, and real-time obstacle avoidance algorithms. Based on this, the vehicle can autonomously perform positioning, sensing, decision-making, and control for autonomous parking operations within the underground garage. The intelligent field endpoint. The intelligent infrastructure at field end sends physical information, such as accurate road data and location and behavior information of other vehicles and pedestrians, from specific points like parking lot channels, parking spaces, and columns, using multi-source sensors to assist in autonomous driving without the need for sensing equipment in the car. This includes car-field fusion. Vehicle-field fusion involves integrating real-time vehicle and road information acquired through sensing technologies at both vehicle and field ends, using advanced wireless communication technology between vehicles and intelligent roadside facilities in parking lots. This enables information interaction and sharing to achieve intelligent and cooperative autonomous parking of vehicles and fields. To achieve this, technical terms are explained when first used, and clear, concise, and value-neutral language is employed throughout. Due to the costs associated with the field's transition to intelligent technology, and the limited areas of application for such technology, most domestic commercial AVP applications rely on the vehicle's 1 own autonomous parking capabilities and use a single-vehicle intelligent technology route. However, most non-intelligent parking lots lack the ability to differentiate between regular and automated vehicles, as well as the capability to communicate with automated vehicles. This lack of awareness creates difficulties for the parking lot management, poses traffic safety hazards, and increases the risk of preventable accidents. Additionally, determining accountability can become problematic in situations where the AVP vehicle parking procedure malfunctions and obstructs traffic flow. AVP vehicles occupy private parking spaces in the parking lot and are not notified; they are also prone to friction accidents with other vehicles. AVP vehicle rapid detection involves accurately and swiftly determining the vehicle type in a parking lot, while addressing AVP vehicle identification issues. This is essential for precise vehicle control, identifying potential hazards, and enabling vehicle-to-road communication. The AVP vehicle detection technology is designed primarily for the car park fusion AVP scenario. Ihc remote device end and AVP vehicle end work together by utilizing a monitoring system and remote assistance features. Tins allows for real-time monitoring of self-parking m the parking lot vehicle status. For instance, the AVP vehicle end can report any abnormal / waming information, and the system can conduct real-time video analysis, while human monitoring can help detect potential dangers. Thus, remote assistance is provided in real-time to assist the vehicle. All technical terms are explained at their first use. However, this detection technology relies on establishing a communication protocol within the car park, limiting its scope of application. Furthermore, it cannot detect AVP vehicles based on the intelligent bicycle technology method, leading to its limited effectiveness. Additionally, the sensors installed on AVP vehicles, such as LIDAR, ultrasonic radar, microwave radar, and camera, are designed to be concealed, with minimal visual differences compared to regular vehicles. Furthermore, the sensor types utilized are nearly identical to those found in vehicles equipped with parking assistance, making it difficult to distinguish solely based on appearance. If the AVP vehicle type is to be accurately recognized, machine vision that rely on video image data is insufficient. Prior art Patent CN113538927A Patent CN106297373B Patent CN109635733B Patent CN201218990Y Explanation of terms In order to make the description of the present invention more accurate and clear, various terms that will appear in the present invention are explained as follows: 1. AVP: Autonomous Valet Parking (AVP) technology enables autonomous driving through field sensors and vehicle control systems. It allows vehicles to autonomously find parking spaces, park and depart the parking space to return to the driver's location upon command. 2. AVP vehicles: vehicles equipped with AVP technology that allows for driverless, autonomous parking and vehicle pickup in parking lots. 3. Assisted Parking: a parking assistance system equipped with on-board radar and camera sensors that activate when parking to transmit obstacle information to the on-board display terminal. Although vehicles with this feature have radar and camera sensors, they are only activated when parking and reversing into a garage. 4. Field-side information: accurate information about the infrastructure and physical attributes 2 of the parking lot, including the number of parking spaces, their size, distribution, and location. Information about the road layout, length, and width is also necessary, along with the number and location of road gates and bollards. Additionally, information about the location, type, and speed of vehicles, as well as the location and speed of pedestrians, is imperative. □ 5. Parking lot manager: refers to the operation and management unit of a parking lot, who wishes to detect AVP vehicles operating in the parking lot in order to accurately control the vehicles and reduce the safety hazards of automated driving in the field. 6. Vehicle radar: refers to a collection of ultrasonic radar and lidar mounted on both sides of the body of the AVP vehicle, which are activated throughout the unmanned process of the AVP vehicle to detect Field-side physical information. 7. Concealed design: refers to the design method in which the AVP vehicle embeds the radar and camera sensors in the vehicle body, hiding the sensor body and retaining only a small area of the lens on the surface of the vehicle body. 8. Dual sensing of radar and heat: refers to the detection of heat radiation and radar wave signals in AVP vehicles while driving, using two types of sensors: near-infrared imagers and radar receivers. 9. Near-infrared imager: refers to an active thermal imaging system with a near-infrared emitter that emits near-infrared waves in the range of 750-1400 nanometers in order to obtain a thermal image of a target object. 10. Radar receiver: refers to a device for receiving radar wave signals, generally consisting of a preamplifier, a mixer, an IF amplifier and a demodulator. 11. Edge computing unit: refers to a computing device deployed in the parking lot to receive, process and store data collected in the field. 12. Otsu: Otsu (Otsu's thresholding method) is an adaptive image thresholding segmentation algorithm that divides the pixels in an image into two classes such that the intra-class variance is minimized and the inter-class variance is maximized. 13. PRI: PRI (Pulse Repetition Interval) is a radar signal processing technique that classifies and sorts received radar signals according to their PRI for radar signal identification, classification and processing 14. Deep learning target detection algorithm: refers to a target detection algorithm based on residual networks, which performs target detection of vehicles and drivers in near-infrared images. 15. Residual Network (ResNet): refers to a deep learning neural network structure whose main idea is to solve the problem of gradient vanishing and model degradation in deep networks by adding cross-layer connections. 16. Dropout: refers to randomly setting the output of some neurons to 0 during residual network training, which can force the model to learn multiple sub-models that are independent of each other, thus avoiding overfitting. 17. Batch Normalization: refers to a technique that normalizes the inputs of each layer in the residual network. This stabilizes the distribution of input data, leading to improved network convergence and generalization ability. 18. Vehicle Type A: vehicle Target Recognition Result Marker, 0 / 1 variable, value 0 for normal vehicles, value 1 for AVP vehicles. 19. Confidence I: refers to the degree of confidence that a target is recognized as belonging to a category . Expressed as a value between [0,1], the closer the value is to 1, the more confident the 3 system is that the target belongs to that category. 20. Main access road: the internal road with the highest volume of traffic in the parking lot. 21. Grayscale value: refers to the brightness value of each pixel in the NIR image, expressed as a number between [0-255], with a higher grayscale value indicating that the pixel is brighter and whiter, and a lower grayscale value indicating that the pixel is less bright and darker 22. Foreground area / background area: in the processing of near-infrared images, only certain parts of the image are of interest, which are called the foreground area, and the rest is the background area. 23. Radar Characteristic Parameters: refers to quantities used to describe and characterize the properties of radar signals, which include: angle of arrival, carrier frequency, time of arrival, pulse width, pulse amplitude, and in-pulse modulation characteristics. 24. AVP Vehicle Mounted Radar (VMR) Database: refers to a database that stores the parameters of the AVP VMR sensors. The database is pre-established and updated regularly with publicly available information. Content of the invention The invention tackles the AVP application scenario through the implementation of single-vehicle intelligent technology, resolving the issue where parking lot managers are unable to detect AVP vehicles in the field with accuracy and speed. It is no longer necessary for the parking lot management to comply with the requirements for constructing intelligent field-end equipment or establishing communication with AVP vehicles to detect information autonomously and prevent traffic accidents that could occur during autonomous valet parking. The main challenge that needs to be addressed is differentiating between AVP vehicles and ordinary vehicles. Since many of the radar and camera sensors carried by AVP vehicles have a concealed design, and some of the vehicles with parking assistance are also fitted with the same type of radar and camera sensors, distinguishing between the two by appearance alone is impossible using machine vision. The current invention analyzes the conduct of AVP vehicles and incorporates the following technical solution concept: A rapid detection method is developed for AVP vehicles, which uses ray-thermal dual perception technology. Uris method collects data through a near infrared thermal imager and a radar signal receiver and analyzes the behavioral characteristics of AVP vehicles that distinguish them from normal vehicles to detect AVP vehicles swiftly. The specific programs are: (1) Select an appropriate location on the parking lot bridge to install an RI near-infrared thermal imager. This location should provide a viewing angle that encompasses the front windshield of vehicles in the primary channel of the parking lot. When a vehicle passes through the channel, capture thermal imaging data. Apply three preprocessing steps to the infrared image, including image denoising, enhancement, and segmentation. Use a deep learning-based target detection algorithm to identify the target of the processed infrared image. Extract the image vehicle and driver targets and filter the detection results accordingly. If the vehicle target is detected, but the driver target is not, designate the current vehicle thermal sensing result as an AVP vehicle. Output the vehicle type marker Ari and confidence level Iri . (2) Deploy radar signal receiver R2 in a suitable area on both sides of the main channel at the parking lot. R2 receives radar wave signals within the direction of the main channel. The received signal undergoes pre-processing and main processing involving two-step signal sorting to determine the 4 characteristics parameters. The feature vector of the radar wave signals is compared to the pre-established AVP vehicle radar database. To identify the main channel, exclude subjective evaluations unless labeled as such. If vehicle-mounted radar is detected in the main channel and matches the database, the current vehicle radar sensing result is an AVP vehicle. The output vehicle type marker is Ar2, with a confidence level of Ir2. For the results detected by the two types of perception, and gate logic decision is used to output a comprehensive decision result for the AVP vehicle when and only when both types of detection judgments are true, and the confidence of the comprehensive decision result is obtained based on the weighting of the perceptual confidence. The present invention specifically and primarily relates to technical issues including the following aspects: 1. NIR image denoising techniques; 2. NIR image enhancement techniques; 3. NIR image segmentation techniques; 4. near-infrared image screen target recognition techniques; 5. radar signal sorting techniques; 6. radar signal feature matching techniques; 7. AVP vehicle type synthesis decision-making method; The current invention utilizes Near-infrared Imager and Radar Receiver devices to quickly detect AVP vehicles in underground garages. Near-infrared Imager and Radar Receiver offer distinct advantages when compared to other types of sensors. The Near-Infrared Imager is an active thermal imaging system that uses a near-infrared emitter to generate thermal radiation to capture images of the target object. The system typically operates at wavelengths between 750-1400 nanometers, enabling infrared waves in this range to penetrate certain materials (such as glass, plastic, etc.) without being absorbed. This allows for better penetration depth and detection sensitivity-, and avoids limitations imposed by the detection range of traditional thermal imaging systems. Technical term abbreviations will be explained upon first use. In the AVP vehicle detection scenario, the NIR thermal imaging camera offers distinct advantages, including: (1) The ability to operate without a light source thanks to its infrared emitter, making it viable for low-light environments, like underground parking lots, where video cameras are less effective. (2) Strong scene applicability. The NIR camera provides precise thermal imaging measurements and utilizes grayscale differences to swiftly and accurately detect vehicles and drivers for AVP vehicle detection. (3) Stable imaging. Thermal infrared imaging is less affected by atmospheric attenuation due to the scattering effects of atmospheric gas molecules and aerosols during transmission, leading to more stable grayscale values in the final thermal infrared image. (4) Reliability. NIR cameras can function effectively in various severe environments, including low and high temperatures, humidity, and more. This feature guarantees stable performance over an extended period of use. Radar receiver is used to receive radar signals, which generally consists of preamplifier, mixer, IF amplifier and demodulator. Its main function is to receive and convert the high-frequency electromagnetic wave signals emitted by the radar transmitter into low-frequency digital signals, and then carry out the signal processing and analysis, to realize the function of radar wave reception, sorting and identification. In the AVP vehicle detection scene, its advantages are mainly the following: (1) High sensitivity: Radar receiver can receive small power radar signals, so it can detect the radar 5 signals emitted by AVP vehicle radar. (2) Strong anti-interference: Radar receiver can suppress and filter the interference signal by means of filter and demodulator, etc., so as to improve its anti-interference performance,. (3) Non-contact detection: Radar receiver does not need to contact the transmitting source, which will not cause harm or interference to the radar transmitter, and is suitable for AVP vehicle detection scenarios. The technical program adopted by the present invention to solve its technical problems is described in the following specific process: 1. Data acquisition: the Near-infrared image data within the field of view is acquired using the Near-infrared imager deployed in the suspension, and the Radar Characteristic Parameters of the points within the Main access road of the parking lot are received using the Radar Signal Receiver deployed on the side of the road. Hie Near-infrared imager and the Radar Signal Receiver can be deployed in several variants as follows: Variant A: Near-infrared imager is suspended from the bridge above the Main access road of the parking lot, and Radar receiver is wall-mounted at the bottom of the column on the roadside of the Main access road of the parking lot, and the horizontal distance between the two sensors is less than 5 meters. Variant B: Near-infrared imager is suspended on the bridge above the parking lot Main access road, support poles are installed on both sides of the parking lot Main access road, Radar receiver is polemounted, and the horizontal distance between the two sensors is less than 5 meters. Variant C: Near-infrared imager is wall-mounted on the top of the column on the left side of the parking lot Main access road, Radar receiver is wall-mounted on the bottom of the columns on both sides of the parking lot Main access road, the horizontal distance between the two sensors is less than 5 meters. After deployment, the following options are available for acquiring data from the Near-infrared imager and Radar receiver: Variant A: The near-infrared imager captures thermal infrared images at a rate of 30 frames per second. Its acquisition range covers the space enclosed by the 60° angle between the two measurements in the center of the channel. The radar receiver acquires radar wave data at a frequency of 500 MHz, and its acquisition range covers the space opposite to the columns on both sides of the channel. Variant B: Tire near-infrared imager collects thermal infrared images at 20 frames per second, the acquisition range is the space surrounded by the 60° angle between the two measurements in the center of the channel; Radar receiver collects radar wave data at a frequency of 1,000 MHz, and the acquisition range is the space opposite to the support poles on both sides of the channel. Variant C: The near-infrared imaging device captures thermal infrared images at a rate of 60 frames per second. It collects data from the space encompassed by a 120° angle extending from the left column of the channel to the center of the channel. In contrast, the radar receiver records radar wave data at a frequency of 300 MHz, drawing information from the area opposite to the columns on either side of the channel. After completing data acquisition, the Near-infrared imager and Radar receiver transmit the acquired data through wired transmission to the Edge computing unit situated in the parking lot's computer room for further calculation. 2. Data Processing: The edge computing unit processes the data acquired by the near-infrared imager and radar receiver, including steps for the near-infrared image data and radar wave signals. (1) For near-infrared image data, accompanied by unavoidable environmental noise or system 6 noise, there are unclear contours and poor contrast, which need to go through the pre-processing steps of image denoising, image enhancement and image segmentation before they can be used for subsequent recognition. Among them, the NIR image denoising adopts the common mean filtering method or median filtering method: Variant A: NIR image denoising using mean filtering. Variant B: NIR image denoising using median filtering. NIR image enhancement uses histogram equalization or gray-scale linear transformation to enhance certain features of the image, such as edges, contours, contrast, etc. for enhancement to achieve the highlighting and emphasizing of image detail information: Variant A: NIR image enhancement using histogram equalization. Variant B: NIR image enhancement processing using gray scale linear transformation. Image segmentation is performed using Otsu threshold separation method or region growing method to separate the region of interest in the image and reduce the influence of other objects in the parking lot. Variant A: Otsu threshold separation method is used for NIR image region of interest segmentation processing. Variant B: Region growing method for region of interest segmentation in NIR image. (2) For radar wave signals, the actual signal received by the receiver is an interlaced column of pulses that is composed of multiple radar transmitting sources and other electromagnetic interference and electromagnetic noise pulse signals interleaved within the parking lot. Signal sorting is the step of stripping out different radar signals from it for subsequent signal identification. Signal sorting is mainly divided into two steps: preprocessing and main processing. The preprocessing of signal sorting uses an adaptive filter-based algorithm or a time-domain peak detection algorithm to perform pulse parameter matching analysis in the radar wave signal stream, from which known radar wave trains are separated and deducted to obtain pre-sorted resolution units: Variant A: near-radar wave signal sorting preprocessing using an adaptive filter. Variant B: near-radar wave signal binning pre-processing using time-domain peak detection. The main processing uses the PRI sorting algorithm to perform PRI sorting on the pre-sorted discriminative units to separate out individual columns of radar pulses. 3. Target recognition: the data processed by the Edge computing unit is used for target recognition. (1) For near-infrared image data, use the pre-trained deep learning target detection algorithm to perform target recognition on the images in the region of interest, and the recognized objects are divided into two categories: vehicle and driver, and get the recognition frames and corresponding confidence levels of the two categories of taigets. (2) For radar wave signals, target recognition is carried out based on the known AVP vehicle radar database, and the signal sorted data are matched with the radar wave signals in the database by radar characteristic parameters to obtain the detected radar type and confidence level in the main access road. 4. AVP Vehicle Type Integrated Decision Making: Determine the vehicle recognition confidence threshold at 0.75 and the driver recognition confidence threshold at 0.5 for the two targets recognized by NIR images. Recognize the vehicle when the vehicle recognition confidence exceeds 0.75 and do not recognize the driver when the driver recognition confidence is less than 0.5. The AVP vehicle can only be identified if the driver is not recognized, and the vehicle is identified. This identification is made using the vehicle type marker Ari with the confidence level Iri, which is an average of two types of target confidence levels. For radar targets, a 0.7 confidence threshold is set and only the target with the highest confidence level in the database signal is chosen. When the confidence level exceeds 0.7 and the judgment coincides with the radar signal in the database, the Vehicle Type Marker Ar2, along with the confidence level Ir2, is outputted. Ari and Ar2 then undergo gate decision-making logic, which results in the integrated decision-making output for the AVP vehicle only when both detection judgments are true. The integrated decision-making output for confidence I is based on the weighted perceptual confidence levels Iri and Ir2. A brief description of the accompanying figures Figure 1 A flowchart of the rapid AVP vehicle detection method based on dual sensing of lidar and near-infrared Sensors technology route; Figure 2 Flowchart of AVP vehicle type synthesis decision; Figure 3 Schematic diagram of near-infrared imager deployment; Figure 4 Radar receiver layout schematic; Figure 5 Near-mfrared image denoising technology roadmap; Figure 6 Near-infrared image enhancement technology roadmap; Figure 7 NIR image segmentation technology roadmap; Figure 8 radar signal sorting pre-processing technology roadmap; Figure 9 radar signal sorting main processing technology roadmap; Specific embodiments of the present invention The following invention is described m detail below alongside specific embodiments and accompanying drawings. It concerns a Rapid AVP Vehicle Detection Method which depends on Dual Sensing of Lidar and Near-infrared Sensors. The overall technical route flowchart appears in Fig. 1 and is divided into four steps, namely sensor deployment, data pre-processing, multi-target identification, and integrated decision-making. Each of these steps is further divided into two parts for discussion. Step 1: Sensor Deployment This study utilizes two types of sensor data, namely Near-infrared imager and Radar receiver, to achieve fast detection of AVP vehicles. Among them, the Near-infrared imager that features a semiconductor laser diode actively emits infrared waves with wavelengths ranging from 750 to 1400 nanometers and emission frequencies spanning from a few hundred megahertz to several gigahertz, exhibiting a high emission frequency, an outstanding detection sensitivity, and excellent system spatial resolution. In this regard, the typical application scenarios and related data acquisition modes are presented and discussed below. Typical application scheme A: As illustrated in Figure 3 for the standard application scheme, the deployment mode of the Nearinfrared imager involves lifting it in the parking lot above the Main access road. The sensor's detection range varies based on the installation height and angle adjustment. Make sure tire lift height is 2.2 meters or higher to comply with the parking lot's net height requirements. Use a 20-degree downward tilt angle and a maximum 120-degree horizontal detection angle to detect vehicles and drivers within 100 meters of the main access road. A radar receiver deployment scheme is depicted in Figure 4. Tire radar receiver installation height is limited to 0.5 meters in order to ensure that it can receive the echo signal of the vehicle radar clearly. The receivers are wall-mounted on both sides of the vehicle through the Main Access Road to ensure the best radar wave signal reception. The wall mount is utilized on either side of the primary access road that the vehicle travels to ensure optimal reception of the radar wave signal. The 8 sensors are placed less than 5 meters apart horizontally. After the installation is complete, the Near-infrared imager collects thermal infrared image data at a rate of 30 frames per second. The collection range covers the space within a 60° angle on both sides of the center of the channel. The Radar receiver collects radar wave data with a frequency of 500 MHz. The collection range is the space opposite to the columns on both sides of the channel. Typical application scheme B: The Near-infrared imager is mounted on the wall and deployed on the left side of the main access road in the parking lot above the top of the column. The sensor detection range is adjusted based on the installation height and angle. Ensure that the wall-mounted sensor is at least 2.2 meters high to comply with the parking lot's clearance requirements. Set the downward tilt angle to 20° and install it at the top of the left column on the main access road. Adjust the horizontal tilt angle to 40°, to ensure detection of vehicles and drivers within 100 meters of the main access road, covering the left side of the vehicles and the drivers. Typical Application Scheme B involves a wall-mounted Radar receiver deployed on the columns located on both sides of the bottom of the Main access road in a parking lot. The installation height should be less than or equal to 0.3 meters to ensure optimal reception of echo signals from the Vehicle radar. Additionally, the vertical distance between the two sensors must be greater than 2 meters for effective functioning. After installation is complete, the Near-infrared imager gathers thermal infrared image data at a rate of 60 frames per second. The collection scope encompasses the area encompassed by a 120° angle centered on the access road's midpoint, beginning from the column on the left side. The Radar receiver records radar wave data at a frequency of 300 MHz, covering the space opposite to the columns on both sides of the access road. Technical abbreviations will be explained upon first usage. Step 2: Data Preprocessing (1) Near-infrared image noise removal Near-infrared image noise is generally caused by sensitive components, transmission channels, etc. and belongs to random noise and particle noise. Removing near-infrared image noise involves filtering out random noise while overcoming image detail blurring caused by the field average. Typical application scheme A: Typical application Scheme A utilizes mean filtering to eliminate noise. Mean value filtering is a leading method to remove image noise, based on linear smoothing, where the average value of pixel grayscale in a local area is applied to establish the pixel transition value, producing an image smoothing effect. The algorithm's steps are as follows: a) Select a suitable filter size, a 3x3 window is selected for this invention. For each pixel in the image, calculate the average value of its neighboring pixels. b) Replace the Grayscale value of that pixel with the average value. c) Repeat steps a) and b) until the entire image is processed. The mathematical formula is given below: n-1n-1 1 y y / n — 1 n — 1\ / f\x + i--—,y+j--— Z—i Z—i \ 2 2 / i=0 / =0 Where, (x,y) is the coordinates of the pixel to be processed, (i, j) is the coordinates of the neighboring pixel, f(x,y) is the grayscale value of the pixel to be processed, the neighboring pixel is f (i, j), and g(x,y) denotes the grayscale value of the pixel after the mean value filtering. The size of the mean value filter is n. Typical application scheme B: Typical application scheme B uses median filtering to remove noise. Median filtering is a commonly used image denoising method based on nonlinear filtering, which is based on the principle of replacing the current pixel's grayscale value with the intermediate grayscale value in the local neighborhood, thus removing the random noise from the image. The steps of the algorithm are as follows: a) Select a suitable filter size, a 5x5 square filter is selected for this invention. b) For each pixel (x, y) in the image, extract the grayscale value of all pixels within the [n,x,n] neighborhood centered on that pixel. c) Sort all pixel grayscale values within the neighborhood and select the middle value as the grayscale value of the current pixel, if the number of pixels in the neighborhood is even, select the average of the middle two pixels as the Grayscale value of the current pixel. d) Repeat steps 2-3 until the whole image is processed. The mathematical formula is given below: g(x,y) = median{f(x + i — 2n — l,y + j — 2n — € [0,n — 1] Where (x,y) is the coordinates of the pixel to be processed, (i,j) is the coordinates of the neighboring pixel, f(x,y) is the grayscale value of the pixel to be processed, the neighboring pixel is f(i, j), and g(x, y) denotes the grayscale value of the pixel after mean filtering. The size of the median filter is n. (2) NIR image enhancement NIR image enhancement is to enhance the contrast of the NIR image and widen the range of image gray scale changes to achieve a clear visual hierarchy effect, which is more conducive to the subsequent target recognition of vehicles and drivers. Typical application scheme A: Typical application scheme A uses the commonly used histogram equalization algorithm to achieve NIR image enhancement. The histogram of a NIR image depicts a Probability Distribution Function of the gray levels. The horizontal axis represents the gray levels, while die vertical axis shows the frequency of the occurrence of each gray level. This relationship between a gray level in an image and its frequency of occurrence is effectively portrayed by the histogram. The range of gray level variation in the general image within 100 will be expanded uniformly to the full range of gray levels through histogram equalization in order to achieve enhanced contrast. Let f(a, b) be the grayscale value of the original image, g(a, b) be the enhanced grayscale value, h(r) be the grayscale histogram of the original image, nr be the number of pixel points with the grayscale level of nr, M and N be the width and height of the image, respectively, and L be the number of the grayscale levels of the image, and the specific steps of histogram equalization are as follows: a) Histogram of statistical images: nr h(r) = r = 0,1,2,•••,L — 1 b) Histogram Normalization: h(r) Pr MN c) Calculating the Cumulative Distribution Function: r d) Normalize the cumulative distribution function: 10 . (L - l)cr r MN e) Replacing pixel values in the original image with values from the cumulative distribution function: g(a, b) = df(a,b) Typical Application scheme B: Typical application scheme B uses a commonly used grayscale linear enhancement algorithm to achieve NIR image enhancement. Grayscale linear transformation enhancement is a commonly used image enhancement method, which adjusts the image pixel grayscale by linear mapping, resulting in enhanced contrast and clearer details. Assuming g (x, y) is the grayscale value of the original image, g'(x, y) is the grayscale value after enhancement, and k and b are the scale factor and constant term of the linear transformation, the specific steps of the grayscale linear enhancement are as follows: a) Calculate the minimum gray level gmin and maximum gray level gmax of the image. b) Calculate the scale factor k and constant term b of the linear transformation: b — a k =--------- gmax gmin k h a (gmax gmin) c) Apply a linear transformation to each pixel to calculate the adjusted pixel value: g'(x,y) = k-g(x,y) + b d) Limit the adjusted image pixel grayscale value to between 0 and 255. g'(x, y) = k-g(x,y) + b (3) NIR image segmentation NIR image segmentation is to find the region of interest in the image, i.e., main access road area, for the foreground area of the scene, and other parking vehicles, infrastructure in the parking lot, etc. for the background area of the scene. Typical Application scheme A: Typical application scheme A uses Otsu threshold segmentation method to realize NIR image segmentation. Otsu threshold segmentation method is an image segmentation method based on histogram analysis, which determines the optimal threshold by calculating the variance of the image. The specific steps of Otsu threshold segmentation method are as follows: a)Set the NIR image f (x, y) and calculate the histogram of its gray levels, i.e., calculate the number of pixels in the image for each gray level. b) Initialize the threshold value to,which takes values cycling between 0-255, to classify the NIR image f(x, y) into two categories and / 2. c) For each possible threshold t, calculate the intra-class variance w^t) and inter-class variance w2 (t) of the two classes of images and calculate their weighted sum w(t): Intra-class variance w1 (t): wi(t) =       0 _ fiiOO) Inter-class variance w2 (t): W2(t) = X where L is the number of gray levels in the image, p(0 is the ratio of the number of pixels of gray level i in the image to the total number of pixels. m(t) and / z2(0 are the average gray levels of the pixels on both sides of the threshold t, respectively. Weighted sum w(t): w(t) = p(t)Wi(t) + (1- p(t))w2(t) Find the threshold t* that minimizes the weighted sum w(t): t* = arg min w (t) The threshold t* is applied to the original NIR image which is segmented into two categories and f2, i.e., the foreground and background areas of the final output image. Typical application scheme B: Typical application scheme B uses region growing method to realize NIR image segmentation. The region growing method clusters neighboring pixels into a region based on the similarity between pixels in the image. In the process of region growing, the initial seed points (i.e., pixels with known categories) are gradually expanded to the surrounding pixels until the neighboring pixels satisfy certain similarity criteria. The specific steps of the region growing method are as follows: a) Select the initial seed point (M,N) and set tire similarity threshold Ag. b) According to the set similarity threshold Ag, add pixel i with similarity higher than the threshold kg to the seed point (M,N) to the current region and mark it as visited. c) iteratively add the neighboring pixel point j in the current region that is not visited to the region. d)Until no new pixel point is added to the region or until a predetermined growth condition is reached, the growth ends. e) Post-processing the region after the end of growth to remove small, isolated regions, merge overlapping regions, and so on. The infrared image segmentation uses a region growing algorithm based on the difference in grayscale value, which is mathematicallv formulated as: |fi-fj| <Ag where fa and fj denote the grayscale value of neighboring pixels i and j, respectively, and &g denotes a set threshold of gray scale difference. When the difference in Grayscale value between pixels is less than the set threshold, these two pixels are considered similar. 2. Radar receiver (1) Signal sorting preprocessing Signal sorting preprocessing separates the known radar wave trains and unknown radar pulse trains to extract the target radar signals, thus obtaining the pre-sorted resolution units. Commonly used preprocessing algorithms include matched filtering, forward-backward averaging filtering, time-domain peak detection and so on. Typical application scheme A: Typical application scheme A uses an adaptive filter to process the original radar signal in order to realize the matching analysis of pulse parameters, and the specific steps of the adaptive dynamic filtering algorithm are as follows: a) Calculate the autocorrelation function The autocorrelation function reflects the periodicity and correlation of the radar signal. For radar 12 signal x(n), its autocorrelation function Rxx(k) is defined as follows: Rxx(k) x(n)x(n — k) b) Calculation of adaptive filter coefficients The coefficients w(n) of the adaptive filter can be calculated based on the autocorrelation function of the signal. The input of the adaptive filter is the original radar signal x(n) and the output is the filtered signal y(n) .The filter coefficients are updated using the LMS algorithm, which can be calculated by the following equation: w(n + 1) = w(n) + pe(n)x(n) where / z denotes the learning rate, and e(n) denotes the error signal of the filter. c) Filter output The output of the adaptive filter is the filtered signal y (n) which can be calculated by the following equation: M-l w(m)x(n — m) m=O where M denotes the order of the filter. d) Detennination of pre-sorting resolution unit The filtered signal y (n) obtained by the adaptive filter is the position of the pre-selection resolution unit, and the pre-selection resolution unit is further processed in the next step. Typical application scheme B: Typical application scheme B uses time-domain peak detection to process the original radar signal, and its basic principle is to find the peak value of the radar signal in the time domain and separate the target signal accordingly. The specific steps of time domain peak detection are as follows: a) Signal filtering. To remove noise and interference, the original signal for low-pass filtering and high-pass filtering processing. Low-pass filter: b(k) * x(n — k) k=O..M High-pass filter: y(n) = x(n) - x(n - 1) Where b(k) is the coefficient of the filter, x(n) is the input signal and y(n) is the output signal. b) Local maximum method peak detection. Set a fixed window size W, divide the original signal into several windows, for each window, calculate its maximum value, and find the peak value. For each peak, determine whether it is greater than a given threshold, if so, mark it as the target signal, otherwise mark it as noise or interference. Extract the peak position information: pos = argmax(x(n)) Where pos is the position of the peak in the original signal and argmax denotes the position where the maximum value is sought c) For all the peaks labeled as target signals, extract their position information in the original signal and use it as the position of the pre-sorted resolution unit. (2) Signal sorting main processing The pre-sorted discriminating unit obtained after signal sorting pre-processing removes the background electromagnetic wave signal, but it may still contain multiple added radar wave signals, and it is necessary to carry out the signal sorting main processing to further obtain a single radar wave signal. The present invention uses PRI sorting algorithm to realize radar signal sorting main processing. The main principle of the PRI sorting algorithm is to classify the received radar signals according to the pulse repetition interval (PRI), so as to separate the individual radar pulse columns. The steps of the PRI sorting algorithm are as follows: a) Analyze the pre-selected discriminators in the time or frequency domain to obtain their time or frequency domain representations. b) Estimate the pulse repetition interval using autocorrelation function or cross-correlation function and perform PRI sorting. The autocorrelation function is the integral of the product of a signal and itself after a certain amount of translation in time, indicating the degree of similarity between the signal and itself; the mutual correlation function is a measure of the degree of similarity between two signals, indicating the correlation between two signals in time. The formula is as follows: Autocorrelation function: R(t) = f r(t)r(t — r)dt J — oo correlation function: C(t) = [ r(t)s(t — r)dt J —CO where t denotes the amount of time of the translation, r(t) and s(t) denote the amplitude of the two original signals at time t, respectively. R(t) denotes the value of the autocorrelation function at a time translation of t. C(t) denotes the value of the cross-correlation function at a time translation of r. PRI Estimation: where k is the number of PRIs and T; is the estimated value of the i th PRI. PRI sorting: |T-T| <AT where T is tire known PRI and AT is the PRI deviation range. c) Compare the PRI estimates with the preset PRI deviation range to filter out the eligible PRIs, and the separated individual radar wave signals are used for subsequent target identification. Step 3: Multi-target Recognition 1. Near-infrared image target identification Typical application scheme A: Typical application scheme A For the preprocessed NIR image data, a deep learning model based on Residual Network for multi-target recognition model is used, which consists of two main parts: a feature extraction network and a classification network. The feature extraction network uses a pretrained ResNet network for extracting features from the image. The classification network uses a frilly connected layer to map the extracted features to a class probability distribution. And regularization and dropout, batch normalization are added to improve the accuracy of the model. Two public datasets are used for pre-training: the NIR-VIS Face Database dataset, which contains face images from two different wavelength bands (visible and NIR), and the FLIR Human and Vehicle dataset, which contains human and vehicle images from FLIR thermal imaging cameras. The model 14 training phase uses cross entropy as the loss function and the model parameters are optimized using the gradient descent method or its variant algorithms. Using the pre-trained ResNet network model for NIR image recognition, the model is deployed in the edge computing unit in the parking lot machine room. The edge computing unit receives the data preprocessed for vehicle and driver target recognition, and outputs the recognition target box, target ty pe T(car / driver), and confidence level It. Typical Application scheme B: Typical application scheme B is based on the Y0L0v8 framework to realize the target recognition of NIR images. Specifically, the Y0L0v8 Nano model is used for detection, which is the fastest and smallest class among the five detection, segmentation, and classification models ofYOLOv8, and is sufficient to meet the needs of target recognition for parking lot vehicles and drivers in NIR images. The NIR-VIS Face Database dataset and FLIR dataset were used for pre-training, the cross-entropy loss function was used to measure the performance of the model, and the model parameters were optimized using gradient descent. The trained Y0L0v8 Nano model is deployed on the edge computing unit or cloud computing platform, and the OpenCV tool is used to integrate the model with other systems. The edge computing unit receives the data preprocessed for vehicle and driver target recognition, and outputs the recognition target frame, target type T(car / driver) and confidence level It. 2. Radar wave signal recognition Radar wave signal identification uses a feature matching algorithm to extract the pre-processed single column radar wave feature vectors and match them with the feature vectors of the pre-established AVP vehicle mounted radar database respectively. The AVP vehicle mounted radar database is established based on the existing publicly available dataset, and different The AVP vehicle mounted radar database is built based on the existing public dataset, and the Vehicle radar models and feature vectors of different brands of AVP vehicles are collected to update the dataset. Extracting the preprocessed single column radar wave feature vectors refers to extracting the feature parameters from the received radar signals that can characterize the signals. The feature parameters usually used include signal spectrum, pulse width, repetition period, etc. Its mathematical expression is as follows: F(f,t) = J f(t)e-i2Hftdt where / (t) is the received radar signal and F{f, t) is the frequency domain representation of the signal. The feature matching algorithm compares the characteristic parameters of the received radar signals with samples from a pre-established AVP vehicle mounted radar database to determine the type of the signal source. Correlation matching is used, which is performed by calculating the similarity between two signals. The correlation coefficient can be expressed as the product of the inner products of the two signals divided by the product of their paradigms with the following mathematical expression: . , fTx(t)s(t)dt r(x, s) = ---- Jf$x2(t)dtfjs2(t)dt Where x(t) is the received signal, s(t) is tire signal sample in the database, and r(x, s) is the correlation coefficient of the two. Sequentially compare the correlation coefficients of the received signal and the radar database 15 sample matching function, take the correlation coefficients as the confidence level of radar perception, and select the radar database sample signal with the highest confidence level as the recognition result. Step 4: Comprehensive Decision Making As depicted in Figure 2 of the integrated decision-making flowchart for AVP vehicles, the system integrates two types of perception results - radar and heat - to make decisions. For both types of targets identified by the NIR image, set the vehicle recognition confidence threshold to 0.75 and the driver recognition confidence threshold to 0.5. If the vehicle recognition confidence Icar>0.75, it is considered recognized. If the driver recognition confidence Idriver<0.5, the driver is considered unrecognized. The AVP vehicle is recognized only when it is identified, and the driver is not. The confidence level Iri for vehicle type marker Ari, which is obtained by averaging the confidence levels of both targets, is calculated as follows: . _ I car + (1 I driver) Iri - 2 For the radar target, a confidence threshold of 0.7 is set, and the identification target is chosen by selecting the database signal with the greatest confidence level, which is then recorded as Ladar max. When the confidence level exceeds 0.7, assess the match with the radar signal in the database of the AVP vehicle mounted radar and generate the Vehicle Type Marker Ar2 at the Ir2 confidence level. When both types of detection judgments are true, Ari and Ar2 utilize gate decision-making logic to produce a comprehensive decision-making result A for the AVP vehicle. The confidence level I of the comprehensive decision-making result is based on the weighted perceptual confidence levels Iri and Ir2. = m * / R1 + n * IR2 m + n Where, m, n are the weighting coefficients of two types of perceptions respectively.

Claims

1. A rapid vehicle detection method for autonomous valet parking using dual sensing with lidar and near-infrared sensors, comprising the following steps:a) Acquiring a target vehicle perception near-infrared image data set d and a radar characteristic parameter data set da within the range of a main channel of a parking lot; establishing a radar wave signal feature library E from public data sets;b) Performing three steps of image denoising, image enhancement, and image segmentation on said target vehicle perception near-infrared image data set di to obtain a processed near-infrared image data set Di;c) Performing two steps of signal sorting preprocessing and signal sorting main processing on said radar characteristic parameter data set d2 to obtain a processed radar characteristic parameter data set D2;d) Using a target detection algorithm based on deep learning on said processed nearinfrared image data set Di to identify targets in a region of interest, categorizing a identified targets T into vehicle Tcar and driver Tdnver, and obtaining an identification confidence Icar for each vehicle object and an identification confidence 1*™« for each driver object;e) Using a database pattern recognition algorithm on said processed radar characteristic parameter data set D2 to perform feature matching between D2 and said radar wave signal feature library E, and obtaining the maximum matching confidence Ladarmax;f) Based on said identified targets T from said processed near-infrared image data set D and said processed radar characteristic parameter data set D2, along with their respective identification confidences, determine a near-infrared-based vehicle type Ari with confidence Iri, and a vehicle-radar-based vehicle type Ar2 with confidence Ir2;g) Based on determined type markers and confidences above steps , determine a final vehicle type A and confidence I.

2. A rapid vehicle detection method according to Claim 1, characterized in that in step a), a near-infrared imager and a radar receiver are deployed at selected positions; said target vehicle perception near-infrared image data set di covers a space enclosed by a 60° angle on both sides of the main channel of said parking lot, and said radar characteristic parameter data set d: covers the facing space of the columns on both sides of the main channel of said parking lot.

3. A rapid vehicle detection method according to Claim 1, characterized in that in step b),image denoising refers to applying an algorithm to remove random granular noise caused by sensitive elements and transmission channels.

4. A rapid vehicle detection method according to Claim 1, characterized in that in step b), image enhancement refers to applying an algorithm to enhance the contrast of near-infrared images and widen the range of grayscale changes.

5. A rapid vehicle detection method according to Claim 1, characterized in that in step b), image segmentation refers to applying an algorithm to segment the region of interest in the image: a foreground area, removing the background area composed of other parked vehicles and infrastructure within said parking lot.

6. A rapid vehicle detection method according to Claim 1, characterized in that in step c), signal sorting preprocessing refers to applying an algorithm to separate known radar wave trains and unknown radar pulse trains from said radar characteristic parameter data set d2 to obtain a pre-sorted resolution unit Ri.

7. A rapid vehicle detection method according to Claim 6, characterized in that in step c), signal sorting main processing refers to applying an algorithm to further classify a presorted resolution unit Ri to separate individual radar pulse trains R (i = 1.2,3...11) to form said processed radar characteristic parameter data set D2.

8. A rapid vehicle detection method according to Claim 1, characterized in that in step d), a multi-target recognition model deep learning framework is used, with a public data set of near-infrared images for pre-training; and a pre-trained deep learning model is used to receive said processed near-infrared image data set Di for vehicle and driver target recognition, outputting the target bounding box, target type T(Car driver), and confidence I(car / driver).

9. A rapid vehicle detection method according to Claim 7, characterized in that in step a), said radar wave signal feature library E is established based on said public data sets, including radar models and feature vectors of AVP vehicles from different brands; in step e), a database pattern recognition algorithm based on feature matching is used to match the radar pulse trains Ri (i = l,2,3...n) with radar pulse trains Ei (i = l,2,3...n) in said radar wave signal feature library E, and the correlation coefficient between the two signals is calculated and recorded as the confidence Iradar, outputting the maximum confidence Iradar max-10. A rapid vehicle detection method according to Claim 9, characterized in that in step f), confidence threshold judgment is used to determine said near-infrared identification result; only when a vehicle is identified and no driver is identified, it is detennined that an AVP vehicle is identified, outputting said near-infrared-based vehicle type Ari; the confidence I r 1 is derived by averaging the confidence levels of the two types of targets:j _ ^car d” (1 ^driver) Iri ~When a signal is detected in R that matches said radar wave signal feature library E, then Iradar max is greater than or equal to the judgment threshold, outputting said vehicleradar-based vehicle type Ar2 and confidence Ir2.

11. A rapid vehicle detection method according to Claim 10, characterized in that in step g). said Ari and Ar2 are subjected to an AND logic decision, and only when both types of detection judgments are true, said vehicle type A is output as an AVP vehicle, with m and n being the weight coefficients of the two types of perception, and a confidence I of the comprehensive decision result is weighted based on the perception confidence Iri and( = m*IR1 + n* IR2m + n

Images