A millimeter wave radar-based parking identification system

Abstract

The application discloses a kind of parking identification systems based on millimeter wave radar, it is related to parking identification technical field, it is difficult to combine different sensor features to carry out multi-angle parking object probability analysis, it is also difficult to combine image detection precision to carry out the technical problem of parking interference analysis;Through millimeter wave radar, it has strong adaptability to bad weather and illumination conditions, can work stably in complex environment, combined with ultrasonic sensor is not affected by illumination, and the sensitivity to environmental changes in short distance is lower, further enhance the environmental adaptability of system. By analyzing the energy value of each distance interval, the system can identify the object type entering the parking space, combined with ultrasonic sensor can quickly measure target distance, can more accurately judge object category. At the same time, by video image combined with thermal infrared, the misjudgment caused by illumination change is excluded, and the accuracy of interference analysis is improved.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of parking identification, and particularly relates to a parking identification system based on a millimeter wave radar. BACKGROUND

[0002] In the field of parking management, traditional methods often rely on manual patrol, ground inductance coils or ultrasonic sensors and other means, which have a large number of problems, for example, manual patrol and marking empty spaces are time-consuming and laborious, and are prone to errors. The ground inductance coil needs to be installed by cutting the ground, and has high maintenance cost and is easily disturbed by the environment; the ultrasonic sensor has a short detection distance, a narrow field of view and weak detection capability for low-height objects. The performance of the traditional sensor decreases in bad weather (such as rain, snow and fog) or low light conditions, which affects the accuracy and reliability of parking management. The millimeter wave radar is a sensor that uses millimeter wave frequency band (30 GHz-300 GHz) electromagnetic waves for target detection and ranging. It can accurately measure the distance, speed and angle information of the target object by emitting millimeter wave signals and receiving the echo signals reflected by the target, and using the time of flight (ToF) and Doppler effect technologies. In order to solve the above problems, the parking identification system based on the millimeter wave radar emerges as the times require.

[0003] However, most of the existing millimeter wave radar parking identification systems use a single millimeter wave radar sensor for parking identification, which is difficult to fully combine different sensor characteristics for multi-angle parking object probability analysis, and is also difficult to combine image detection for accurate parking interference analysis. Using multiple sensors for real-time operation for parking identification will greatly increase the energy consumption of the equipment. SUMMARY

[0004] The present application aims to at least solve one of the technical problems existing in the prior art; for this purpose, the present application provides a parking identification system based on a millimeter wave radar, which is used to solve the technical problem that it is difficult to fully combine different sensor characteristics for multi-angle parking object probability analysis, and it is also difficult to combine image detection for accurate parking interference analysis.

[0005] To solve the above problems, the first aspect of the present application provides a parking identification system based on a millimeter wave radar, comprising: a wireless video pile and a management platform in the cloud, the wireless video pile comprising: a radar module, a video acquisition module, a visual processing module, a parking space detection module and a wireless communication module;

[0006] The radar module is used for emitting and receiving millimeter waves, converting the echo signals into energy values in each distance interval, outputting the energy values to the parking space detection module, and measuring the target distance through an ultrasonic auxiliary detection unit and outputting the echo time;

[0007] The parking space detection module is configured to determine the type of object entering the parking space according to the energy value of each distance interval and the ultrasonic auxiliary detection data, and to give a pre-warning for the interference object during parking.

[0008] The video acquisition module is configured to acquire the video image and the thermal infrared image of the parking space and send them to the visual processing module.

[0009] The visual processing module is configured to perform license plate recognition and parking interference analysis according to the image data of the parking space, and send the analysis result to the management platform through the wireless communication module and interact with the management platform.

[0010] The management platform is configured to send a remote reminder signal to the vehicle according to the parking interference analysis data.

[0011] Optionally, in one example of the above aspect, the wireless video pile further comprises a light supplementing lamp module, a solar energy conversion module and a battery module.

[0012] The light supplementing lamp module is configured to provide ambient light for the video acquisition module in a low brightness environment.

[0013] The solar energy conversion module is configured to convert light energy into electrical energy to charge the battery module.

[0014] The battery module is configured to provide working power for the wireless video pile.

[0015] Optionally, in one example of the above aspect, the wireless video pile controls the operation of each module, including:

[0016] When the parking space detection module detects that the parking space is empty, the radar module and the parking space detection module are in a running state, the parking space detection module performs echo signal analysis, and the remaining functional modules and the ultrasonic auxiliary detection unit of the radar module are in a dormant state.

[0017] When the parking space detection module detects that a motor vehicle enters, the video acquisition module and the ultrasonic auxiliary detection unit are dispatched to run by the parking space detection module, the image of the entering vehicle is acquired by the video acquisition module, the license plate recognition and parking interference analysis are performed by the visual processing module, the analysis result is uploaded to the management platform by the wireless communication module, and the modules dispatched to run by the parking space detection module re-enter the standby state.

[0018] When a non-motor vehicle is detected, the parking space detection module reports the parking space occupation state information through the wireless communication module, and the device re-enters the standby state after uploading.

[0019] Optionally, in one example of the above aspect, the parking space detection module performs echo signal analysis, including the following steps.

[0020] The received echo signal is subjected to analog-digital conversion and one-dimensional Fourier transform to obtain a complex number array when the parking space detection module detects that the parking space is empty at regular intervals, the complex number array is converted into an energy value, the energy distribution data of the radar reflection signal in this state is recorded, and an energy value reference value array is formed;

[0021] The energy array is formed by the energy value of every 0.1 m distance in the range of the radar module echo signal;

[0022] The energy array and the energy value reference value array are compared, and if the energy difference between the energy array and the energy value reference value array is greater than or equal to 1, it is determined that there is a valid target signal in the parking space, and the collected energy array is classified as a target energy array;

[0023] Otherwise, the energy difference is less than 1, and it is considered to belong to environmental noise interference, and the energy value at this position is zeroed.

[0024] Optionally, in one example of the above aspect, the type of object entering the parking space is determined according to the energy value of each distance interval and the ultrasonic auxiliary detection data, and the interference object during parking is warned, including the following steps:

[0025] The target energy array corresponding to the valid target signal of the parking space detection is obtained, and when the target energy array is 0, it is determined that the parking space is empty;

[0026] Otherwise, the target energy array is classified by a neural network-based binary classification system, the type of object entering the parking space is determined according to the target probability calculated from the weight array and the bias array and the ultrasonic auxiliary detection data, and the interference object during parking is warned.

[0027] Optionally, in one example of the above aspect, the target energy array is classified by a neural network-based binary classification system, including the following steps:

[0028] The target energy array Eti is subjected to normalization processing:

[0029]

[0030] Wherein, Eni is the normalized target energy array, μE is the average value, and σE is the standard deviation;

[0031] The output target probability P is calculated from the normalized target energy array Eni, the weight array W, the bias array b, and the comprehensive target energy z, and the mathematical model is:

[0032] z=W*En+b

[0033]

[0034] The weight array W is initialized as a zero-mean Gaussian distribution, and the bias array b is initialized as 0.

[0035] The prediction bias is measured by cross-entropy loss, and the weight array W and the bias array b are optimized. The target probability P is calculated according to the optimized weight array and bias array.

[0036] Optionally, in one example of the above aspect, the prediction bias is measured by cross-entropy loss, and the weight array W and the bias array b are optimized, including the following steps:

[0037] The prediction bias is measured by cross-entropy loss, and the loss function is set to optimize the weight array W and the bias array b:

[0038]

[0039] Where Pi is the i-th output probability, N is the number of training samples, i∈(1,2, …, N), and yi is the true label.

[0040] The optimized weight array W and the bias array b are updated by the Adam optimizer:

[0041]

[0042] Where η is a fixed step size.

[0043] Optionally, in one example of the above aspect, the target probability calculated according to the weight array and the bias array, and the ultrasonic auxiliary detection data are used to judge the type of the object entering the parking space and to give a warning for the interference object during parking, including the following steps:

[0044] The detection data of the ultrasonic sensor with edge array distribution at the parking position are obtained, including distance data, reflected signal intensity and moving speed;

[0045] The distance from the surface of the object detected by each sensor to the edge of the parking position is extracted, and the edge distribution point array of the object in the parking position contour image is obtained by combining the contour data of the parking position edge;

[0046] The edge distribution point array of the object in the parking position contour image is processed by PointNet++, and the geometric features are extracted;

[0047] The reflected signal intensity and the moving speed are encoded into feature vectors by a fully connected layer, and are spliced with the geometric features;

[0048] A neural network model is constructed, the output layer is a Softmax activation function combined with a cross-entropy loss function; a validation set is constructed by known historical data of object classification, and the model training is stopped when the validation set loss does not decrease for 5 consecutive rounds, and the model training is completed;

[0049] The spliced feature vector is input into the trained model to output a category probability vector Pf=[Pmotor vehicle, Pnon-motor vehicle, Pobstacle], and the final probability is corrected in combination with the ultrasonic physical constraint condition that the reflection signal intensity of the vehicle is greater than 50 dB;

[0050] The target probability calculated by the weight array and the bias array is obtained, the category probability vector obtained by the ultrasonic auxiliary detection data is weighted and averaged with the target probability obtained by the ultrasonic echo analysis, and the final target classification probability is obtained;

[0051] After detecting that a motor vehicle exists in the parking space, in the case that a non-motor vehicle or an obstacle is detected in the parking space again within a preset time period, a parking interference warning is performed.

[0052] Optionally, in one example of the above aspect, the license plate recognition and parking interference analysis according to the parking space image data comprises the following steps:

[0053] The parking space video image and the thermal infrared image are obtained, the license plate recognition is performed according to the parking space video image, and after the license plate recognition is successful, the number and the moving speed of the interference objects in the parking space video image and the thermal infrared image are identified by the pre-trained deep learning model within a preset time period;

[0054] The parking interference coefficient g=(c-2) / 10+((La / vmin)-t0) / t0 is analyzed according to the number and the moving speed of the interference objects, wherein c is the number of the interference objects, La is the diagonal length of the parking space, and vmin is the minimum value of the moving speed of the interference objects.

[0055] Optionally, in one example of the above aspect, the remote reminder signal is sent to the vehicle according to the parking interference analysis data, comprising the following steps:

[0056] The historical data of the parking interference analysis when the parking accident occurs is obtained, the mean value and the minimum value of the parking interference coefficient are calculated, and the mean value and the minimum value are weighted and averaged to obtain the parking interference coefficient warning threshold;

[0057] According to the real-time uploaded parking interference coefficient, if the parking interference coefficient is greater than the parking interference coefficient warning threshold, a remote reminder signal is sent to the vehicle.

[0058] Compared with the prior art, the present application has the following advantages:

[0059] The application has strong adaptability to bad weather and light conditions through the millimeter wave radar, can stably work in a complex environment, and further enhances the environmental adaptability of the system in combination with the ultrasonic sensor which is not affected by light and has low sensitivity to environmental changes in a short distance. The system can identify the object type (such as a vehicle, a pedestrian, an obstacle, etc.) entering the parking space by analyzing the energy values in each distance interval, and can more accurately judge the object category in combination with the ultrasonic sensor which can quickly measure the target distance, and give a warning for potential interference objects. The combination of the millimeter wave radar and the ultrasonic auxiliary detection forms a redundant detection mechanism, when one sensor fails, the other sensor can still maintain the basic function, ensuring the reliability of the system.

[0060] The application excludes misjudgment caused by light changes through video images combined with thermal infrared, improves the accuracy of interference analysis, identifies object contours through temperature differences, is not affected by light, can detect night vehicles, pedestrians or abnormal heat sources, and makes up for the night blind area of visible light images. In combination with the object motion speed in the image and the temperature anomaly in the thermal infrared, such as illegally parked pedestrians or equipment, parking interference analysis can be more accurately performed. There is no need for manual registration, the vehicle license plate is automatically identified when the vehicle enters, and the database is matched to realize fast lifting and release, and reduce the queuing time. BRIEF DESCRIPTION OF DRAWINGS

[0061] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiment or prior art description, obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can be obtained without creative labor on the basis of these drawings.

[0062] Figure 1 It is a schematic diagram of the system framework of the present application;

[0063] Figure 2 It is a schematic diagram of standby power consumption of the equipment of the present application;

[0064] Figure 3 It is a schematic diagram of power consumption when scheduling vision of the present application;

[0065] Figure 4 It is a schematic diagram of power consumption when reporting non-motor vehicle occupation of the present application. DETAILED DESCRIPTION

[0066] The technical solutions of the present application will be described in detail below in combination with the embodiments, obviously, the described embodiments are only some of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0067] Referring to Figures 1-4 The first aspect of the present application provides a millimeter wave radar-based parking identification system, comprising a wireless video pile and a cloud-based management platform. The wireless video pile comprises a radar module, a video acquisition module, a visual processing module, a parking space detection module, and a wireless communication module.

[0068] The radar module is used to transmit and receive millimeter waves, convert the echo signal into energy values in each distance interval, and output the energy values to the parking space detection module. The ultrasonic auxiliary detection unit measures the target distance and outputs the echo time.

[0069] The parking space detection module is used to determine the type of object entering the parking space based on the energy values in each distance interval and the ultrasonic auxiliary detection data, and to provide early warning for interference during parking.

[0070] The video acquisition module is used to acquire video images and thermal infrared images of the parking space and send them to the visual processing module.

[0071] The visual processing module is used to perform license plate recognition and parking interference analysis based on the parking space image data, and send the analysis results to the management platform through the wireless communication module and interact with the management platform.

[0072] The management platform is used to send a remote reminder signal to the vehicle based on the parking interference analysis data.

[0073] In this embodiment, the millimeter wave radar can accurately measure the distance, speed, and angle information of the target object by transmitting and receiving millimeter wave signals. Its high resolution characteristics enable the system to accurately distinguish the energy values in different distance intervals, thereby accurately determining the parking space status. The ultrasonic sensor calculates the target distance by measuring the echo time, and has high precision and fast response characteristics in close-range detection. The combination of the two can compensate for the potential blind spot of the millimeter wave radar in close-range detection, further improving the detection accuracy.

[0074] The millimeter wave radar has strong adaptability to adverse weather conditions (such as rain, snow, and fog) and lighting conditions (such as strong light and darkness), and can work stably in complex environments. Combined with the ultrasonic sensor, which is not affected by light and has low sensitivity to environmental changes (such as temperature and humidity) in short distances, the system's environmental adaptability is further enhanced.

[0075] The millimeter wave radar can effectively avoid interference with other wireless devices through frequency band selection and signal processing technology, ensuring the accuracy of the detection results. The ultrasonic sensor works in short distances, with a short signal propagation path and low possibility of external interference, further improving the system's anti-interference ability.

[0076] By analyzing the energy value of each distance interval, the system can identify the type of object entering the parking space (such as vehicles, pedestrians, obstacles, etc.), and combined with the ultrasonic sensor which can quickly measure the target distance, it can more accurately determine the object category and provide early warning for potential interference.

[0077] The combination of millimeter wave radar and ultrasonic auxiliary detection forms a redundant detection mechanism. When one sensor fails, the other sensor can still maintain basic functions to ensure system reliability.

[0078] In this embodiment, the video acquisition module acquires parking space video images and thermal infrared images and sends them to the vision processing module; license plate recognition and parking interference analysis are performed based on parking space image data, and the analysis results are sent to the management platform through the wireless communication module and interact with the management platform; remote reminder signals are sent to vehicles based on parking interference analysis data.

[0079] Parking space video images provide high-resolution details such as license plates, vehicle colors, and vehicle models, suitable for license plate recognition and vehicle feature analysis in daylight and well-lit environments. By excluding false positives caused by changes in light (such as tree shadows), the accuracy of interference analysis is improved, and by recognizing object outlines based on temperature differences, it is not affected by light and can detect vehicles, pedestrians, or abnormal heat sources at night, filling the night blind spot of visible light images.

[0080] For example: At the entrance of a parking lot, license plates are recognized by visible light images during the day, and vehicle entry is detected by thermal infrared images at night, ensuring uninterrupted monitoring throughout the day.

[0081] Combining the speed of object motion in the image and temperature anomalies in the thermal infrared, such as illegally parked pedestrians or equipment, can more accurately analyze parking interference.

[0082] No manual registration is required. When a vehicle enters, the license plate is automatically recognized and matched with the database, allowing for quick lift and release, reducing queuing time. License plate numbers, entry times, parking locations, and other information are automatically recorded for subsequent queries and billing. It is also convenient for the system to send a text message or APP push to the vehicle owner through the wireless communication module when a parking interference anomaly is detected, reminding them to handle it in a timely manner.

[0083] In one embodiment of the present application, the wireless video stake further comprises a light supplement module, a solar energy conversion module, and a battery module.

[0084] Light supplement module: used to provide ambient light for the video acquisition module in low brightness environment;

[0085] Solar energy conversion module: used to convert light energy into electrical energy to charge the battery module;

[0086] Battery module: used to provide working power for the wireless video stake.

[0087] In one embodiment of the present application, the wireless video pile controls the operation of each module, including:

[0088] When the parking space detection module detects that the parking space is empty, the radar module and the parking space detection module are in operation, the parking space detection module analyzes the echo signal, and the remaining function modules and the ultrasonic auxiliary detection unit of the radar module are in sleep state;

[0089] When the parking space detection module detects that the motor vehicle enters, the video acquisition module and the ultrasonic auxiliary detection unit are scheduled to operate by the parking space detection module, the image of the entering vehicle is acquired by the video acquisition module, and the license plate recognition and parking interference analysis are carried out by the visual processing module, the analysis result is uploaded to the management platform by the wireless communication module, and the module scheduled to operate by the parking space detection module reenters standby state;

[0090] When non-motor vehicle occupation is detected, the parking space occupation state information is reported by the parking space detection module through the wireless communication module, and the device reenters standby state after uploading.

[0091] In this embodiment, the power consumption of the device in standby state is very low through the low-power hardware design, and the device only calls video recognition when the motor vehicle enters through radar target classification, reduces false triggering, so that the daily average power consumption of the device is far less than the daily average solar energy conversion power. At the same time, the solar energy conversion module continuously converts light energy into electric energy to charge the battery module, so that the device can work continuously after installation, without manual maintenance and regular battery replacement.

[0092] In order to better utilize solar energy, the device arranges the solar energy conversion module at the top, and the effective light receiving area is about 4000mm 2 According to the data of China Meteorological Administration, the average total radiation of the experimental area is about 1496.1kwh / m 2 , which is 1.58% less than the average value in the past 30 years. According to the average radiation level of the experimental area, the solar energy conversion module with an effective light receiving area of 4000mm 2 can receive 16.4wh of energy per day, and the photoelectric conversion efficiency of the solar energy conversion module is about 20%, that is, the device can provide 3.28wh of energy per day.

[0093] As shown in Figure 2 , the average power consumption of the device in standby state is only 3.9mW; as shown in Figure 3 , the average power consumption of the parking space detection module scheduling visual license plate recognition and uploading is about 483mW; as shown in Figure 4As shown, the average power consumption of the parking space detection module scheduling the wireless communication module to report the parking space occupancy state information is about 16 mW. According to the statistical data, the daily parking space turnover of the parking space in the peak section is about 10 times, and the daily average of the non-motor vehicle occupancy is about 50 times. The daily average energy consumption of the device is about 2.63 wh. Therefore, after the device is installed, it does not need to be maintained and the battery does not need to be replaced regularly.

[0094] In one embodiment of the present application, the parking space detection module performs echo signal analysis, including the following steps:

[0095] Periodically, when the parking space detection module detects that the parking space is empty, the received echo signal is subjected to analog-to-digital conversion and one-dimensional Fourier transform to obtain a complex array, the complex array is converted into an energy value, the energy distribution data of the radar reflection signal in this state is recorded, and an energy value reference value array is formed;

[0096] Real-time acquisition of the radar module echo signal, the energy value of every 0.1 m distance in the range of the energy array is formed;

[0097] The energy array collected in real time is compared with the energy value reference value array, if the energy difference between the energy array and the energy value reference value array is greater than or equal to 1, it is determined that there is an effective target signal in the parking space, and the collected energy array is classified as a target energy array;

[0098] Otherwise, the energy difference is less than 1, and it is considered to be environmental noise interference, and the energy value at this position is zeroed.

[0099] In this embodiment, the received echo signal is subjected to analog-to-digital conversion and one-dimensional Fourier transform to obtain a complex array X, which is converted into an energy value E. The radar module forms an energy array Er={e0, e1, …, e59} of the energy value of every 0.1 m distance in the range, and outputs it to the parking space detection module. The parking space detection module records the energy array of the parking space background when the parking space is empty, i.e. the energy value reference value array, and compares the energy array Er collected by the radar module each time with the energy value reference value array Eb to generate a target energy array Et, wherein Eri-Ebi≥1, Eti=Eri, Eri-Ebi<1, Eti=0.

[0100] In one embodiment of the present application, according to the energy value of each distance interval and the ultrasonic auxiliary detection data, the type of the object entering the parking space is judged, and the interfering object during parking is warned, including the following steps:

[0101] Acquire the target energy array corresponding to the effective target signal of the parking space detection, when the target energy array is 0, it is judged that the parking space is empty;

[0102] Otherwise, the target energy array is classified by the neural network-based binary classification system, the target probability calculated according to the weight array and the bias array, and the ultrasonic auxiliary detection data are used to determine the object type entering the parking space and to give a warning for the interference object during parking.

[0103] In one embodiment of the present application, the target energy array is classified by the neural network-based binary classification system, including the following steps:

[0104] The target energy array Eti is normalized:

[0105]

[0106] Wherein, Eni is the normalized target energy array, μE is the average value, and σE is the standard deviation;

[0107] The output target probability P is calculated from the normalized target energy array Eni, the weight array W, the bias array b, and the comprehensive target energy z, and the mathematical model is:

[0108] z = W * En + b

[0109]

[0110] The weight array W is initialized as a zero-mean Gaussian distribution, and the initial value of the bias array b is 0;

[0111] The prediction deviation is measured by cross-entropy loss, and the weight array W and the bias array b are optimized, and the target probability P is calculated according to the optimized weight array and bias array.

[0112] In one embodiment of the present application, the prediction deviation is measured by cross-entropy loss, and the weight array W and the bias array b are optimized, including the following steps:

[0113] The prediction deviation is measured by cross-entropy loss, and the loss function is set to optimize the weight array W and the bias array b:

[0114]

[0115]

[0116] Wherein, Pi is the i-th output probability, N is the number of training samples, i ∈ (1, 2, …, N), and yi is the true label;

[0117] The optimized weight array W and the bias array b are updated by the Adam optimizer:

[0118]

[0119] wherein, η is a fixed step size.

[0120] In one embodiment of the present application, the target probability calculated according to the weight array and the bias array, and the ultrasonic auxiliary detection data are used to determine the type of object entering the parking space and to give a pre-warning of the interference object during parking, including the following steps:

[0121] The detection data of the ultrasonic sensor distributed along the edge of the parking position are obtained, including distance data, reflected signal intensity and moving speed;

[0122] The distance from the surface of the object detected by each sensor to the edge of the parking position is extracted, and the edge distribution point array of the object in the parking position contour image is obtained in combination with the contour data of the edge of the parking position;

[0123] The edge distribution point array of the object in the parking position contour image is processed by PointNet++ to extract geometric features;

[0124] The reflected signal intensity and the moving speed are encoded into feature vectors through a fully connected layer, and are spliced with the geometric features;

[0125] A neural network model is constructed, the output layer is a Softmax activation function combined with a cross-entropy loss function; a validation set is constructed by using historical data of known object classification, and the model training is stopped when the loss of the validation set does not decrease for 5 consecutive rounds, and the model training is completed;

[0126] The spliced feature vectors are input into the trained model to output a probability vector of each category: Pf = [Pmotor vehicle, Pnon-motor vehicle, Pobstacle], and the final probability is corrected in combination with the ultrasonic physical constraint condition: the reflected signal intensity of the vehicle > 50 dB;

[0127] The target probability calculated by using the weight array and the bias array is obtained, the probability vector of each category obtained by using the ultrasonic auxiliary detection data is weighted and averaged with the target probability obtained by using the ultrasonic echo analysis, and the final target classification probability is obtained;

[0128] After detecting that there is a motor vehicle in the parking space, if a non-motor vehicle or an obstacle is detected in the parking space again within a preset time period, a pre-warning of the parking interference object is given.

[0129] In one embodiment of the present application, license plate recognition and parking interference analysis are performed according to the parking space image data, including the following steps:

[0130] The parking space video image and the thermal infrared image are obtained, license plate recognition is performed according to the parking space video image, and after the license plate recognition is successful, the number and moving speed of the interference objects in the parking space video image and the thermal infrared image are identified through a pre-trained deep learning model within a preset time period;

[0131] According to the number and moving speed of the interference objects, analyze the parking interference coefficient g=(c-2) / 10+((La / vmin)-t0) / t0, wherein c is the number of the interference objects, La is the diagonal length of the parking space, and vmin is the minimum value of the moving speed of the interference objects.

[0132] In one embodiment of the present application, according to the parking interference analysis data, a remote reminder signal is sent to the vehicle, including the following steps:

[0133] Obtain the historical data of the parking interference analysis when the parking accident occurs, calculate the mean and minimum value of the parking interference coefficient, and obtain the parking interference coefficient early warning threshold by weighted average of the mean and minimum value;

[0134] According to the real-time uploaded parking interference coefficient, if the parking interference coefficient is greater than the parking interference coefficient early warning threshold, a remote reminder signal is sent to the vehicle.

[0135] The above embodiments are only used to illustrate the technical method of the present application and are not limited. Although the present application is described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that the technical method of the present application can be modified or replaced equivalently without departing from the spirit and scope of the technical method of the present application.

Claims

1. A parking recognition system based on millimeter-wave radar, characterized in that, include: The wireless video parking system and cloud-based management platform include: a radar module, a video acquisition module, a vision processing module, a parking space detection module, and a wireless communication module. Radar module: Used to transmit and receive millimeter waves, convert the echo signal into energy values ​​for each distance range and output them to the parking space detection module, and measure the target distance through the ultrasonic-assisted detection unit and output the echo time; Parking space detection module: Used to determine the type of object entering the parking space based on the energy value of each distance range and ultrasonic-assisted detection data, and to provide early warning of interference during the parking process; Video acquisition module: used to acquire video images and thermal infrared images of the parking space and send them to the vision processing module; The vision processing module is used to perform license plate recognition and parking interference analysis based on parking space image data, and to send the analysis results to the management platform through the wireless communication module, and to interact with the management platform. Management platform: Used to send remote reminder signals to vehicles based on parking disturbance analysis data; The process involves determining the type of object entering the parking space based on the energy values ​​within each distance range and ultrasonic-assisted detection data, and issuing warnings for interfering objects during parking. This includes the following steps: Obtain the target energy array corresponding to the valid target signal of the parking space detection. When the target energy array is 0, the parking space is determined to be empty. Otherwise, the target energy array is classified by a neural network-based binary classification system. Based on the target probability calculated by the weight array and bias array, as well as the ultrasonic-assisted detection data, the type of object entering the parking space is determined, and warnings are given for interference during the parking process. License plate recognition and parking interference analysis are performed based on parking space image data, including the following steps: Acquire parking space video images and thermal infrared images, perform license plate recognition based on parking space video images, and within a preset time period after successful license plate recognition, use a pre-trained deep learning model to identify the number and movement speed of interfering objects in the parking space video images and thermal infrared images. Based on the number and speed of the interfering objects, the parking interference coefficient is calculated as g = (c-2) / 10 + ((La / vmin)-t0) / t0, where c is the number of interfering objects, La is the diagonal length of the parking space, and vmin is the minimum speed of the interfering object.

2. The parking recognition system based on millimeter-wave radar according to claim 1, characterized in that, The wireless video stake also includes: a supplementary lighting module, a solar power conversion module, and a battery module; Fill light module: Used to provide ambient light for the video capture module in low-light environments; Solar energy conversion module: Used to convert solar energy into electrical energy to charge the battery module; Battery module: Used to provide power for wireless video stakes.

3. The parking recognition system based on millimeter-wave radar according to claim 1, characterized in that, The wireless video stake controls the operation of various modules, including: When the parking space detection module detects that the parking space is empty, the radar module and the parking space detection module are in operation. The parking space detection module performs echo signal analysis, while the other functional modules and the ultrasonic auxiliary detection unit of the radar module are in sleep mode. When the parking space detection module detects a motor vehicle entering, it schedules the video acquisition module and the ultrasonic-assisted detection unit to operate. The video acquisition module captures images of the entering vehicle, and the visual processing module performs license plate recognition and parking interference analysis. The analysis results are uploaded to the management platform via the wireless communication module, and the modules scheduled by the parking space detection module return to standby mode. When a non-motorized vehicle is detected occupying the space, the parking space detection module reports the parking space occupancy status information through the scheduling wireless communication module. After the information is uploaded, the device returns to standby mode.

4. A parking recognition system based on millimeter-wave radar according to claim 3, characterized in that, The parking space detection module performs echo signal analysis, including the following steps; When the parking space detection module periodically detects that a parking space is empty, the received echo signal is converted from analog to digital and then subjected to a one-dimensional Fourier transform to obtain a complex array. The complex array is then converted into energy values, and the energy distribution data of the radar reflection signal under this state is recorded to form an energy value reference array. The radar module echo signal is acquired in real time, and the energy value at every 0.1m distance within the range is formed into an energy array; The real-time collected energy array is compared with the energy value reference array. If the energy difference between the energy array and the energy value reference array is greater than or equal to 1, it is determined that there is a valid target signal in the parking space, and the collected energy array is classified as a target energy array. Otherwise, if the energy difference is less than 1, it is considered to be environmental noise interference, and the energy value at that location is set to zero.

5. A parking recognition system based on millimeter-wave radar according to claim 1, characterized in that, The target energy array is classified using a neural network-based binary classification system, including the following steps: Normalize the target energy array Eti: Where Eni is the normalized target energy array. This is the average value. Standard deviation; The output target probability P is calculated using the normalized target energy array Eni, weight array W, bias array b, and combined target energy z. The mathematical model is as follows: The weight array W is initialized to a zero-mean Gaussian distribution, and the bias array b is initialized to 0. The prediction bias is measured by cross-entropy loss. The weight array W and the bias array b are optimized. The target probability P is calculated based on the optimized weight array and bias array.

6. A parking recognition system based on millimeter-wave radar according to claim 5, characterized in that, The prediction bias is measured using cross-entropy loss, and the weight array W and bias array b are optimized, including the following steps: The prediction bias is measured using cross-entropy loss, and the loss function is set to optimize the weight array W and the bias array b: Where Pi is the output probability of the i-th time, N is the number of training samples, i∈(1,2,…,N), and yi is the true label; The Adam optimizer is used to update the parameters of the optimized weight array W and bias array b: in, For fixed step size.

7. A parking recognition system based on millimeter-wave radar according to claim 1, characterized in that, Based on the target probability calculated using the weight array and bias array, and the ultrasonic-assisted detection data, the type of object entering the parking space is determined, and warnings are issued for interfering objects during the parking process, including the following steps: Acquire detection data from ultrasonic sensors distributed along the edge of the parking location, including distance data, reflected signal intensity, and moving speed; Extract the distance from the object surface detected by each sensor to the edge of the parking position, and combine it with the contour data of the parking position edge to obtain the edge distribution point array of the object in the contour image of the parking position. The edge distribution of the object's contour image at the parking location is processed using PointNet++ to extract geometric features; The reflected signal strength and moving speed are encoded into feature vectors through a fully connected layer and then concatenated with geometric features; Construct a neural network model with the output layer using the Softmax activation function combined with the cross-entropy loss function; build a validation set using historical data of known object classifications, and stop training when the validation set loss does not decrease for 5 consecutive rounds, thus completing the model training; The concatenated feature vectors are input into the trained model to output probability vectors for each category: Pf=[P motor vehicle, P non-motor vehicle, P obstacle]. The final probability is corrected by combining the ultrasonic physical constraint condition: the reflected signal intensity of the vehicle is >50dB. The target probability is obtained by calculating the weight array and bias array. The probability vectors of each category obtained from the ultrasound-assisted detection data are weighted and averaged with the target probability obtained from the ultrasound echo analysis to obtain the final target classification probability. If a non-motorized vehicle or obstacle is detected in a parking space again within a preset time period after a motor vehicle is detected in the parking space, a parking disturbance warning will be issued.

8. A parking recognition system based on millimeter-wave radar according to claim 1, characterized in that, Based on parking disturbance analysis data, a remote alert signal is sent to the vehicle, including the following steps: Historical data on parking interference analysis at the time of parking accidents are obtained, the mean and minimum values ​​of the parking interference coefficient are calculated, and the mean and minimum values ​​are weighted and averaged to obtain the warning threshold of the parking interference coefficient. Based on the real-time uploaded parking interference coefficient, if the parking interference coefficient exceeds the parking interference coefficient warning threshold, a remote reminder signal will be sent to the vehicle.

Images