Blind zone recognition system and road vehicle control system and method based on blind zone recognition

Abstract

The invention belongs to the field of road vehicle control systems, and relates to a blind area recognition system and a road vehicle control system and method based on blind area recognition, and the blind area recognition system comprises a laser emission module, a photon-level echo receiving module, an algorithm processing module, a target reconstruction module and a recognition and classification module. The algorithm processing module represents the echo signal as a linear combination of a series of sparse basis functions, reconstructs reflectivity information of the non-vision-field target surface, and extracts geometric information of the non-vision-field target surface; the target reconstruction module is used for performing joint optimization on the non-vision-field target surface information and the echo signal by using a regularization technology, and recovering a sparse coefficient by solving a joint regularization problem to obtain a reconstruction signal; and the identification and classification module is used for judging the type and probability of the non-vision target. According to the system, the hidden target is observed in real time through the non-vision field imaging technology, the physical characteristics of the non-vision field target are combined during signal processing, the recognition accuracy is high, and the safety risk caused by a blind area is reduced.

Description

Technical Field

[0001] The present invention belongs to the field of road vehicle control systems, and in particular relates to a blind spot recognition system, and a road vehicle control system and method based on blind spot recognition. Background Art

[0002] With the development of autonomous driving technology, environmental perception has become one of the key technologies to ensure driving safety. At present, autonomous vehicles mainly rely on sensors such as cameras, radars, and laser radar (LiDAR) to perceive the surrounding environment, detect and identify objects around the vehicle, and then control the vehicle based on the detection or identification results. These existing sensors can provide real-time information about the vehicle's surrounding environment, including the location, speed, and shape of objects. However, due to the physical limitations of the sensors themselves, such as viewing angle range, resolution, and detection distance, they cannot fully cover all areas around the vehicle in some cases, especially in complex scenes such as urban roads, intersections, and highway entrances and exits. Obstacles such as buildings, trees, vehicles, and pedestrians will hinder the line of sight and form blind spots, causing the sensor to be unable to detect the obscured area, unable to fully perceive the surrounding environment, and unable to know in advance the driving conditions of vehicles or pedestrians within the blind spot on the intersecting road, thereby increasing the risk of collision and accidents.

[0003] Chinese patent CN113348119A discloses a vehicle blind spot recognition method and an automatic driving assistance system. The system obtains surrounding environment information and obstacle speed through sensors, gives risk coefficients based on the blind spots and the scenes in which they are located, and classifies them into levels. Due to the physical limitations of the sensors and the influence of environmental factors, the method cannot identify objects in the blind spots, and can only assess the current risks. It cannot take corresponding measures in advance according to the types of objects and potential risks. Although it can determine dangerous driving blind spots, its processing capabilities in complex environments are insufficient.

[0004] Chinese patent CN114035200A discloses an active LiDAR blind spot detection system and method for autonomous driving. The system simulates the physiological structure of scallop eyes, receives scattered light from the environment after being illuminated, identifies and generates images of objects in the blind spot. Although this improves the ability to detect potential risks, it only relies on the collected scattered light for imaging without combining the surface information of the object, which will limit the reconstruction accuracy and affect judgment. It is not accurate enough when dealing with rapidly changing urban traffic environments.

[0005] Chinese patent CN117558163A discloses a method for dynamic blind spot risk identification and early warning in the platform area based on autonomous driving buses. It determines potential blind spots and judges collision risks by predicting the future trajectory of traffic individuals. The accuracy of the prediction is highly dependent on the algorithm and cannot detect sudden dangers. The calculation complexity and processing difficulty are relatively large. There are delays in data processing and decision response, and it cannot meet real-time requirements.

[0006] In summary, it can be seen that existing technologies cannot provide continuous and real-time blind spot information in complex urban environments, which may lead to serious consequences in emergency situations. Therefore, developing a technology that can effectively identify and manage blind spots is crucial to improving the safety of autonomous vehicles. Summary of the invention

[0007] In view of the above-mentioned technical problems and defects, the first object of the present invention is to provide a blind spot identification system, which is designed based on the joint regularization of echo signals and non-line-of-sight targets, observes hidden targets in real time through non-line-of-sight imaging technology, improves identification accuracy and efficiency, and reduces the safety risks brought by blind spots.

[0008] To achieve the above object, the present invention adopts the following technical solution: A blind spot identification system includes a laser emission module, a photon-level echo receiving module, an algorithm processing module, a target reconstruction module, a data compression module, a feature extraction module, and an identification and classification module; Wherein, the laser emission module is used to emit laser toward the intermediate surface; The photon-level echo receiving module is used to receive the photon-level echo diffusely reflected by the intermediate surface and the non-viewing target surface; The algorithm processing module is used to process the echo signal, express the echo signal as a linear combination of a series of sparse basis functions, perform fast Fourier transform on the discrete echo signal to obtain the frequency domain of the signal, analyze the spectrum obtained by the fast Fourier transform, and reconstruct the reflectivity information of the non-line-of-sight target surface; at the same time, the diffuse reflection incident point of the non-line-of-sight target surface is calculated according to the coordinates of the laser emission point of the laser emission module, and for each diffuse reflection incident point of the non-line-of-sight target surface, a point set within a radius r is queried, and a normal vector of each diffuse reflection incident point is calculated, and the geometric information of the non-line-of-sight target surface is extracted according to the normal vector of the diffuse reflection incident point; The target reconstruction module uses regularization technology to jointly optimize the surface information of the non-viewing area target and the echo signal, restores the sparse coefficients by solving the joint regularization problem, and converts the restored sparse coefficients back to the original domain to obtain a reconstructed signal; The data compression module is used to perform pulse compression on the reconstructed signal to obtain a pulse signal; The feature extraction module is used to extract key features from the pulse signal, including the peak value, duration, and frequency distribution of the signal; The recognition and classification module performs recognition and classification based on the extracted key features to determine the type and probability of the non-viewing area target.

[0009] As a preferred embodiment of the present invention, the expression for expressing the echo signal as a linear combination of a series of sparse basis functions is: ; In the formula, is the observed echo signal vector representation, is a sparse coefficient vector, is the sparse transformation matrix, The dictionary matrix representation of represents the number of atoms in the dictionary, and the noise term is ; The method of performing fast Fourier transform on discrete echo signals to obtain the signal frequency domain is: ; In the formula, It is a frequency domain signal, which means at the frequency point The spectrum value at represents the total number of times the signal is sampled, Indicates sampling points, Indicates frequency components, is an imaginary unit; By analyzing the spectrum obtained by fast Fourier transform, the position and amplitude of each frequency peak are extracted. The amplitude of the peak corresponds to the reflectivity information of the non-viewing target surface. .

[0010] As a preferred embodiment of the present invention, the diffuse reflection incident point of the non-viewing area target surface is obtained as follows: Assume the coordinates of the laser emission point of the laser emission module ( , , ), the incident angle of each reflection in the horizontal direction is , the vertical pitch angle is , Indicates the number of reflections; The coordinates of the incident point of each reflection are calculated based on the coordinates of the previous reflection point, the reflection angle, and the distance. The coordinates of the incident point of the secondary reflection are ( , , ), No. The flight distance of the first reflection is , then The coordinates of the incident point of the secondary reflection ( , , )for: ; ; ; Set up i The second reflection occurs on the non-viewing target surface. i The coordinate information of all incident points can be obtained by reflecting it once.

[0011] As a preferred embodiment of the present invention, the normal vector calculation method of each diffuse reflection incident point is: Non-line-of-sight target surface formation diffuse reflection incident points, select any one of the diffuse reflection incident points As the query point, the KD-Tree method is used to perform a radius search on the three-dimensional point set. For each diffuse reflection incident point and query point Calculate Euclidean distance : ; Then the point set with query point q as the center and r as the radius It is expressed as: ; in, is the radius; set , is an empirical constant with a value of 2; is the average distance between points, and the calculation formula is: ; In the formula, Indicates sampling Diffuse reflection incident points on non-viewing target surfaces; The least squares method is used to fit the surface of the point set with the query point q as the center and r as the radius. Calculate the tangent plane, the direction of the gradient is the diffuse reflection incident point of the non-viewing target surface The normal vector ;in, express right The partial derivative of express right The partial derivative of express right The partial derivative of Normal vectors are calculated for all diffuse reflection incident points that can be extracted from the non-viewing target surface to obtain a series of normal vector sets.

[0012] As a preferred embodiment of the present invention, the method of extracting the geometric information of the non-viewing area target surface according to the normal vector of the diffuse reflection incident point is: represent q The normal vector of the query point, represent q The normal vectors of other diffuse reflection incident points in the neighborhood of the query point are calculated q The residual sum of squares of the normal vector of the query point and the normal vectors of other diffuse incident points in the neighborhood: ; In the formula, Indicates the number of points in the neighborhood; The sampled The normal vectors of the incident points on the non-viewing target surface are processed in turn, and the sum of the squared residuals is added and then divided by , and get the average residual sum of squares: ,use Represents the non-viewing area target surface geometry information.

[0013] As a preferred embodiment of the present invention, before the target reconstruction module uses the regularization technology to jointly optimize the non-viewing area target surface information and the echo signal, the non-viewing area target surface geometric information and the reflectivity information are first jointly optimized: ; In the formula, About Smoothness constraints on surface normals, It's about reflectivity The sparsity constraint is is a regularization term that includes normal information and reflectivity, , is the weighting coefficient, which is controlled by and Make the solution smooth and sparse; Then, the regularization technology is used to jointly optimize the surface information of the non-viewing target and the echo signal. The expression is: ; In the formula, Contains the original observations The regularization term of is a joint regularization term that includes both the echo signal and the non-viewing target information. , is the weighting coefficient, and the weighting ratio can be controlled by changing its value; Afterwards, the alternating iterative method is used to solve the joint regularization problem to recover the sparse coefficients, which is expressed as: ; In the formula, is the observed echo signal vector representation, is a sparse coefficient vector, is the sparse transformation matrix, Dictionary matrix representation of ; It is a joint regularization term that includes both the echo signal and the surface information of the non-viewing target; Finally, the restored sparse coefficients Convert back to the original domain to get the reconstructed signal on the final two-dimensional view : ; In the formula, Indicates the number of atoms in the dictionary.

[0014] As a preferred embodiment of the present invention, the recognition and classification module uses a trained twin support vector machine, and when the twin support vector machine is trained, the following steps are included: Step 1. Collect sample data; The sample data includes pedestrians and vehicles. Multiple signal transmission and collection are performed for each sample, and the data is divided into training set and test set. Step 2. Data preprocessing; The algorithm processing module and the target reconstruction module process the collected data and finally obtain a series of two-dimensional reconstructed signals; Performing pulse compression on each reconstructed signal to obtain a pulse signal; Step 3. Feature extraction; Extract key features from each pulse signal, including the peak value, duration, and frequency distribution of the signal; use principal component analysis to reduce the dimension of the features; Step 4. Input the data processed in step 3 into the twin support vector machine model for training, and define the classification decision function as: ; In the formula, represents the input sample, is the number of samples, is the Lagrange multiplier, It is i The labels of the training samples, is an adaptive kernel function, is the bias term.

[0015] As a preferred embodiment of the present invention, the blind spot recognition system also includes an aggregation module, which is used to further process the classification results output by the recognition and classification module, perform aggregation processing on multiple signals with the same recognition target, calculate the aggregation confidence, and output the recognized target and the aggregation confidence as the final recognition result.

[0016] The second object of the present invention is to provide a road vehicle control system based on blind spot recognition, the system comprising the above-mentioned blind spot recognition system, feedback module, and control module; wherein the blind spot recognition system is used to identify targets within the driving blind spot and determine the type and probability of the targets; the feedback module is used to feed back the recognition results of the blind spot recognition system to the control module in real time; and the control module is used to control the vehicle operation according to the blind spot recognition results.

[0017] The third object of the present invention is to provide a road vehicle control method based on blind spot recognition, the method adopts the road vehicle control system based on blind spot recognition described above, and comprises the following steps: Step 1. Collect data; The laser emission module emits laser to the intermediate surface, and the photon-level echo receiving module receives the photon-level echo diffusely reflected by the intermediate surface and the non-viewing target surface; Step 2. Data preprocessing; The algorithm processing module and the target reconstruction module process the collected data to obtain a series of two-dimensional reconstructed signals; then the data compression module is used to perform pulse compression on each reconstructed signal; Step 3. Feature extraction; The feature extraction module extracts key features from each pulse signal, including the peak value, duration, frequency distribution, etc. of the signal; principal component analysis is then used to reduce the dimension of the features; Step 4. Real-time classification; Use the recognition and classification module for classification. For each signal, the recognition and classification module outputs a category and probability distribution, and selects the category with the highest probability as the final classification result; Step 5. Post-processing; If multiple signals are identified as the same target, the classification results are aggregated, the aggregate confidence is calculated, and the identified target and the aggregate confidence are output as the final recognition result; Step 6. The feedback module feeds back the recognition results to the control module in real time, and the control module controls the vehicle driving according to the type and aggregate confidence of the target in the driving blind spot.

[0018] Advantages and beneficial effects of the present invention: (1) In the prior art, signal processing usually relies on traditional filtering or simple signal reconstruction methods. These methods have limited effects in processing weak signals and noise interference in non-line-of-sight imaging. In complex environments and extremely weak signal conditions, it is often impossible to effectively separate signals and noise, resulting in inaccurate signal recovery. To solve this problem, the present invention represents the echo signal as a linear combination of a series of sparse basis functions. This sparse representation method can more effectively extract useful signals from noise, and is particularly suitable for scenes with weak signals and strong noise in non-line-of-sight imaging.

[0019] (2) The present invention calculates the normal vector of each diffuse reflection incident point of the non-line-of-sight target, and extracts the surface geometric information of the non-line-of-sight target based on the normal vector; through the joint regularization of the echo signal and the surface characteristics of the non-line-of-sight target, the reflectivity and surface geometry (normal) information extracted from the non-line-of-sight target are effectively used. This regularization method can constrain the geometric characteristics of the non-line-of-sight target surface, so that signal processing not only depends on the signal itself, but also combines the physical characteristics of the non-line-of-sight target; this technology is suitable for processing weak signals and noise interference caused by multiple scattering, and can provide more reliable target detection results in complex environments, significantly improving the recognition accuracy of targets in blind spots.

[0020] (3) The sparse representation of the signal in the present invention can significantly reduce the signal acquisition time; through the joint regularization of the echo signal and the surface characteristics of the non-line-of-sight target, the synergistic relationship of each data source is controlled, and the synchronous processing of the signal and the surface characteristics of the non-line-of-sight target is realized, the complexity of the calculation is reduced, and at the same time, the information fusion is effectively performed to quickly obtain concise and easy-to-judgment non-line-of-sight information; in addition, the twin support vector machine (TWSVM) is used for classification, which can process the non-line-of-sight imaging data more quickly and further improve the response speed, so as to facilitate real-time feedback of the recognition results to the vehicle control system. This rapid response capability enables the driver to make timely decisions, such as slowing down, avoiding or stopping, thereby significantly improving driving safety.

[0021] (4) The blind spot recognition system provided by the present invention is not limited to a single scenario, but can adapt to a variety of different traffic environments, including urban streets, intersections and other complex scenarios. The system can adjust the laser emission parameters and algorithm processing strategies according to the actual environment and has strong adaptability to complex scenarios.

[0022] (5) The present invention applies non-line-of-sight imaging technology to road vehicle control. The vehicle control system provided can "see" areas blocked by obstacles, effectively expanding the field of vision of the driving car. This means that the system can detect more potential dangers, warn the driver in advance or directly intervene in vehicle control, reducing the possibility of accidents.

[0023] (6) The present invention significantly reduces the safety risks caused by blind spots by real-time identification and management of blind spots. In particular, in complex traffic scenarios such as urban streets and intersections, it can effectively reduce traffic accidents caused by blind spots due to obstruction by obstacles and protect the safety of pedestrians and vehicles. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] By referring to the following description in conjunction with the accompanying drawings, and with a more comprehensive understanding of the present invention, other objects and results of the present invention will become more apparent and easy to understand. In the accompanying drawings: Figure 1 A structural block diagram of the blind spot recognition system provided by the present invention; Figure 2 A data processing flow chart of the blind spot recognition system provided by the present invention; Figure 3 A structural block diagram of a road vehicle control system based on blind spot recognition provided by the present invention; Figure 4 A flow chart of a road vehicle control method based on blind spot recognition provided by the present invention; Figure 5 It is a two-dimensional schematic diagram of the diffuse reflection incident point; Figure 6 A three-dimensional schematic diagram of the diffuse reflection incident point. DETAILED DESCRIPTION

[0025] In order to enable those skilled in the art to better understand the technical solution and advantages of the present invention, the present application is described in detail below in conjunction with the accompanying drawings, but it is not intended to limit the protection scope of the present invention.

[0026] Embodiment 1:

[0027] like Figure 1 , Figure 2 As shown, this embodiment provides a blind spot identification system, including a laser emission module, a photon-level echo receiving module, an algorithm processing module, a target reconstruction module, a data compression module, a feature extraction module, and an identification and classification module; Wherein, the laser emission module is used to emit laser toward the intermediate surface; The photon-level echo receiving module is used to receive the photon-level echo diffusely reflected by the intermediate surface and the non-viewing target surface; The algorithm processing module is used to process the echo signal, express the echo signal as a linear combination of a series of sparse basis functions, perform fast Fourier transform on the discrete echo signal to obtain the frequency domain of the signal, analyze the spectrum obtained by the fast Fourier transform, and reconstruct the reflectivity information of the non-line-of-sight target surface; at the same time, the diffuse reflection incident point of the non-line-of-sight target surface is calculated according to the coordinates of the laser emission point of the laser emission module, and for each diffuse reflection incident point of the non-line-of-sight target surface, a point set within a radius r is queried, and a normal vector of each diffuse reflection incident point is calculated, and the geometric information of the non-line-of-sight target surface is extracted according to the normal vector of the diffuse reflection incident point; The target reconstruction module uses regularization technology to jointly optimize the surface information of the non-viewing area target and the echo signal, restores the sparse coefficients by solving the joint regularization problem, and converts the restored sparse coefficients back to the original domain to obtain a reconstructed signal; The data compression module is used to perform pulse compression on the reconstructed signal to obtain a narrow pulse signal; The feature extraction module is used to extract key features from the pulse signal, including the peak value, duration, and frequency distribution of the signal; The recognition and classification module performs recognition and classification based on the extracted key features to determine the type and probability of the non-viewing area target.

[0028] In this embodiment, the emission angle and power of the laser beam emitted by the laser emission module can be adjusted according to actual needs to ensure that the detection points are more concentrated in the key area; the signal emitted by the laser emission module is reflected on the intermediate surface, and the reflected signal is emitted to the non-visual target (object hidden from sight); wherein, the intermediate surface can be a reflective surface such as a wall or the ground.

[0029] Specifically, in this embodiment, the algorithm processing module processes the data in the following manner: (1) Echo signal indicates: Represent the echo signal as a linear combination of a series of sparse basis functions: ; In the formula, is the observed echo signal vector representation, is a sparse coefficient vector, is the sparse transformation matrix, The dictionary matrix representation of represents the number of atoms in the dictionary, and the noise term is .

[0030] (2) Reconstruction of non-view target surface reflectivity: Perform fast Fourier transform on the discrete echo signal to obtain the frequency domain of the signal: ; In the formula, It is a frequency domain signal, which means at the frequency point The spectrum value at represents the total number of times the signal is sampled, Indicates sampling points, Indicates frequency components, Is an imaginary unit.

[0031] By analyzing the spectrum obtained by fast Fourier transform, the position and amplitude of each frequency peak can be extracted. The amplitude of the peak corresponds to the reflectivity information of the non-viewing target surface. .

[0032] There are differences in the reflectivity of the detection targets. Generally, the surface of a car tends to be specular with high reflectivity, while the surface of a pedestrian is closer to scattering with low reflectivity.

[0033] (3) Diffuse reflection incident point data acquisition: Assume the coordinates of the laser emission point of the laser emission module ( , , ), the incident angle of each reflection in the horizontal direction is , the vertical pitch angle is , Indicates the number of reflections; In the actual detection process, in order to simplify the analysis, it is assumed that the laser is reflected three times by the intermediate surface, the non-viewing target, and the intermediate surface, and the difference in the optical path distance is approximately negligible, such as Figure 5 , Figure 6 As shown, based on the time of flight (ToF) of each reflection , we can calculate the Flight distance of the second reflection , is the speed of light; The coordinates of the incident point of each reflection need to be calculated based on the coordinates of the previous reflection point, the reflection angle, and the distance. The coordinates of the incident point of the secondary reflection are ( , , ), No. The flight distance of the first reflection is , then The coordinates of the incident point of the secondary reflection ( , , )for: ; ; ; The second reflection occurs on the non-viewing target surface, so all The incident point coordinate information at that time is used for subsequent normal vector estimation.

[0034] (4) Normal vector estimation: Assume that the non-viewing target surface is formed diffuse reflection incident points, and estimate the normal vectors of these 3D points.

[0035] Choose one of the diffuse reflection incident points As the query point for analysis, the KD-Tree method is used to perform a radius search on the three-dimensional point set. For each diffuse reflection incident point and query point Calculate Euclidean distance : ; Then the point set with query point q as the center and r as the radius It can be expressed as: ; radius Set to , is an empirical constant with a value of 2; Compute the average point-to-point distance: ; in, Indicates sampling Diffuse reflection incident points on non-viewing target surfaces; The least squares method is used to fit the surface of the point set with the query point q as the center and r as the radius. Calculate the tangent plane, the direction of the gradient is the diffuse reflection incident point of the non-viewing target surface The normal vector ; in, Represents the fitted surface The gradient vector, that is, the rate of change of the surface at a certain point, Can be any point on the fitted surface, including the diffuse reflection incident point ; express right The partial derivative of express right The partial derivative of express right The partial derivative of The component is negated, indicating that the normal vector points to the outside of the surface, so the normal vector The component is -1.

[0036] For all diffuse reflection incident points that can be extracted from the non-viewing target surface, normal vectors are calculated according to the above steps to obtain a series of normal vector sets.

[0037] (5) Extraction of non-viewing target surface geometric information: represent q The normal vector of the query point, represent q The normal vectors of other diffuse reflection incident points in the neighborhood of the query point are calculated q The residual sum of squares of the normal vector of the query point and the normal vectors of other diffuse incident points in the neighborhood: ; In the formula, Indicates the number of points in the neighborhood; The sampled The normal vectors of the incident points on the non-viewing target surface are processed in turn, and the sum of the squared residuals is added and then divided by , and get the average residual sum of squares: .

[0038] right For analysis, usually, the car surface is relatively smooth and the body normal vector is fixed. The value is small; the surface normal vectors of non-motor vehicles or pedestrians are complex and irregular. The value is relatively large, and different targets can be distinguished by different values.

[0039] Specifically, in this embodiment, the target reconstruction module is reconstructed in the following manner: The reconstructed morphology of non-line-of-sight targets is described as a combination of echo signals and surface information of non-line-of-sight targets, and sparse reconstruction is performed using the total variation regularization method.

[0040] The surface information of the non-viewing target includes normal information and reflectivity information. In this embodiment, the normal information of the non-viewing target surface is combined with the reflectivity for joint optimization to jointly constrain and Spatial distribution of: ; In the formula, About Smoothness constraints on surface normals, It's about reflectivity The sparsity constraint is is a regularization term that includes normal information and reflectivity, and is controlled by and This makes the solution smooth and sparse.

[0041] It is difficult to directly introduce the surface information of the non-viewing target. In this embodiment, the parameter , , , is the weighting coefficient, and the two constraints are balanced by changing its value to relax the spatial distribution information of the object.

[0042] Since the echo signal and the surface information of the non-viewing object do not belong to the same dimension, the present invention controls the collaborative relationship of each data source through a joint regularization term, reduces the complexity of the model, and realizes effective information fusion, so as to quickly obtain simple and easy-to-judge non-viewing information.

[0043] The joint regularization term is expressed as a weighted combination of two prior terms: ; in, Contains the original observations The regularization term of is a joint regularization term that includes both the observed echo signal and the non-viewing target surface information. , is the weighting coefficient, and the weighting ratio can be controlled by changing its value.

[0044] Afterwards, the sparse coefficients are recovered by solving the following joint regularization problem : ; Specifically, the problem is solved using the alternating iteration method, and the mathematical form of the recovered sparse coefficients can be expressed as the solution to the iterative optimization problem: ; After the recovery, the sparse coefficients Convert back to the original domain and get the reconstructed signal on the final two-dimensional view (two-dimensional reconstructed signal) : ; Where n is the number of atoms in the dictionary.

[0045] The above reconstruction method can not only realize the synchronous processing of echo signals and surface characteristics of non-viewing objects, but also make the reconstruction more accurate and provide more reliable target detection results in complex environments.

[0046] Furthermore, in this embodiment, the recognition and classification module uses a trained twin support vector machine (TWSVM) for recognition and classification. When the twin support vector machine is trained, the following steps are included: Step 1. Collect sample data; At the intersection of urban streets, the non-visual targets to be identified are pedestrians and vehicles. The training set contains 3D models of 50 pedestrians and 50 vehicles, a total of 100 samples, 10 signal transmission and collection for each sample, and the number of sampling points for each signal is 1024; the test set contains 3D models of 10 pedestrians and 10 vehicles, and the sampling method is the same as above.

[0047] Step 2. Data preprocessing; The algorithm processing module and the target reconstruction module process the collected data and finally obtain a series of two-dimensional reconstructed signals; Pulse compression is performed on each reconstructed signal to obtain a narrower pulse signal, thereby improving the resolution of the signal and facilitating subsequent detection.

[0048] Step 3. Feature extraction; Extract key features from each pulse signal, including the peak value, duration, frequency distribution, etc. of the signal.

[0049] Principal component analysis (PCA) is used to reduce the dimension of features to reduce the computational burden of subsequent classification algorithms.

[0050] Step 4. Model training; Since the amount of data is large in actual situations, the twin support vector machine (TWSVM) is used for classification. TWSVM has low computational complexity and can significantly reduce the calculation time.

[0051] Specifically, the data processed in step 3 is input into the twin support vector machine model for training, and the classification decision function is defined: ; In the formula, represents the input sample, is the number of samples, is the Lagrange multiplier, It is i The labels of the training samples, is an adaptive kernel function (in the kernel function In the selection of is the bias term.

[0052] In this embodiment, when searching for the kernel function K, the kernel width is automatically adjusted according to the local density of the signal and the distance between the signal peaks. For sparse areas, a larger kernel width is used. , and use smaller .

[0053] Step 5. Model evaluation; The performance of the TWSVM classifier is evaluated on the test set, using a cross-validation method to reduce the risk of overfitting.

[0054] The accuracy, precision, recall and F1 score of the TWSVM classifier are calculated to ensure that the TWSVM model has good generalization ability in the non-visual field object classification task.

[0055] Furthermore, in this embodiment, the blind spot recognition system also includes an aggregation module, which is used to further process the classification results output by the recognition and classification module, perform aggregation processing on multiple signals with the same recognition target, calculate the aggregation confidence, and output the recognized target and the aggregation confidence as the final recognition result.

[0056] Embodiment 2: like Figure 3 As shown, this embodiment provides a road vehicle control system based on blind spot recognition, and the system includes the blind spot recognition system, feedback module, and control module described in the above-mentioned embodiment 1; wherein the blind spot recognition system is used to identify targets in the driving blind spot (non-visual area) and determine the type and probability of the target; the feedback module is used to feed back the recognition results of the blind spot recognition system to the control module in real time; the control module is used to control the vehicle operation according to the blind spot recognition results.

[0057] Embodiment 3: like Figure 4 As shown, this embodiment provides a road vehicle control method based on blind spot recognition, the method comprising the following steps: Step 1. Collect data; The laser emission module emits laser to the intermediate surface, and the photon-level echo receiving module receives the photon-level echo diffusely reflected by the intermediate surface and the non-viewing target surface; Step 2. Data preprocessing; The algorithm processing module and the target reconstruction module process the collected data to obtain a series of two-dimensional reconstructed signals; then the data compression module is used to perform pulse compression on each reconstructed signal to obtain a narrower pulse signal; Step 3. Feature extraction; The feature extraction module extracts key features from each pulse signal, including the peak value, duration, frequency distribution, etc. of the signal; then principal component analysis (PCA) is used to reduce the dimension of the features; Step 4. Real-time classification; Use the trained TWSVM model (recognition and classification module) for classification. For each signal, the TWSVM model outputs a category and probability distribution, and selects the category with the highest probability as the final classification result; Step 5. Post-processing; If multiple signals are identified as the same target, the classification results are aggregated, the aggregate confidence is calculated, and the identified target and the aggregate confidence are output as the final recognition result; Assume that the three signals obtained after the above processing are all identified as pedestrians, with probabilities of 0.88, 0.9, and 0.92 respectively. Aggregation processing is performed on them, and the aggregate confidence is calculated as P = (0.88 + 0.9 + 0.92) / 3 = 0.9.

[0058] Example description of classification results: Assume that the following two signals are collected in the non-viewing area of ​​the intersection: Signal 1: Feature extraction results: The peak height is moderate, the duration is short, and the frequency distribution is concentrated in the low frequency band.

[0059] TWSVM classification results: The probability distribution shows that the probability of pedestrian class is 0.9, that of bicycle is 0.05, and the probability of other classes is less than 0.05.

[0060] Judgment: Sample 1 is identified as a pedestrian with high confidence.

[0061] Signal 2: Feature extraction results: lower peak height, longer duration, and wider frequency distribution.

[0062] TWSVM classification results: The probability distribution shows that the probability of the car class is 0.85, the motorcycle is 0.1, and the probability of other classes is less than 0.05.

[0063] Judgment: Sample 2 is identified as a car with a high confidence level.

[0064] Through the above steps, the recognition and classification module can accurately determine the type and probability of non-visual targets and provide real-time decision support for road vehicle control systems; Step 6. The feedback module feeds back the recognition results to the control module in real time, and the control module controls the vehicle driving according to the type and confidence of the target in the driving blind spot.

[0065] Specifically, if the target in the blind spot is a pedestrian and the confidence level is greater than 0.85, avoidance measures are taken; if the confidence level is between 0.7-0.8, the vehicle is decelerated or stopped; if the target in the blind spot is a vehicle and the confidence level is between 0.7-0.8, the vehicle remains vigilant and is prepared to take avoidance measures; if the target in the blind spot is a vehicle and the confidence level is greater than 0.85, the vehicle is decelerated and a safe distance is maintained to improve driving safety and reliability.

[0066] The present invention also provides an electronic device, comprising: one or more processors and a memory; wherein the memory is used to store one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors implement the above-mentioned road vehicle control method based on blind spot recognition.

[0067] The present invention also provides a computer-readable medium having a computer program stored thereon, and when the computer program is executed by a processor, the road vehicle control method based on blind spot recognition described above is implemented.

[0068] Those skilled in the art will appreciate that all or part of the functions of the various methods / modules in the above embodiments may be implemented by hardware or by computer programs. When all or part of the functions in the above embodiments are implemented by computer programs, the program may be stored in a computer-readable storage medium, which may include: a read-only memory, a random access memory, a disk, an optical disk, a hard disk, etc. The program is executed by a computer to implement the above functions. For example, the program is stored in the memory of the device, and when the program in the memory is executed by the processor, all or part of the above functions can be implemented.

[0069] In addition, when all or part of the functions in the above-mentioned embodiments are implemented by means of a computer program, the program can also be stored in a storage medium such as a server, another computer, a disk, an optical disk, a flash drive or a mobile hard disk, and saved to the memory of a local device by downloading or copying, or the system of the local device is updated. When the program in the memory is executed by the processor, all or part of the functions in the above-mentioned embodiments can be implemented.

[0070] The above specific examples are used to illustrate the present invention, which is only used to help understand the present invention and is not intended to limit the present invention. For those skilled in the art of the present invention, according to the idea of ​​the present invention, some simple deductions, deformations or substitutions can be made. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A blind spot recognition system, characterized in that: It includes laser emission module, photon-level echo receiving module, algorithm processing module, target reconstruction module, data compression module, feature extraction module, recognition and classification module; Wherein, the laser emission module is used to emit laser toward the intermediate surface; The photon-level echo receiving module is used to receive the photon-level echo diffusely reflected by the intermediate surface and the non-viewing target surface; The algorithm processing module is used to process the echo signal, express the echo signal as a linear combination of a series of sparse basis functions, perform fast Fourier transform on the discrete echo signal to obtain the frequency domain of the signal, analyze the spectrum obtained by the fast Fourier transform, and reconstruct the reflectivity information of the non-line-of-sight target surface; at the same time, the diffuse reflection incident point of the non-line-of-sight target surface is calculated according to the coordinates of the laser emission point of the laser emission module, and for each diffuse reflection incident point of the non-line-of-sight target surface, a point set within a radius r is queried, and a normal vector of each diffuse reflection incident point is calculated, and the geometric information of the non-line-of-sight target surface is extracted according to the normal vector of the diffuse reflection incident point; The target reconstruction module uses regularization technology to jointly optimize the surface information of the non-viewing area target and the echo signal, restores the sparse coefficients by solving the joint regularization problem, and converts the restored sparse coefficients back to the original domain to obtain a reconstructed signal; The data compression module is used to perform pulse compression on the reconstructed signal to obtain a pulse signal; The feature extraction module is used to extract key features from the pulse signal, including the peak value, duration, and frequency distribution of the signal; The recognition and classification module performs recognition and classification based on the extracted key features to determine the type and probability of the non-viewing area target.

2. A blind spot recognition system according to claim 1, characterized in that: The expression for expressing the echo signal as a linear combination of a series of sparse basis functions is: ； In the formula, is the observed echo signal vector representation, is a sparse coefficient vector, is the sparse transformation matrix, The dictionary matrix representation of represents the number of atoms in the dictionary, and the noise term is ; The method of performing fast Fourier transform on discrete echo signals to obtain the signal frequency domain is: ； In the formula, It is a frequency domain signal, which means at the frequency point The spectrum value at Represents the total number of times the signal is sampled, Indicates sampling points, Indicates frequency components, is an imaginary unit; By analyzing the spectrum obtained by fast Fourier transform, the position and amplitude of each frequency peak are extracted. The amplitude of the peak corresponds to the reflectivity information of the non-viewing target surface. .

3. A blind spot recognition system according to claim 1, characterized in that: The method for obtaining the diffuse reflection incident point of the non-viewing target surface is as follows: Assume the coordinates of the laser emission point of the laser emission module ( , , ), the incident angle of each reflection in the horizontal direction is , the vertical pitch angle is , Indicates the number of reflections; The coordinates of the incident point of each reflection are calculated based on the coordinates of the previous reflection point, the reflection angle, and the distance. The coordinates of the incident point of the secondary reflection are ( , , ), No. The flight distance of the first reflection is , then The coordinates of the incident point of the secondary reflection ( , , )for: ； ； ； Set up i The second reflection occurs on the non-viewing target surface. i The coordinate information of all incident points can be obtained by reflecting it once.

4. A blind spot recognition system according to claim 3, characterized in that: The normal vector calculation method for each diffuse reflection incident point is: Non-line-of-sight target surface formation diffuse reflection incident points, select any one of the diffuse reflection incident points As the query point, the KD-Tree method is used to perform a radius search on the three-dimensional point set. For each diffuse reflection incident point and query point Calculate Euclidean distance : ； Then the point set with query point q as the center and r as the radius It is expressed as: ； in, For the radius, set , is an empirical constant with a value of 2; is the average distance between points, and the calculation formula is: ； In the formula, Indicates the sampling Diffuse reflection incident points on non-viewing target surfaces; The least squares method is used to fit the surface of the point set with the query point q as the center and r as the radius. Calculate the tangent plane, the direction of the gradient is the diffuse reflection incident point of the non-viewing target surface The normal vector ;in, express right The partial derivative of express right The partial derivative of express right The partial derivative of Normal vectors are calculated for all diffuse reflection incident points that can be extracted from the non-viewing target surface to obtain a series of normal vector sets.

5. A blind spot recognition system according to claim 4, characterized in that: The method of extracting the geometric information of the non-viewing target surface based on the normal vector of the diffuse reflection incident point is: represent q The normal vector of the query point, represent q The normal vectors of other diffuse reflection incident points in the neighborhood of the query point are calculated q The residual sum of squares of the normal vector of the query point and the normal vectors of other diffuse incident points in the neighborhood: ； In the formula, Indicates the number of points in the neighborhood; The sampled The normal vectors of the incident points on the non-viewing target surface are processed in turn, and the sum of the squared residuals is added and then divided by , and get the average residual sum of squares: ,use Represents the non-viewing area target surface geometry information.

6. A blind spot recognition system according to claim 5, characterized in that: The target reconstruction module uses regularization technology to jointly optimize the non-viewing target surface information and the echo signal, and first jointly optimizes the non-viewing target surface geometry information and reflectivity information: ； In the formula, About Smoothness constraints on surface normals, It's about reflectivity The sparsity constraint is is a regularization term that includes normal information and reflectivity, , is the weighting coefficient; Then, the regularization technology is used to jointly optimize the surface information of the non-viewing target and the echo signal. The expression is: ； In the formula, Contains the original observations The regularization term of is a joint regularization term that includes both the echo signal and the non-viewing target information. , is the weighting coefficient, and the weighting ratio can be controlled by changing its value; Afterwards, the alternating iterative method is used to solve the joint regularization problem to recover the sparse coefficients, which is expressed as: ； In the formula, is the observed echo signal vector representation, is a sparse coefficient vector, is the sparse transformation matrix, Dictionary matrix representation of ; It is a joint regularization term that includes both the echo signal and the surface information of the non-viewing target; Finally, the restored sparse coefficients Convert back to the original domain to get the reconstructed signal on the final two-dimensional view : ； In the formula, Indicates the number of atoms in the dictionary.

7. A blind spot recognition system according to claim 6, characterized in that: The recognition and classification module uses a trained twin support vector machine; when training the twin support vector machine, the following steps are included: Step 1. Collect sample data; The sample data includes pedestrians and vehicles. Multiple signal transmission and collection are performed for each sample, and the data is divided into training set and test set. Step 2. Data preprocessing; The algorithm processing module and the target reconstruction module process the collected data and finally obtain a series of two-dimensional reconstructed signals; Performing pulse compression on each reconstructed signal to obtain a pulse signal; Step 3. Feature extraction; Extract key features from each pulse signal, including the peak value, duration, and frequency distribution of the signal; use principal component analysis to reduce the dimension of the features; Step 4. Input the data processed in step 3 into the twin support vector machine model for training, and define the classification decision function as: ; In the formula, represents the input sample, is the number of samples, is the Lagrange multiplier, It is i The labels of the training samples, is an adaptive kernel function, is the bias term.

8. The blind spot recognition system according to claim 7, characterized in that: The blind spot recognition system also includes an aggregation module, which is used to further process the classification results output by the recognition and classification module, perform aggregation processing on multiple signals with the same recognition target, calculate the aggregation confidence, and output the recognized target and the aggregation confidence as the final recognition result.

9. A road vehicle control system based on blind spot recognition, characterized in that: The system includes the blind spot recognition system, feedback module, and control module as described in any one of claims 1 to 8; wherein the blind spot recognition system is used to identify targets in the driving blind spot and determine the type and probability of the targets; the feedback module is used to feed back the recognition results of the blind spot recognition system to the control module in real time; and the control module is used to control the operation of the vehicle according to the blind spot recognition results.

10. A road vehicle control method based on blind spot recognition, characterized in that: The method adopts the road vehicle control system described in claim 9, comprising the following steps: Step 1. Collect data; The laser emission module emits laser to the intermediate surface, and the photon-level echo receiving module receives the photon-level echo diffusely reflected by the intermediate surface and the non-viewing target surface; Step 2. Data preprocessing; The algorithm processing module and the target reconstruction module process the collected data to obtain a series of two-dimensional reconstructed signals; then the data compression module is used to perform pulse compression on each reconstructed signal; Step 3. Feature extraction; The feature extraction module extracts key features from each pulse signal, including the peak value, duration, and frequency distribution of the signal; principal component analysis is then used to reduce the dimension of the features; Step 4. Real-time classification; Use the recognition and classification module for classification. For each signal, the recognition and classification module outputs a category and probability distribution, and selects the category with the highest probability as the final classification result; Step 5. Post-processing; If multiple signals are identified as the same target, the classification results are aggregated, the aggregate confidence is calculated, and the identified target and the aggregate confidence are output as the final recognition result; Step 6. The feedback module feeds back the recognition results to the control module in real time, and the control module controls the vehicle driving according to the type and aggregate confidence of the target in the driving blind spot.

Images