Warning method and device for disabled people, electronic equipment and storage medium
Through the micro Doppler feature extraction of radar detection data and the classification identification of SVM models, the problem of poor detection accuracy of disabled people in the prior art is solved, and accurate detection and early warning of disabled people under small sample sizes is achieved, and traffic safety is improved.
Patent Information
- Application Number
- CN202311814696.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-12-26
- Publication Date
- 2025-06-27
AI Technical Summary
The prior art is difficult to achieve accurate detection of people with disabilities through deep learning, mainly because of the small sample data of people with disabilities.
By obtaining radar detection data, micro Doppler feature extraction is performed, including trunk Doppler information, total Doppler bandwidth and limb motion cycle, and these features are input into the pre-trained SVM model for classification identification. If a person with a disability is detected, an early warning will be issued.
Accurate detection of disabled people with disabilities under a small sample size has been achieved, the early warning ability of disabled people in assisted driving or autonomous driving has been improved, and traffic safety has been enhanced.
Smart Images
Figure CN120214722A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of assisted driving technology, and particularly to a warning method, device, electronic device and storage medium for disabled persons. Background Art
[0002] Disabled persons are a major social issue worldwide. As traffic participants, they may not be able to avoid traffic accidents in a timely manner due to their inconvenient mobility. Therefore, if disabled persons can be pre-perceived and identified during assisted driving and autonomous driving, and a warning can be issued to the driver, the traffic safety of disabled persons can be protected to a great extent.
[0003] Currently, pedestrian detection and recognition are usually carried out through deep learning. In this way, relatively accurate pedestrian detection and segmentation accuracy can be obtained through a large amount of data sampling and neural network training. However, for disabled persons, due to the small amount of their sample data, it is difficult to obtain good detection and segmentation accuracy through large-scale training of deep learning, and thus it is difficult to achieve accurate detection of disabled persons. Summary of the Invention
[0004] The present disclosure is completed to solve the above problems, and aims to provide a warning method, device, electronic device and storage medium for disabled persons, which can achieve relatively accurate detection of disabled persons with a small sample size, so as to give an early warning and further ensure the traffic safety of disabled persons.
[0005] According to one aspect of the present disclosure, a warning method for disabled persons is provided, including: obtaining data of a pedestrian to be detected, where the data of the pedestrian to be detected is radar detection data; extracting micro-Doppler features from the data of the pedestrian to be detected, and the extracted features include torso Doppler information, total Doppler bandwidth and limb movement period of the pedestrian to be detected; inputting the extracted features into an SVM model for classification and recognition, where the SVM model is a pre-trained model; and if the pedestrian to be detected is classified and recognized as a disabled person, issuing a warning.
[0006] Preferably, the SVM model is pre-trained through the following steps: obtaining a pedestrian sample data set, where the pedestrian sample data set is radar detection data and includes different types of pedestrian samples and corresponding label information; extracting micro-Doppler features from each pedestrian sample in the pedestrian sample data set to obtain signal statistical features of different types of pedestrian samples, where the signal statistical features include statistical torso Doppler information, statistical total Doppler bandwidth and statistical limb movement period; determining the signal statistical feature ranges of different types of pedestrians based on the signal statistical features of different types of pedestrian samples; and training the SVM model based on the signal statistical feature ranges of different types of pedestrians and the corresponding label information.
[0007] Preferably, in the training of the SVM model based on the signal statistical feature ranges of different types of pedestrians and the corresponding label information, it includes: extracting the SVM hyperplane based on the signal statistical feature ranges of different types of pedestrians and the corresponding label information; and obtaining the SVM classification decision function through the SVM hyperplane.
[0008] Preferably, micro-Doppler feature extraction is performed on the pedestrian data to be detected, and the extracted features include the torso Doppler information, the total Doppler bandwidth, and the limb movement period of the pedestrian to be detected, including: performing range-Doppler processing on the pedestrian data to be detected; performing time-Doppler processing based on the result of the range-Doppler processing; and extracting the torso Doppler information, the total Doppler bandwidth, and the limb movement period of the pedestrian to be detected based on the results of the range-Doppler processing and the time-Doppler processing.
[0009] Preferably, micro-Doppler feature extraction is performed on each pedestrian sample in the pedestrian sample dataset to obtain the signal statistical features of different types of pedestrian samples, including: performing range-Doppler processing on each pedestrian sample in the pedestrian sample dataset; performing time-Doppler processing based on the result of the range-Doppler processing; and analyzing and statistically processing the processed result signals of different types of pedestrian samples in combination with the label information of each pedestrian sample, so as to obtain the signal statistical features of different types of pedestrian samples.
[0010] Preferably, the warning method for disabled persons further includes the following steps: after obtaining the pedestrian data to be detected, calculating the distance and speed of the pedestrian to be detected relative to the radar based on the pedestrian data to be detected, and only performing subsequent processing when the distance and the speed meet the preset conditions.
[0011] Preferably, the pedestrian sample dataset includes normal pedestrians and disabled persons.
[0012] Preferably, the torso Doppler information is the torso Doppler frequency, and the statistical torso Doppler information is the statistical torso Doppler frequency.
[0013] Preferably, the torso Doppler information is the torso Doppler bandwidth, and the statistical torso Doppler information is the statistical torso Doppler bandwidth.
[0014] According to another aspect of the present disclosure, there is provided a warning device for disabled persons, including: an acquisition module configured to acquire pedestrian data to be detected, where the pedestrian data to be detected is radar detection data; a feature extraction module configured to perform micro-Doppler feature extraction on the pedestrian data to be detected, and the extracted features include the torso Doppler information, the total Doppler bandwidth, and the limb movement period of the pedestrian to be detected; a classification and recognition module configured to input the extracted features into an SVM model for classification and recognition, where the SVM model is a pre-trained model; and a warning module configured to issue a warning if the pedestrian to be detected is classified and recognized as a disabled person.
[0015] According to another aspect of the present disclosure, there is provided an electronic device, where the electronic device includes a processor and a memory, and at least one instruction or at least one program segment is stored in the memory, and the at least one instruction or the at least one program segment is loaded and executed by the processor to perform the method described in the above aspect.
[0016] According to another aspect of the present disclosure, there is provided a computer storage medium, where at least one instruction or at least one program segment is stored in the storage medium, and the at least one instruction or the at least one program segment is loaded and executed by a processor to perform the method described in the above aspect as claimed. Description of the Drawings
[0017] The drawings exemplarily show embodiments and form a part of the description, and are used together with the written description of the description to explain the exemplary embodiments of the embodiments. The shown embodiments are only for illustrative purposes and do not limit the scope of the claims. In all the drawings, the same reference numerals refer to similar but not necessarily identical elements.
[0018] Figure 1 It shows a schematic flowchart of the disabled person warning method provided by the embodiment of the present disclosure;
[0019] Figure 2 It shows a schematic flowchart of the pre-training of the SVM model in the disabled person warning method of the embodiment of the present disclosure;
[0020] Figure 3 It shows a structural block diagram of the radar control device provided by the embodiment of the present disclosure;
[0021] Figure 4 It shows a structural block diagram of the electronic device provided by the embodiment of the present disclosure. Detailed Embodiments
[0022] Next, the present disclosure will be further described in detail with reference to the accompanying drawings and embodiments. It can be understood that the specific embodiments described herein are only used to explain the related invention, rather than limiting the invention. Additionally, it should be noted that for the convenience of description, only the parts related to the relevant invention are shown in the drawings.
[0023] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. As used herein, the singular forms "a" and "the" are also intended to include the plural forms, unless the context clearly indicates otherwise. It will also be understood that when the terms "comprises" and / or "consists of" are used in this specification, the specified features, wholes, steps, operations, elements, and / or components are present, but one or more other features, wholes, steps, operations, elements, components, and / or their groups are not excluded.
[0024] The embodiments described herein can be described with reference to plan views and / or cross-sectional views by means of the ideal schematic diagrams of the present disclosure. Therefore, the example illustrations can be modified according to manufacturing techniques and / or tolerances. Therefore, the embodiments are not limited to the embodiments shown in the drawings, but include modifications of the configurations formed based on the manufacturing process. Therefore, the regions illustrated in the drawings have schematic properties, and the shapes of the regions shown in the drawings illustrate the specific shapes of the regions of the elements, but are not intended to be restrictive.
[0025] Unless otherwise defined, the meanings of all terms (including technical and scientific terms) used herein are the same as those commonly understood by those of ordinary skill in the art. It will also be understood that terms such as those defined in common dictionaries should be interpreted as having meanings consistent with their meanings in the context of the relevant art and the present disclosure, and will not be interpreted as having idealized or overly formal meanings, unless clearly defined herein.
[0026] The present disclosure provides a warning method, device, electronic device, and storage medium for disabled persons, which can achieve relatively accurate detection of disabled persons with a relatively small sample size, so as to give early warnings and further ensure the traffic safety of disabled persons.
[0027] Figure 1 A flowchart showing the warning method for disabled persons provided by an embodiment of the present disclosure is shown. This specification provides method operation steps such as in the embodiments or flowcharts, but may include more or fewer operation steps based on routine or non-creative labor. The order of steps listed in the embodiments is only one way among the execution orders of numerous steps and does not represent the only execution order. When the actual system or server product executes, it can be executed in the order of the embodiments or the method shown in the drawings, or executed in parallel (for example, in an environment of parallel processors or multi-threaded processing).
[0028] S101: Obtain the pedestrian data to be detected, where the pedestrian data to be detected is radar detection data.
[0029] The radar detection data is obtained by receiving the echo of the electromagnetic wave emitted by the radar device, and can calculate information such as the distance and speed of the target (vehicle, pedestrian, obstacle, etc.). The radar detection data can be obtained by detecting with the forward millimeter-wave radar and corner millimeter-wave radar arranged on the vehicle.
[0030] S102: Extract the micro-Doppler features from the pedestrian data to be detected, and the extracted features include the torso Doppler information, the total Doppler bandwidth, and the limb movement period of the pedestrian to be detected.
[0031] The extraction of the micro-Doppler features from the pedestrian data to be detected is specifically processed as follows.
[0032] First, perform range-Doppler processing on the pedestrian data to be detected.
[0033] Then, perform time-Doppler processing based on the result of the range-Doppler processing. Specifically, after the range-Doppler processing, n range-Doppler maps are obtained, and each element in the range-Doppler map is called a "range cell". Then, for each range-Doppler map, the range cells are added together along the range axis to form a vector e. Finally, n temporally consecutive frames are combined together to form a time-Doppler spectrogram E with a time length of n frames.
[0034] Finally, based on the results of the range-Doppler processing and the time-Doppler processing, the features such as the torso Doppler information, the total Doppler bandwidth, and the limb movement period of the pedestrian to be detected are extracted.
[0035] (I) The torso Doppler information can be the torso Doppler frequency x1, which corresponds to the torso speed of the pedestrian. The human torso speed is a very basic but important piece of information, and there are significant differences in the torso speed for different activities. It is shown as follows in Equation (1):
[0036]
[0037] where, dopplerArgmax(e i ) represents the Doppler shift corresponding to the maximum signal intensity in the vector e, V i represents the torso movement speed in one frame, λ represents the wavelength, and Z represents the total number of frames during the entire observation period.
[0038] In addition, the Doppler information can also be the torso Doppler bandwidth.
[0039] (II) The total bandwidth x2 of the Doppler signal is related to the movement speed of the limbs. A rapid swinging of the arm or leg will result in a larger bandwidth. It is shown as the following formula (2):
[0040]
[0041] where E Tw is the time-Doppler spectrogram with a time window size of T w . and respectively represent extracting the upper envelope and the lower envelope within the time window.
[0042] (III) The limb movement period x3 corresponds to the swinging rate of the arm and leg. It is shown as the following formula (3):
[0043]
[0044] where represents the number of extreme points of the upper envelope and the lower envelope within the time window.
[0045] S103: Input the extracted features into the SVM model for classification and recognition. The SVM model is a pre-trained model.
[0046] This SVM model is pre-trained and can classify pedestrians to identify whether the pedestrian to be detected is a disabled person. Here, the disabled person mainly refers to a pedestrian with a disability and inconvenient movement.
[0047] S104: If the pedestrian to be detected is classified and recognized as a disabled person, issue a warning.
[0048] When it is classified and recognized as a disabled person in step S103, a warning is issued, so as to remind the driver to pay attention to the disabled person existing around the vehicle.
[0049] In addition, in some embodiments, the following steps may further be included: After obtaining the data of the pedestrian to be detected, calculate the distance and speed of the pedestrian to be detected relative to the radar based on the data of the pedestrian to be detected, and perform subsequent processing only when the distance and the speed meet the preset conditions. That is, after step S101, calculate the distance and speed of the pedestrian to be detected relative to the radar based on the data of the pedestrian to be detected obtained in step S101, and perform subsequent classification and recognition and warning processing only when the distance and the speed meet the preset conditions.
[0050] Figure 2 shows a schematic flowchart of the pre-training of the SVM model in the disabled person warning method of the present disclosure embodiment. The following is an explanation of the pre-training process of the SVM model based on Figure 2 .
[0051] S201: Obtain a pedestrian sample dataset, which is radar detection data and contains different types of pedestrian samples and corresponding label information;
[0052] Among them, the pedestrian sample dataset includes normal pedestrians and disabled persons. Here, the disabled persons mainly refer to pedestrians with disabilities and inconvenient movement. Moreover, all pedestrian samples in the pedestrian sample dataset have been pre-labeled with corresponding label information. Through this label information, it can be known which type of pedestrian a pedestrian sample belongs to, whether it is a normal pedestrian or a disabled person.
[0053] In addition, the pedestrian sample dataset is also radar detection data, which is obtained by receiving the echo of the electromagnetic wave emitted by the radar device and can calculate information such as the distance and speed of the object.
[0054] S202: Extract the micro-Doppler features of each pedestrian sample in the pedestrian sample dataset to obtain the signal statistical features of different types of pedestrian samples. The signal statistical features include statistical torso Doppler information, statistical Doppler total bandwidth, and statistical limb movement period.
[0055] Specifically, the micro-Doppler feature extraction refers to the following processing.
[0056] First, perform range-Doppler processing on each pedestrian sample (radar detection data) in the pedestrian sample dataset.
[0057] Then, perform time-Doppler processing. Specifically, after range-Doppler processing, n range-Doppler maps are obtained. Each element in the range-Doppler map is called a "range bin". Then, for each range-Doppler map, the range bins are summed along the range axis to form a vector e. Finally, n temporally consecutive frames are combined together to form a time-Doppler spectrogram E with a time length of n frames.
[0058] Finally, after the above range-Doppler processing and time-Doppler processing, combined with the label information of each pedestrian sample, analyze and statistically process the processed result signals of different types of pedestrian samples, so as to obtain the signal statistical features of different types of pedestrian samples. The processed result signals here include the result signals of range-Doppler processing and the result signals of time-Doppler processing. The signal statistical features include statistical torso Doppler information, statistical Doppler total bandwidth, and statistical limb movement period. They are statistically obtained according to the torso Doppler information, Doppler total bandwidth, and limb movement period of different types of pedestrian samples respectively. For example, the statistical torso Doppler information, statistical Doppler total bandwidth, and statistical limb movement period of normal pedestrians are statistically obtained according to normal pedestrian samples, and the statistical torso Doppler information, statistical Doppler total bandwidth, and statistical limb movement period of disabled persons are statistically obtained according to disabled person samples.
[0059] The torso Doppler information, the total bandwidth of the Doppler signal, and the limb movement cycle have been introduced above and will not be elaborated here. Similarly, the torso Doppler information can be statistically analyzed by counting the torso Doppler frequency or the torso Doppler bandwidth.
[0060] S203: Determine the signal statistical feature ranges of different types of pedestrians based on the signal statistical features of different types of pedestrian samples.
[0061] After obtaining the signal statistical features of different types of pedestrian samples based on step S202, the signal statistical feature ranges of different types of pedestrians can be determined.
[0062] S204: Train the SVM model based on the signal statistical feature ranges of different types of pedestrians and the corresponding label information.
[0063] Specifically, first extract the SVM hyperplane based on the signal statistical feature ranges of different types of pedestrians and the corresponding label information.
[0064] Assume that each sample "feature - label" pair is represented as [x i , y i , where x represents the feature and y represents the label. Then the SVM hyperplane to be found can be expressed as the following formula (4).
[0065] wx + b = 0 (4)
[0066] where w represents the weight vector and b is a scalar.
[0067] Assume that sample C1 is below the hyperplane, then the following formula (5) holds.
[0068] wx i + b < -1, x i ∈ C1, y i = -1 (5)
[0069] For the second - type sample C2 that falls on the other side of the hyperplane, then the following formula (6) holds.
[0070] wx i + b > -1, x i ∈ C2, y i = 1 (6)
[0071] Then, obtain the SVM classification decision function through the SVM hyperplane.
[0072] According to the above formulas (4), (5), and (6), the SVM classification decision function shown in the following formula (7) can be obtained.
[0073]
[0074] Among them, α i is the Lagrange multiplier of the support vector, which can be used to determine the weight of the support vector, and σ is the bandwidth of the Gaussian kernel.
[0075] Figure 3 The structural block diagram of the disabled person warning device provided by the embodiment of the present disclosure is shown. As Figure 3 shown, the disabled person warning device 200 includes: an acquisition module 201, a feature extraction module 202, a classification and recognition module 203, and a warning module 204.
[0076] Among them, the acquisition module 201 is used to acquire the pedestrian data to be detected, and the pedestrian data to be detected is radar detection data. The feature extraction module 202 is used to perform micro-Doppler feature extraction on the pedestrian data to be detected, and the extracted features include the torso Doppler information, the total Doppler bandwidth, and the limb movement period of the pedestrian to be detected. The classification and recognition module 203 is used to input the extracted features into the SVM model for classification and recognition, and the SVM model is a pre-trained model. If the pedestrian to be detected is classified and recognized as a disabled person, the warning module 204 issues a warning.
[0077] Figure 4 The structural block diagram of the electronic device provided by the embodiment of the present disclosure is shown. As Figure 4 shown, the present disclosure also provides an electronic device 300, which includes a processor and a memory. At least one instruction or at least one program segment is stored in the memory, and the at least one instruction or the at least one program segment is loaded and executed by the processor to perform the method described in the above embodiments.
[0078] The present disclosure also provides a computer storage medium, in which at least one instruction or at least one program segment is stored, and the at least one instruction or the at least one program segment is loaded and executed by the processor to implement the method described in the above embodiments.
[0079] Optionally, in this embodiment, the above storage medium may be located in at least one of multiple network servers of a computer network. Optionally, in this embodiment, the above storage medium may include, but is not limited to: USB flash drive, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk, or optical disc and other various media that can store program codes.
[0080] Those skilled in the art should be able to realize that the modules, units, and method steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of electronic hardware and software, the composition and steps of each example have been generally described according to functions in the above description. Whether these functions are executed in the form of electronic hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of this disclosure.
[0081] Although this disclosure has been described with reference to the current specific embodiments, those of ordinary skill in the art should recognize that the scope of the invention involved in this disclosure is not limited to the technical solutions formed by the specific combination of the above technical features, and should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the inventive concept. For example, the technical solutions formed by mutually replacing the above features with the technical features (but not limited to) having similar functions disclosed in this disclosure.
Claims
1. A warning method for disabled persons, characterized in that, Including: Obtain pedestrian data to be detected, where the pedestrian data to be detected is radar detection data; Extract micro-Doppler features from the pedestrian data to be detected, and the extracted features include the torso Doppler information, the total Doppler bandwidth, and the limb movement period of the pedestrian to be detected; Input the extracted features into an SVM model for classification and recognition, where the SVM model is a pre-trained model; and If the pedestrian to be detected is classified and recognized as a disabled person, issue a warning.
2. The warning method for disabled persons according to claim 1, wherein The SVM model is pre-trained through the following steps: Obtain a pedestrian sample data set, where the pedestrian sample data set is radar detection data, including different types of pedestrian samples and corresponding label information; Extract micro-Doppler features from each pedestrian sample in the pedestrian sample data set to obtain the signal statistical features of different types of pedestrian samples, where the signal statistical features include the statistical torso Doppler information, the statistical total Doppler bandwidth, and the statistical limb movement period; Determine the signal statistical feature ranges of different types of pedestrians based on the signal statistical features of different types of pedestrian samples; And Train the SVM model based on the signal statistical feature ranges of different types of pedestrians and the corresponding label information.
3. The warning method for disabled persons according to claim 1 or 2, characterized in that, During the training of the SVM model based on the signal statistical feature ranges of different types of pedestrians and the corresponding label information, it includes: Extract the SVM hyperplane based on the signal statistical feature ranges of different types of pedestrians and the corresponding label information; and Obtain the SVM classification decision function through the SVM hyperplane.
4. The warning method for disabled persons according to claim 1 or 2, characterized in that When extracting the micro-Doppler features from the pedestrian data to be detected, and the extracted features include the torso Doppler information, the total Doppler bandwidth, and the limb movement period of the pedestrian to be detected, it includes: Perform range-Doppler processing on the pedestrian data to be detected; Perform time-Doppler processing based on the results of the range-Doppler processing; and Extract the features such as the torso Doppler information, the total Doppler bandwidth, and the limb movement period of the pedestrian to be detected based on the results of the range-Doppler processing and the time-Doppler processing.
5. The warning method for disabled persons according to claim 2, wherein When extracting the signal statistical features of different types of pedestrian samples by extracting micro-Doppler features from each pedestrian sample in the pedestrian sample data set, it includes: Perform range-Doppler processing on each pedestrian sample in the pedestrian sample data set; Perform time-Doppler processing based on the results of the range-Doppler processing; and Combine the label information of each pedestrian sample, and analyze and statistically process the processed result signals of different types of pedestrian samples to obtain the signal statistical features of different types of pedestrian samples.
6. The warning method for disabled persons according to claim 1 or 2, characterized in that It further includes the following steps: After obtaining the pedestrian data to be detected, calculate the distance and speed of the pedestrian to be detected relative to the radar, and only perform subsequent processing when the distance and the speed meet the preset conditions.
7. The disabled person warning method according to claim 2, wherein The pedestrian sample data set includes normal pedestrians and disabled persons.
8. The disabled person warning method according to claim 2, wherein The torso Doppler information is the torso Doppler frequency, The statistical torso Doppler information is the statistical torso Doppler frequency.
9. The warning method for disabled persons according to claim 2, wherein the torso Doppler information is the torso Doppler bandwidth, the statistical torso Doppler information is the statistical torso Doppler bandwidth.
10. A warning device for disabled persons, characterized in that, It includes: an acquisition module, configured to acquire pedestrian data to be detected, and the pedestrian data to be detected is radar detection data; a feature extraction module, configured to perform micro-Doppler feature extraction on the pedestrian data to be detected, and the extracted features include the torso Doppler information, the total Doppler bandwidth, and the limb movement period of the pedestrian to be detected; a classification and recognition module, configured to input the extracted features into an SVM model for classification and recognition, and the SVM model is a pre-trained model; and a warning module, which issues a warning if the pedestrian to be detected is classified and recognized as a disabled person.
11. An electronic device, characterized in that, The electronic device includes a processor and a memory, and at least one instruction or at least one segment of program is stored in the memory, and the at least one instruction or the at least one segment of program is loaded and executed by the processor to perform the method according to any one of claims 1 to 9.
12. A computer storage medium, characterized in that, At least one instruction or at least one segment of program is stored in the storage medium, and the at least one instruction or the at least one segment of program is loaded and executed by a processor to perform the method according to any one of claims 1 to 9.