In-vehicle biological detection method and system based on wireless signal

Abstract

The invention provides an in-vehicle biological detection method and system based on a wireless signal. The method comprises the following steps: acquiring a radio frequency echo signal; determining position information and attitude information of each organism in the vehicle according to the radio frequency echo signals; generating a corresponding Doppler frequency shift thermodynamic diagram according to the radio frequency echo signal, each piece of position information and each piece of attitude information; the Doppler frequency shift thermodynamic diagram and the radio frequency echo signals are input into a preset reasoning model, so that the reasoning model conducts reasoning according to the Doppler frequency shift thermodynamic diagram, detection information of all organisms in the vehicle is output, and the detection information comprises classification information, behavior information, pose information and contour information of the corresponding organisms. The reasoning model is constructed based on a teacher and apprentice network and is obtained based on training of a plurality of historical image data and a plurality of historical radio frequency echo signals, and accurate in-vehicle retention organism detection is achieved.

Description

Technical Field

[0001] The present application relates to the technical field of machine learning and wireless signal processing, and in particular to a data transmission control method and system for a power system. Background Art

[0002] As the public pays more attention to vehicle safety and personal and property safety, more vehicle safety scenarios, demands, and products are being put on the market, leading to a rapid growth in automotive networking software and hardware. Sudden illness while driving, people stranded in the car, and biosensing monitoring are important safety needs for drivers, passengers, and people with children and pets, and are gradually becoming standard features of family cars. Communication, sense, and perception are integrated, that is, a system completes the tasks of communication and perception, and uses the underlying principles of communication and perception to operate in the form of radio signals. Communication transmits information by sending and receiving wireless signals between base stations and mobile phones, while perception radars send out radio signals and then detect and analyze the received reflected signals for perception. Therefore, the same wireless signal can be used to transmit information and perceive the environment.

[0003] Existing in-vehicle biological detection technologies are mainly divided into computer vision solutions, infrared sensing technologies, and wearable device positioning technologies. Computer vision recognition technology mainly uses industrial high-definition CMOS cameras to collect in-vehicle video images and then uses algorithms to identify drivers who suddenly fall down due to illness, babies and pets left in the car. Capturing and obtaining in-vehicle biological information through computer vision recognition technology requires night vision capabilities, and increasing software and hardware costs to compensate for visual defects at night and in dark light conditions. The infrared sensing solution uses infrared imaging equipment through infrared thermal imaging MEMS chips and special cobalt lenses to achieve infrared imaging and perception of stranded people in the car. The infrared solution is divided into infrared camera solutions and thermal imaging solutions. Back-end personnel identification mainly relies on visual algorithms. The seat sensor solution is affected by child seats, heavy objects, etc., with large deviations and cannot detect pets lying on the ground, so it has a small range of applications. Existing wearable device recognition technology generally deploys wearable device transmitters on in-vehicle creatures, and uses RF multi-antenna passenger cabin positioning or GPS positioning solutions for people left in the car. These technical solutions have certain technical defects to a greater or lesser extent. For example, the pure camera solution has the problem of unstable recognition rate in special scenes such as low light, dark light, bright and dark light; the equipment cost of infrared thermal imaging and wearable devices is relatively high, and they are not suitable for promotion in civilian vehicles.

[0004] With the popularization of 5G spectrum, we found that most of the communication and perception hardware based on WiFi are the same, including antennas, amplifiers, filters, etc. The reuse of traditional 5G hardware can greatly reduce hardware costs; and in terms of software, communication and perception based on WiFi have similar information processing processes, including encoding and decoding, modulation and demodulation, etc. Based on the familiarity with the hardware and software of the vehicle-mounted communication network, we study the in-vehicle biological detection method that integrates synaesthesia, which has great software and hardware advantages and research prospects. Summary of the invention

[0005] In response to the above technical problems, the present application provides an in-vehicle biological detection method and system based on wireless signals, which can achieve accurate in-vehicle retained biological detection with lower software and hardware costs.

[0006] In a first aspect, the present application provides a method for detecting biological substances in a vehicle based on wireless signals, comprising:

[0007] Acquire radio frequency echo signal;

[0008] Determine the position information and posture information of each living thing in the vehicle according to the radio frequency echo signal;

[0009] Generate a corresponding Doppler frequency shift heat map according to the radio frequency echo signal, each of the position information and each of the attitude information;

[0010] The Doppler frequency shift heat map and the radio frequency echo signal are input into a preset inference model, so that the inference model performs inference according to the Doppler frequency shift heat map, and outputs detection information of each organism in the vehicle, wherein the detection information includes classification information, behavior information, posture information, and contour information of the corresponding organism. The inference model is constructed based on a teacher-apprentice network and is obtained through training based on a number of historical image data and a number of historical radio frequency echo signals.

[0011] The embodiment of the present application provides a method for detecting biological organisms in a vehicle based on wireless signals, which determines the position information and posture information of each biological organism in the vehicle according to the received wireless signals, and further generates a corresponding Doppler frequency shift heat map. Then, the inference model is used to infer the Doppler frequency shift heat map and the radio frequency echo signal to generate the detection information of each biological organism in the vehicle. Compared with the existing computer vision scheme, infrared sensing technology and wearable device positioning technology, the method for detecting biological organisms in a vehicle based on wireless signals provided by the embodiment of the present application is based on radio frequency echo signals for biological detection, which can maintain a relatively accurate detection effect even at night, in dim light or strong light conditions. At the same time, the vehicle itself has wireless communication hardware and related sensing hardware, and does not rely on expensive infrared thermal imaging and wearable devices, which reduces the deployment cost of the embodiment of the present application. In terms of model construction, the embodiment of the present application constructs an inference model based on a teacher-apprentice network. During the training process, the characteristics of the teacher-apprentice network are used to perform joint training through image data and wireless signals. During the inference process, only wireless signals are required for inference, which not only improves the recognition accuracy of the model, but also simplifies the data required for the model in the actual application process. In summary, the in-vehicle biological detection method provided in the embodiments of the present application can achieve accurate in-vehicle retained biological detection with relatively low software and hardware costs.

[0012] Further, the obtaining of the radio frequency echo signal includes:

[0013] Sending radio frequency signals into the vehicle through a preset radio frequency antenna array;

[0014] Acquiring the radio frequency echo signal through two preset symmetrical receiving antenna arrays;

[0015] The receiving antenna array is symmetrically arranged on both sides of the radio frequency antenna array.

[0016] The embodiment of the present application is inspired by the use of matrix antennas to enhance signal reception by traditional millimeter-wave sensing radars, and sets receiving antenna groups on both sides according to the spatial position of the transmitting antenna group to form a symmetrical design. During signal processing, the frequency-modulated continuous wave signals received on both sides are mixed and superimposed using axisymmetry, which can eliminate the influence of some low-intensity echo noise on the generation of Doppler heat maps. In addition, the design position of the transmitting antenna in the embodiment of the present application is not limited to the geometric center point of the receiving antennas on both sides, and is also applicable to the symmetrical arrangement of the transmitting antenna on the perpendicular midline of the receiving antenna connection line. The antenna array of this design can obtain a planar two-dimensional Doppler image when placed horizontally, and a depth two-dimensional Doppler image when placed vertically. Combined with the switch design of the matrix antenna, multiple two-dimensional planar Doppler images can be obtained. The symmetrically designed receiving antenna can reduce the interference of the echo path and small and medium-sized radio frequency shielding objects on the wifi signal, allowing the signal processing module to output richer tensor parameter information, thereby improving the accuracy of in-vehicle biological detection.

[0017] Further, determining the position information and posture information of each living thing in the vehicle according to the radio frequency echo signal includes:

[0018] Modulating the radio frequency echo signal to obtain a corresponding modulation signal;

[0019] Based on the method of finding the quotient of symmetrical signals, the position information and posture information of each creature in the vehicle are calculated according to the modulated signal.

[0020] Based on a symmetrically designed receiving antenna array, the embodiment of the present application can calculate the position information and posture information of each organism in the vehicle by taking the quotient of symmetrical signals, thereby providing reference information for the subsequent generation of a Doppler frequency shift heat map, thereby improving the accuracy of the Doppler frequency shift heat map.

[0021] In a possible implementation, generating a corresponding Doppler frequency shift heat map according to the radio frequency echo signal, each piece of the position information, and each piece of the attitude information includes:

[0022] According to each of the position information and each of the attitude information, performing Doppler frequency shift calculation on each corresponding position in the radio frequency echo signal to generate a plurality of Doppler frequency shift values;

[0023] One-hot encoding is performed on each of the Doppler frequency shift values ​​to generate a plurality of encoded data;

[0024] Performing feature fusion on each of the coded data to generate fused coded data;

[0025] Decoding the fused coded data to generate an initial Doppler frequency shift heat map;

[0026] Gaussian blur processing is performed on the initial Doppler frequency shift heat map to generate the Doppler frequency shift heat map.

[0027] An embodiment of the present application provides a method for generating a Doppler shift heat map. Based on position information and attitude information, one-hot encoding is used to preprocess the RF echo signal at the input end to generate a number of encoded data, and then the key point position is converted into a continuous intensity distribution through Gaussian blur. A Gaussian kernel is used to generate a continuous intensity distribution around each key point position to obtain a more continuous and stable Doppler heat map, thereby realizing the conversion of wireless signals to image data and preparing data for subsequent model reasoning.

[0028] In a possible implementation, the Doppler frequency shift heat map and the radio frequency echo signal are input into a preset inference model, so that the inference model performs inference according to the Doppler frequency shift heat map and outputs detection information of each organism in the vehicle, including:

[0029] extracting two-dimensional information from the Doppler shift thermal map;

[0030] Combining the two-dimensional information with the transmission frequency of the radio frequency echo signal to construct a plurality of three-dimensional data with a time series;

[0031] Based on the time series, each of the three-dimensional data is input into the reasoning model, so that the reasoning model processes each of the three-dimensional data based on a preset multi-core convolutional network, and then outputs the detection information of each organism in the vehicle.

[0032] In an embodiment of the present application, the Doppler shift heat map is further processed before model inference, and the two-dimensional information in the Doppler shift heat map is combined with the transmission frequency of the radio frequency echo signal to construct three-dimensional data with a time series, so that the input data has a time dimension, thereby being able to better describe and identify behaviors that last for a period of time, such as the driver's sudden illness and collapse, thereby improving the recognition accuracy of the inference model and further improving the accuracy of in-vehicle biological detection.

[0033] In a possible implementation, the inference model is constructed based on a teacher-apprentice network and trained based on a number of historical image data and a number of historical radio frequency echo signals, including:

[0034] Constructing a teacher network training data set based on the plurality of historical image data;

[0035] constructing a learner network training data set according to the plurality of historical radio frequency echo signals;

[0036] According to several preset detection targets, construct several corresponding recognition convolution kernels;

[0037] Extracting position information from the plurality of historical image data and the plurality of historical radio frequency echo signal data, and constructing a posture convolution kernel according to the position information;

[0038] Combining the pose convolution kernel with each of the recognition convolution kernels, constructing an initial teacher-apprentice network, wherein the initial teacher-apprentice network includes an initial teacher network and an initial apprentice network;

[0039] Based on the ResNet framework, the teacher network training data set and the apprentice network training data set are respectively input into the initial teacher network and the initial apprentice network for training to obtain the teacher network and the apprentice network. Meanwhile, during the training process, the knowledge in the initial teacher network is transferred to the initial apprentice network based on a preset migration cross entropy loss function;

[0040] The apprentice network is used as the inference model.

[0041] The embodiment of the present application provides a method for constructing an inference model. Since there is a fundamental problem in the method of using antenna clusters and Doppler frequency shift heat maps for biometric positioning, that is, the heart and trunk of the organism are highly reflective areas when the organism is stationary, and the hands and feet of children and the head and mouth of pets become highly reflective areas when in motion, which makes the capture of biological activities of key points (such as heart and trunk) interfere and cause drift. Therefore, in addition to using radio frequency echo signals during model training, the embodiment of the present application also adds time dimension feature information in units of visual frames, combines the biological video RGB feature information (pixel width W*pixel height H*video frame Frame) calculated by the teacher network and the biological Doppler frequency shift feature information (antenna cluster 2x*frequency f*receiving round N) of the apprentice network to construct a new three-dimensional tensor method, realizes the neural network tensor parameters based on time series, to construct cross-time and space aggregation information, and improve the training effect of the model. At the same time, during the training process, the knowledge in the teacher network is transferred to the apprentice network based on the preset migration cross entropy loss function, and the apprentice network is deployed as the reasoning model, realizing the knowledge transfer in the training process, improving the recognition accuracy of the reasoning model, and reducing the required data in the reasoning process. In addition, during the model construction process, the embodiment of the present application constructs a corresponding CNN convolution kernel for each type of recognition, which can flexibly couple various requirements of the vehicle project in the later stage, and the scheme can realize multiple spatiotemporal convolutions, so it is not limited by the low precision of the machine learning scheme and can only recognize a single object. It can recognize the continuous life actions of different detected organisms (such as: the driver suddenly lying down, etc.) by using multiple groups of convolutional neural networks.

[0042] Furthermore, constructing a teacher network training data set based on the plurality of historical image data includes:

[0043] Acquire a plurality of frames of image data based on a preset timing cycle through a preset camera in the vehicle, and construct and obtain the plurality of historical image data;

[0044] extracting a plurality of two-dimensional information from the plurality of historical image data;

[0045] The plurality of two-dimensional information are combined with the frame rate per second information of the camera to construct a teacher network training data set with a time series.

[0046] In an embodiment of the present application, similar to the processing method of the Doppler frequency shift heat map, the two-dimensional information of the image data is combined with the frame rate information per second to construct a teacher network training data set with a time series. This processing method aligns the data dimensions of the teacher network and the apprentice network, so that both the teacher network and the apprentice network are trained based on three-dimensional training data, thereby improving the training effect of the model.

[0047] Furthermore, the radio frequency echo signal is a Wifi signal.

[0048] In a possible implementation, the in-vehicle biological detection method further includes:

[0049] Obtain real-time vehicle operation information;

[0050] A safety check of several preset items is performed based on the detection information and the real-time operation information of the vehicle, and a safety warning is issued for each of the preset items based on the detection results.

[0051] In the embodiment of the present application, the detection information and the real-time operation information of the vehicle are combined to realize the vehicle's inter-sensory integration, and the communication and perception tasks are completed through a set of systems, thereby realizing real-time safety detection and safety warning, and providing data support for the subsequent vehicle intelligence.

[0052] In a second aspect, accordingly, an embodiment of the present application provides an in-vehicle biological detection system based on wireless signals, including an acquisition module, a posture information determination module, an image generation module, and an inference module;

[0053] Wherein, the acquisition module is used to acquire the radio frequency echo signal;

[0054] The position and posture information determination module is used to determine the position information and posture information of each living thing in the vehicle according to the radio frequency echo signal;

[0055] The image generation module is used to generate a corresponding Doppler frequency shift heat map according to the radio frequency echo signal, each position information and each posture information;

[0056] The inference module is used to input the Doppler frequency shift heat map and the radio frequency echo signal into a preset inference model, so that the inference model performs inference according to the Doppler frequency shift heat map and outputs detection information of each organism in the vehicle, wherein the detection information includes classification information, behavior information, posture information, and contour information of the corresponding organism. The inference model is constructed based on a teacher-apprentice network and is obtained through training based on a number of historical image data and a number of historical radio frequency echo signals. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 : A flow chart of an in-vehicle biological detection method based on wireless signals provided in an embodiment of the present application.

[0058] Figure 2 : A schematic diagram of the process of generating a Doppler frequency shift heat map in a method for in-vehicle biological detection based on wireless signals provided in an embodiment of the present application.

[0059] Figure 3: A schematic diagram of the training process of a teacher-apprentice network in an in-vehicle biological detection method based on wireless signals provided in an embodiment of the present application.

[0060] Figure 4 : A schematic diagram of the training process of an inference model in an in-vehicle biological detection method based on wireless signals provided in an embodiment of the present application.

[0061] Figure 5 : A schematic diagram of the integrated hardware and software of a wireless signal-based in-vehicle biological detection method provided in an embodiment of the present application.

[0062] Figure 6 : A structural diagram of an in-vehicle biological detection system based on wireless signals provided in an embodiment of the present application. DETAILED DESCRIPTION

[0063] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0064] It should be noted that the step numbers in the text are only for the convenience of explanation of the specific embodiments and do not serve to limit the order in which the steps are executed. In the description of this application, the terms "first" and "second" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features.

[0065] Embodiment 1:

[0066] like Figure 1 As shown, the first embodiment provides a method for detecting biological organisms in a vehicle based on wireless signals, including steps S1-S4:

[0067] Step S1, obtaining a radio frequency echo signal;

[0068] Step S2, determining the position information and posture information of each living thing in the vehicle according to the radio frequency echo signal;

[0069] Step S3, generating a corresponding Doppler frequency shift heat map according to the radio frequency echo signal, each piece of position information and each piece of attitude information;

[0070] Step S4, inputting the Doppler frequency shift heat map and the radio frequency echo signal into a preset inference model, so that the inference model performs inference according to the Doppler frequency shift heat map, and outputs detection information of each organism in the vehicle, wherein the detection information includes classification information, behavior information, posture information, and contour information of the corresponding organism, and the inference model is constructed based on a teacher-apprentice network and obtained through training based on a number of historical image data and a number of historical radio frequency echo signals.

[0071] The embodiment of the present application provides a method for detecting biological organisms in a vehicle based on wireless signals, which determines the position information and posture information of each biological organism in the vehicle according to the received wireless signals, and further generates a corresponding Doppler frequency shift heat map. Then, the inference model is used to infer the Doppler frequency shift heat map and the radio frequency echo signal to generate the detection information of each biological organism in the vehicle. Compared with the existing computer vision scheme, infrared sensing technology and wearable device positioning technology, the method for detecting biological organisms in a vehicle based on wireless signals provided by the embodiment of the present application is based on radio frequency echo signals for biological detection, which can maintain a relatively accurate detection effect even at night, in dim light or strong light conditions. At the same time, the vehicle itself has wireless communication hardware and related sensing hardware, and does not rely on expensive infrared thermal imaging and wearable devices, which reduces the deployment cost of the embodiment of the present application. In terms of model construction, the embodiment of the present application constructs an inference model based on a teacher-apprentice network. During the training process, the characteristics of the teacher-apprentice network are used to perform joint training through image data and wireless signals. During the inference process, only wireless signals are required for inference, which not only improves the recognition accuracy of the model, but also simplifies the data required for the model in the actual application process. In summary, the in-vehicle biological detection method provided in the embodiments of the present application can achieve accurate in-vehicle retained biological detection with relatively low software and hardware costs.

[0072] Further, in step S1, the obtaining of the radio frequency echo signal includes:

[0073] Sending radio frequency signals into the vehicle through a preset radio frequency antenna array;

[0074] Acquiring the radio frequency echo signal through two preset symmetrical receiving antenna arrays;

[0075] The receiving antenna array is symmetrically arranged on both sides of the radio frequency antenna array.

[0076] The embodiment of the present application is inspired by the use of matrix antennas to enhance signal reception by traditional millimeter-wave sensing radars, and sets receiving antenna groups on both sides according to the spatial position of the transmitting antenna group to form a symmetrical design. During signal processing, the frequency-modulated continuous wave signals received on both sides are mixed and superimposed using axisymmetry, which can eliminate the influence of some low-intensity echo noise on the generation of Doppler heat maps. In addition, the design position of the transmitting antenna in the embodiment of the present application is not limited to the geometric center point of the receiving antennas on both sides, and is also applicable to the symmetrical arrangement of the transmitting antenna on the perpendicular midline of the receiving antenna connection line. The antenna array of this design can obtain a planar two-dimensional Doppler image when placed horizontally, and a depth two-dimensional Doppler image when placed vertically. Combined with the switch design of the matrix antenna, multiple two-dimensional planar Doppler images can be obtained. The symmetrically designed receiving antenna can reduce the interference of the echo path and small and medium-sized radio frequency shielding objects on the wifi signal, allowing the signal processing module to output richer tensor parameter information, thereby improving the accuracy of in-vehicle biological detection.

[0077] In a preferred embodiment, a matrix antenna is designed using a 1 to 2x ratio of transmission and reception, and better signal sampling clarity can be obtained in a two-dimensional plane. At the same time, the symmetrical design of the antenna array of this embodiment can share a group of transmitting antenna families in the horizontal and vertical layout schemes, so that the number of transmitting antennas and the layout of the matrix antenna can be optimized to the greatest extent. In addition, in order to solve the difficulty that the traditional WiFi radar can only obtain a single spatial position receiving signal when transmitting a frequency-modulated continuous wave, this embodiment uses a combination of SP4T switch and WiFi sensing control module to control the antenna array. This solution is different from the shortcomings of the traditional array antenna group in cost and accuracy, and accurately realizes the spatial separability of WiFi signals, so that the system can obtain the reflected transmission signal information of multiple spatial azimuths. Through the control algorithm, the precise switching of the antenna group and the receiving and transmitting frequency is realized, and the corresponding relationship between the WiFi transmission frequency and the receiving and transmitting antenna group is freely controlled, so that the reflected transmission signal information of multiple spatial azimuths under the corresponding frequency is projected onto the vertical plane to construct a complete visual-like two-dimensional picture. This method solves the problem that traditional RF (such as millimeter wave) signals can only obtain single-dimensional Doppler shift information through FMCW, which is insufficient to meet the needs. The collected echo signal contains sufficient information (such as antenna cluster 2x*frequency f) and can be used together with the two-dimensional tensor information collected by the camera for input into subsequent deconvolutional neural network calculations.

[0078] Furthermore, in step S2, determining the position information and posture information of each living thing in the vehicle according to the radio frequency echo signal includes:

[0079] Modulating the radio frequency echo signal to obtain a corresponding modulation signal;

[0080] Based on the method of finding the quotient of symmetrical signals, the position information and posture information of each creature in the vehicle are calculated according to the modulated signal.

[0081] Based on a symmetrically designed receiving antenna array, the embodiment of the present application can calculate the position information and posture information of each organism in the vehicle by taking the quotient of symmetrical signals, thereby providing reference information for the subsequent generation of a Doppler frequency shift heat map, thereby improving the accuracy of the Doppler frequency shift heat map.

[0082] In one possible implementation, in step S3, as Figure 2 As shown, the generating of the corresponding Doppler frequency shift heat map according to the radio frequency echo signal, each of the position information and each of the attitude information includes:

[0083] According to each of the position information and each of the attitude information, performing Doppler frequency shift calculation on each corresponding position in the radio frequency echo signal to generate a plurality of Doppler frequency shift values;

[0084] One-hot encoding is performed on each of the Doppler frequency shift values ​​to generate a plurality of encoded data;

[0085] Performing feature fusion on each of the coded data to generate fused coded data;

[0086] Decoding the fused coded data to generate an initial Doppler frequency shift heat map;

[0087] Gaussian blur processing is performed on the initial Doppler frequency shift heat map to generate the Doppler frequency shift heat map.

[0088] An embodiment of the present application provides a method for generating a Doppler shift heat map. Based on position information and attitude information, one-hot encoding is used to preprocess the RF echo signal at the input end to generate a number of encoded data, and then the key point position is converted into a continuous intensity distribution through Gaussian blur. A Gaussian kernel is used to generate a continuous intensity distribution around each key point position to obtain a more continuous and stable Doppler heat map, thereby realizing the conversion of wireless signals to image data and preparing data for subsequent model reasoning.

[0089] In a possible implementation, in step S4, the Doppler frequency shift heat map and the radio frequency echo signal are input into a preset inference model, so that the inference model performs inference according to the Doppler frequency shift heat map and outputs detection information of each organism in the vehicle, including:

[0090] extracting two-dimensional information from the Doppler shift thermal map;

[0091] Combining the two-dimensional information with the transmission frequency of the radio frequency echo signal to construct a plurality of three-dimensional data with a time series;

[0092] Based on the time series, each of the three-dimensional data is input into the reasoning model, so that the reasoning model processes each of the three-dimensional data based on a preset multi-core convolutional network, and then outputs the detection information of each organism in the vehicle.

[0093] In an embodiment of the present application, the Doppler shift heat map is further processed before model inference, and the two-dimensional information in the Doppler shift heat map is combined with the transmission frequency of the radio frequency echo signal to construct three-dimensional data with a time series, so that the input data has a time dimension, thereby being able to better describe and identify behaviors that last for a period of time, such as the driver's sudden illness and collapse, thereby improving the recognition accuracy of the inference model and further improving the accuracy of in-vehicle biological detection.

[0094] In a possible implementation, the inference model is constructed based on a teacher-apprentice network and trained based on a number of historical image data and a number of historical radio frequency echo signals, including:

[0095] Constructing a teacher network training data set based on the plurality of historical image data;

[0096] constructing a learner network training data set according to the plurality of historical radio frequency echo signals;

[0097] According to several preset detection targets, construct several corresponding recognition convolution kernels;

[0098] Extracting position information from the plurality of historical image data and the plurality of historical radio frequency echo signal data, and constructing a posture convolution kernel according to the position information;

[0099] Combining the pose convolution kernel with each of the recognition convolution kernels, constructing an initial teacher-apprentice network, wherein the initial teacher-apprentice network includes an initial teacher network and an initial apprentice network;

[0100] Based on the ResNet framework, the teacher network training data set and the apprentice network training data set are respectively input into the initial teacher network and the initial apprentice network for training to obtain the teacher network and the apprentice network. Meanwhile, during the training process, the knowledge in the initial teacher network is transferred to the initial apprentice network based on a preset migration cross entropy loss function;

[0101] The apprentice network is used as the inference model.

[0102] The embodiment of the present application provides a method for constructing an inference model. Since there is a fundamental problem in the method of using antenna clusters and Doppler frequency shift heat maps for biometric positioning, that is, the heart and trunk of the organism are highly reflective areas when the organism is stationary, and the hands and feet of children and the head and mouth of pets become highly reflective areas when in motion, which makes the capture of biological activities of key points (such as heart and trunk) interfere and cause drift. Therefore, in addition to using radio frequency echo signals during model training, the embodiment of the present application also adds time dimension feature information in units of visual frames, combines the biological video RGB feature information (pixel width W*pixel height H*video frame Frame) calculated by the teacher network and the biological Doppler frequency shift feature information (antenna cluster 2x*frequency f*receiving round N) of the apprentice network to construct a new three-dimensional tensor method, realizes the neural network tensor parameters based on time series, to construct cross-time and space aggregation information, and improve the training effect of the model. At the same time, during the training process, the knowledge in the teacher network is transferred to the apprentice network based on the preset migration cross entropy loss function, and the apprentice network is deployed as the inference model, thereby realizing knowledge transfer in the training process, improving the recognition accuracy of the inference model, and reducing the required data in the inference process.

[0103] The embodiment of the present application proposes to construct a spatiotemporal convolution model (such as ultra-small target models such as a driver lying down, a baby sitting, and a dog lying prone) by combining multiple groups of convolutional neural networks. Compared with the KNN machine learning scheme proposed by other research groups in their perception schemes, the difference between the multiple groups of CNN combined scheme in this embodiment is that a CNN convolutional network is added for each type of recognition, which can flexibly couple various requirements of the whole vehicle project in the later stage, and the scheme can realize multiple spatiotemporal convolutions, so it is not limited by the low accuracy of the machine learning scheme and can only recognize a single object; compared with traditional LSTM, XGboost and other time-related machine learning models, which need to be retrained after the vehicle cabin environment and the object being tested change (which is difficult to implement after the vehicle is sold), the convolutional neural network has higher adaptability and availability. The combination of multiple groups of convolutional neural networks can identify the continuous life actions of different detected organisms (such as: the driver suddenly lying down, etc.).

[0104] In a preferred embodiment, based on the residual mapping F(x)=H(x)-x in Res-Net, a CNN convolutional network is designed as the overall architecture to solve the gradient vanishing problem of multi-layer CNN joint in the back-propagation gradient training process. Different from the traditional scheme that usually uses multiple groups of prepared samples for training sets and validation sets for training, this embodiment designs an improved Teacher-Student Net based on the overall architecture of Res-Net, using the camera frame spatiotemporal information as the teacher network input, and the wifi spatiotemporal Doppler heat map as the apprentice network input. This embodiment uses an end-to-end training improvement model to minimize a loss function to capture the difference between the network's probability prediction and obtaining correct continuous labels (such as babies and pet dogs), such as Figure 3 As shown. This embodiment proposes a time-series-based network, which specifically targets the disadvantage of understanding the correlation between actions and signals in CNN networks in processing RF radio frequency SDR (software defined radio) signal analysis. Several frames of video information in a time sequence period are combined with corresponding multiple sets of radio signal sets to form a tensor parameter, which can capture behavioral actions (such as: the driver suddenly falls down due to illness, etc.) and classify the target's action behaviors, such as recovering to stand up normally after lying down, slowly lying prone, and other complex actions. The classifier proposed in this embodiment, combined with a high real-time antenna switch control module and an antenna cluster hardware structure, can achieve good generalization effects in small and medium training sets. The experimental results in scenarios such as seat occlusion and vehicle screen occlusion show that the combination of deep learning and return signal acquisition hardware solutions has good detection accuracy for multipath reflection interference, including completely occluded target biological information and motion status, and balances the relationship between the training sample size and accuracy of each target scene.

[0105] Furthermore, constructing a teacher network training data set based on the plurality of historical image data includes:

[0106] Acquire a plurality of frames of image data based on a preset timing cycle through a preset camera in the vehicle, and construct and obtain the plurality of historical image data;

[0107] extracting a plurality of two-dimensional information from the plurality of historical image data;

[0108] The plurality of two-dimensional information are combined with the frame rate per second information of the camera to construct a teacher network training data set with a time series.

[0109] In an embodiment of the present application, similar to the processing method of the Doppler frequency shift heat map, the two-dimensional information of the image data is combined with the frame rate information per second to construct a teacher network training data set with a time series. This processing method aligns the data dimensions of the teacher network and the apprentice network, so that both the teacher network and the apprentice network are trained based on three-dimensional training data, thereby improving the training effect of the model.

[0110] Furthermore, the radio frequency echo signal is a Wifi signal.

[0111] The popularity of 5G-wifi has made the civilian wifi frequency band take another step towards the millimeter wave frequency band. The 5.xGHz wifi802.11b / g / n / ax signal has both high-frequency reflectivity and certain obstacle bypassing capabilities. Thanks to the development of wireless technology, the support and application of CSI in the Wifi6 standard are more extensive and in-depth. Wifi signals carry more information for software-defined radio SDR deep learning (Deeplearning) signal processing. Provide more possibilities. Therefore, the embodiment of the present application performs in-cabin biosensing by collecting and analyzing the end-to-end model of Wifi echo signals, without changing the normal wifi communication frequency and hardware framework in the car, reducing the deployment cost of the method proposed in this application.

[0112] In a preferred embodiment, the training process for the teacher network and the apprentice network is as follows: Figure 4 As shown, it includes the following three steps:

[0113] Step 1: Dual-end data collection of vision and radio frequency

[0114] Multiple cameras are used to collect visual signals inside the vehicle, while the RF antenna array collects WIFI echo signals;

[0115] Step 2: Visual signal processing and RF signal processing

[0116] The signals collected by the camera use, including but not limited to (CV computer vision recognition technology such as Mask-RCNN) to obtain the biological information in the car (including biological classification, position, posture, contour, etc.). The MCU modulates the feedback signal from the receiving antenna to the radio frequency sensing service unit, and obtains the position and posture information by taking the quotient of the symmetric signal. The vehicle-mounted sensing unit then converts the one-hot encoding into a Doppler frequency shift heat map. This step is mainly to align the data dimensions of the teacher network and the apprentice network and complete the basic signal processing.

[0117] Step 3: Put the data into Teacher-Student Net to complete the training

[0118] a) The two-dimensional information of the camera image (x1, y1 axis) plus the three-dimensional data of the frame rate per second (t1) is used as the teacher network, and the two-dimensional information of the Doppler map (x2, y2 axis) plus the three-dimensional data of one-f of the WIFI transmission frequency (1 / f) is used as the apprentice network to form a CNN convolution kernel for biological classification and number recognition;

[0119] b) In order to use the same method to complete the tasks of detecting the driver's sudden illness and lying down and identifying different types of pets (such as babies and different types of pets (such as cats and dogs), multiple convolution kernels are used here, that is, the method in "Step 3 a)" is used to establish separate CNN recognition and judgment convolution kernels for different target recognition scenarios (including: actions, people, pets and other heterogeneous animals);

[0120] c) Taking the quotient of the symmetric signal completed by the perception service unit in step 2 to obtain the position, and combining it with the position information obtained by the multi-camera in the teacher network through the visual positioning method (including but not limited to CameraTrace, etc.), to form another CNN pose convolution kernel for center positioning and contour positioning;

[0121] d) The multi-convolution kernels of the teacher and apprentice networks complete learning and data pooling under the ResNet framework;

[0122] e) After learning converges, the transfer cross entropy loss of the apprentice network to the teacher network is obtained, that is, the end-to-end inference model from RF input to classification and pose is obtained.

[0123] Furthermore, in the application process of the end-to-end reasoning model, the reasoning perception hardware part has not changed much compared with the training hardware. The main reduction is the Teacher Net hardware of multi-modal input such as video signal acquisition (such as Figure 4 As shown on the right, the vehicle bus signal interface is added, which is mainly used to collect vehicle control related data. The software adds functions such as bus signal processing, event arbitration and space fence, completes the system startup and exit mechanism control, supervision of privacy regulations, reduces interference from external targets and waste of computing resources, etc., so that the solution can be applied to the calibration of different space size models in the later stage.

[0124] Furthermore, in view of the high hardware requirements of the deep learning reasoning process of the convolutional neural network, this embodiment proposes a two-end optimization method: the first input end includes using one-hot encoding in the end-to-end model to pre-process the input Wifi return signal (including but not limited to the amplitude and frequency in CSI), converting the key point position into a continuous intensity distribution through Gaussian blur, using a Gaussian kernel to generate a continuous intensity distribution around each key point position to obtain a more continuous and stable Doppler heat map, and inputting the Teacher-Student Net model to calculate the migration loss function to obtain the end-to-end model. Directly using the key point information of the wifi signal as input during reasoning can reduce the required tensor information; the second output end includes flexibly cutting the number of ResNet groups according to different vehicle configurations and design specification requirements, and combining the use of low-precision quantization models to reduce the model size, reduce memory usage and increase the reasoning speed, while trying to keep the performance of the model to meet the design verification requirements. This enables the vehicle-side embedded device to run reasoning tasks quickly without affecting communication transactions.

[0125] In a possible implementation, the in-vehicle biological detection method further includes:

[0126] Obtain real-time vehicle operation information;

[0127] A safety check of several preset items is performed based on the detection information and the real-time operation information of the vehicle, and a safety warning is issued for each of the preset items based on the detection results.

[0128] In the embodiment of the present application, the detection information and the real-time operation information of the vehicle are combined to realize the vehicle's inter-sensory integration, and the communication and perception tasks are completed through a set of systems, thereby realizing real-time safety detection and safety warning, and providing data support for the subsequent vehicle intelligence.

[0129] In a preferred embodiment, Figure 5As shown, based on a kind of in-vehicle biological detection method based on wireless signals provided by the embodiment of the present application, the mode in which the traditional communication function and perception function of the vehicle are independently operated is converted into a synesthesia integrated mode, and the highly integrated design of the synesthesia integrated module is realized under the condition of relatively small vehicle-side hardware changes and cost investment. Compared with the traditional visual perception scheme and radar perception scheme, the utilization rate of the in-vehicle space is improved, and the obstacle occlusion, multipath interference and wiring problems are eliminated. In the actual application process, the perception service unit of the vehicle receives the detection information generated by the vehicle-mounted perception module, and combines the vehicle information including but not limited to the vehicle driving state, speed, gear position, seat sensor, door lock, etc. received by the vehicle-mounted communication module (such as from CAN, Ethernet), completes the preset safety warning items, including but not limited to: seat belt tightening reminder (for driver lodging), mobile phone reminder (pet left behind), vehicle external warning sound, same side window opening ventilation seam (owner or key near the car), etc., and completes the relevant command sending through the vehicle-mounted communication module.

[0130] In summary, the first embodiment has the following beneficial effects:

[0131] 1. Use embedded devices to realize the perception function of civilian standard wifi on organisms, making vehicle-mounted synaesthesia possible. Shared hardware and lower transmission power than civilian wifi reduce the impact of radiation emission on passengers. The synaesthesia design reduces the overall hardware equipment cost, saving 10,000 yuan per vehicle compared to the cost of traditional high-definition cameras plus visual algorithm terminals;

[0132] 2. This embodiment proposes to adopt a sampling method of reflected WiFi signals, so that the transceiver antenna array is arranged near the WiFi wireless terminal. Compared with the front and rear discrete transceiver antenna solution, it saves 50% of space and wiring costs about 5,000 yuan, and improves the installation and deployment compatibility of this method in different vehicle models and different installation locations;

[0133] 3. In this embodiment, a horizontally symmetrical receiving antenna family structure design is adopted, combined with a symmetrically controlled antenna switch technology architecture, which can play a role in canceling and weakening clutter reflections, so that the WiFi Doppler signal processing noise reduction is effectively improved;

[0134] 4. The symmetrically controlled antenna switch technology architecture and the multiplexing method of the MCU (including but not limited to MCU, FPGA, etc.) acquisition interface can reduce the overall transceiver system cost while meeting the sampling accuracy requirements of the smaller space in the cabin, compared with the traditional multi-antenna matrix solution;

[0135] 5. In the process of model training, this embodiment proposes an improved teacher-apprentice network model to improve the misidentification problem of reflected virtual images caused by multipath effects in traditional RF solutions, greatly optimize the complexity of chip selection, and save hardware costs;

[0136] 6. This embodiment proposes receiving and transmitting return signals (including but not limited to CSI information) via WiFi, and simplifies the end-to-end model training of generating Doppler heat maps that are directly available to the training end through one-hot encoding, thereby optimizing the computing power requirements of the real vehicle reasoning model.

[0137] 7. This embodiment proposes a time-series-related ResNet framework Teacher-Student Net end-to-end model to eliminate the problem of insufficient accuracy of traditional machine learning models in capturing human micro-motion information such as heartbeats, and improves the perception accuracy of small spaces in the passenger cabin to centimeter level;

[0138] 8. This embodiment proposes a highly integrated hardware and software design solution through synaesthesia, which improves the utilization rate of the space inside the vehicle and eliminates the problems of obstacle occlusion and wiring in traditional visual perception solutions.

[0139] Embodiment 2:

[0140] like Figure 6 As shown, embodiment 2 provides an in-vehicle biological detection system based on wireless signals, including an acquisition module 10, a posture information determination module 20, an image generation module 30 and an inference module 40;

[0141] Wherein, the acquisition module 10 is used to acquire the radio frequency echo signal;

[0142] The position and posture information determination module 20 is used to determine the position information and posture information of each living thing in the vehicle according to the radio frequency echo signal;

[0143] The image generation module 30 is used to generate a corresponding Doppler frequency shift heat map according to the radio frequency echo signal, each position information and each posture information;

[0144] The inference module 40 is used to input the Doppler frequency shift heat map and the radio frequency echo signal into a preset inference model, so that the inference model performs inference according to the Doppler frequency shift heat map and outputs detection information of each organism in the vehicle, wherein the detection information includes classification information, behavior information, posture information, and contour information of the corresponding organism. The inference model is constructed based on a teacher-apprentice network and is obtained through training based on a number of historical image data and a number of historical radio frequency echo signals.

[0145] Furthermore, the acquisition module 10 acquires the radio frequency echo signal, including:

[0146] Sending radio frequency signals into the vehicle through a preset radio frequency antenna array;

[0147] Acquiring the radio frequency echo signal through two preset symmetrical receiving antenna arrays;

[0148] The receiving antenna array is symmetrically arranged on both sides of the radio frequency antenna array.

[0149] Furthermore, the position information determination module 20 determines the position information and posture information of each living thing in the vehicle according to the radio frequency echo signal, including:

[0150] Modulating the radio frequency echo signal to obtain a corresponding modulation signal;

[0151] Based on the method of finding the quotient of symmetrical signals, the position information and posture information of each creature in the vehicle are calculated according to the modulated signal.

[0152] In a possible implementation, the image generation module 30 generates a corresponding Doppler frequency shift heat map according to the radio frequency echo signal, each piece of position information and each piece of attitude information, including:

[0153] According to each of the position information and each of the attitude information, performing Doppler frequency shift calculation on each corresponding position in the radio frequency echo signal to generate a plurality of Doppler frequency shift values;

[0154] One-hot encoding is performed on each of the Doppler frequency shift values ​​to generate a plurality of encoded data;

[0155] Performing feature fusion on each of the coded data to generate fused coded data;

[0156] Decoding the fused coded data to generate an initial Doppler frequency shift heat map;

[0157] Gaussian blur processing is performed on the initial Doppler frequency shift heat map to generate the Doppler frequency shift heat map.

[0158] In a possible implementation, the inference module 40 inputs the Doppler frequency shift heat map and the radio frequency echo signal into a preset inference model, so that the inference model performs inference according to the Doppler frequency shift heat map and outputs detection information of each organism in the vehicle, including:

[0159] extracting two-dimensional information from the Doppler shift thermal map;

[0160] Combining the two-dimensional information with the transmission frequency of the radio frequency echo signal to construct a plurality of three-dimensional data with a time series;

[0161] Based on the time series, each of the three-dimensional data is input into the reasoning model, so that the reasoning model processes each of the three-dimensional data based on a preset multi-core convolutional network, and then outputs the detection information of each organism in the vehicle.

[0162] In a possible implementation, the in-vehicle biological detection system further includes a model construction module, which is used to construct the inference model based on a teacher-apprentice network and train based on a number of historical image data and a number of historical radio frequency echo signals, and includes a first training data construction unit, a second training data construction unit, a first convolution kernel construction unit, a second convolution kernel construction unit, an initial model construction unit, a model training unit, and an inference model acquisition unit;

[0163] Wherein, the first training data construction unit is used to construct a teacher network training data set according to the plurality of historical image data;

[0164] The second training data construction unit is used to construct an apprentice network training data set according to the plurality of historical radio frequency echo signals;

[0165] The first convolution kernel construction unit is used to construct a number of corresponding recognition convolution kernels according to a number of preset detection targets;

[0166] The second convolution kernel construction unit is used to extract position information from the plurality of historical image data and the plurality of historical radio frequency echo signal data, and construct a posture convolution kernel according to the position information;

[0167] The initial model construction unit is used to combine the posture convolution kernel and each of the recognition convolution kernels to construct an initial teacher-apprentice network, wherein the initial teacher-apprentice network includes an initial teacher network and an initial apprentice network;

[0168] The model training unit is used to use the teacher network training data set and the apprentice network training data set to input into the initial teacher network and the initial apprentice network for training based on the ResNet framework, so as to obtain the teacher network and the apprentice network. Meanwhile, during the training process, the knowledge in the initial teacher network is transferred to the initial apprentice network based on a preset migration cross entropy loss function;

[0169] The reasoning model acquisition unit is used to use the apprentice network as the reasoning model.

[0170] Furthermore, the first training data construction unit constructs a teacher network training data set according to the plurality of historical image data, including:

[0171] Acquire a plurality of frames of image data based on a preset timing cycle through a preset camera in the vehicle, and construct and obtain the plurality of historical image data;

[0172] extracting a plurality of two-dimensional information from the plurality of historical image data;

[0173] The plurality of two-dimensional information are combined with the frame rate per second information of the camera to construct a teacher network training data set with a time series.

[0174] Furthermore, the radio frequency echo signal is a Wifi signal.

[0175] Furthermore, the in-vehicle biological detection system further includes an early warning module, and the early warning module includes an acquisition unit and an early warning unit;

[0176] Wherein, the acquisition unit is used to acquire real-time vehicle operation information;

[0177] The early warning unit is used to perform safety detection of several preset items according to the detection information and the real-time operation information of the vehicle, and to issue a safety warning for each of the preset items according to the detection results.

[0178] The embodiment of the present application provides an in-vehicle biological detection system based on wireless signals, which determines the position information and posture information of each organism in the vehicle according to the received wireless signals, and further generates the corresponding Doppler frequency shift heat map. Then, the inference model is used to infer the Doppler frequency shift heat map and the radio frequency echo signal to generate the detection information of each organism in the vehicle. Compared with the existing computer vision scheme, infrared sensing technology and wearable device positioning technology, the in-vehicle biological detection method based on wireless signals provided by the embodiment of the present application is based on radio frequency echo signals for biological detection, which can maintain a relatively accurate detection effect even at night, in dim light or strong light conditions. At the same time, the vehicle itself has wireless communication hardware and related sensing hardware, and does not rely on expensive infrared thermal imaging and wearable devices, which reduces the deployment cost of the embodiment of the present application. In terms of model construction, the embodiment of the present application constructs an inference model based on a teacher-apprentice network. During the training process, the characteristics of the teacher-apprentice network are used to conduct joint training through image data and wireless signals. During the inference process, only wireless signals are required for inference, which not only improves the recognition accuracy of the model, but also simplifies the data required for the model in the actual application process. In summary, the in-vehicle biological detection method provided in the embodiments of the present application can achieve accurate in-vehicle retained biological detection with relatively low software and hardware costs.

[0179] The more detailed working principle and step flow of this embodiment can refer to, but are not limited to, the relevant records of Embodiment 1.

[0180] The specific embodiments described above further describe the purpose, technical solutions and beneficial effects of the present application in detail. It should be understood that the above description is only a specific embodiment of the present application and is not intended to limit the scope of protection of the present application. It is particularly pointed out that for those skilled in the art, any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of protection of the present application.

Claims

1. A method for detecting biological organisms in a vehicle based on wireless signals, characterized in that: include: Acquire radio frequency echo signal; Determine the position information and posture information of each living thing in the vehicle according to the radio frequency echo signal; Generate a corresponding Doppler frequency shift heat map according to the radio frequency echo signal, each of the position information and each of the attitude information; The Doppler frequency shift heat map and the radio frequency echo signal are input into a preset inference model, so that the inference model performs inference according to the Doppler frequency shift heat map, and outputs detection information of each organism in the vehicle, wherein the detection information includes classification information, behavior information, posture information, and contour information of the corresponding organism. The inference model is constructed based on a teacher-apprentice network and is obtained through training based on a number of historical image data and a number of historical radio frequency echo signals.

2. The in-vehicle biological detection method based on wireless signals as claimed in claim 1, characterized in that: The step of acquiring a radio frequency echo signal comprises: Sending radio frequency signals into the vehicle through a preset radio frequency antenna array; Acquiring the radio frequency echo signal through two preset symmetrical receiving antenna arrays; The receiving antenna array is symmetrically arranged on both sides of the radio frequency antenna array.

3. The in-vehicle biological detection method based on wireless signals as claimed in claim 2, characterized in that: The determining of the position information and posture information of each living thing in the vehicle according to the radio frequency echo signal includes: Modulating the radio frequency echo signal to obtain a corresponding modulation signal; Based on the method of finding the quotient of symmetrical signals, the position information and posture information of each creature in the vehicle are calculated according to the modulated signal.

4. The in-vehicle biological detection method based on wireless signals as claimed in claim 1, characterized in that: The generating a corresponding Doppler frequency shift heat map according to the radio frequency echo signal, each piece of the position information and each piece of the attitude information comprises: According to each of the position information and each of the attitude information, performing Doppler frequency shift calculation on each corresponding position in the radio frequency echo signal to generate a plurality of Doppler frequency shift values; One-hot encoding is performed on each of the Doppler frequency shift values ​​to generate a plurality of encoded data; Performing feature fusion on each of the coded data to generate fused coded data; Decoding the fused coded data to generate an initial Doppler frequency shift heat map; Gaussian blur processing is performed on the initial Doppler frequency shift heat map to generate the Doppler frequency shift heat map.

5. The in-vehicle biological detection method based on wireless signals as claimed in claim 1, characterized in that: The step of inputting the Doppler frequency shift heat map and the radio frequency echo signal into a preset inference model so that the inference model performs inference according to the Doppler frequency shift heat map and outputs detection information of each organism in the vehicle, including: extracting two-dimensional information from the Doppler shift thermal map; Combining the two-dimensional information with the transmission frequency of the radio frequency echo signal to construct a plurality of three-dimensional data with a time series; Based on the time series, each of the three-dimensional data is input into the reasoning model, so that the reasoning model processes each of the three-dimensional data based on a preset multi-core convolutional network, and then outputs the detection information of each organism in the vehicle.

6. The in-vehicle biological detection method based on wireless signals as claimed in claim 1, characterized in that: The inference model is constructed based on the teacher-apprentice network and trained based on a number of historical image data and a number of historical radio frequency echo signals, including: Constructing a teacher network training data set based on the plurality of historical image data; constructing a learner network training data set according to the plurality of historical radio frequency echo signals; According to several preset detection targets, construct several corresponding recognition convolution kernels; Extracting position information from the plurality of historical image data and the plurality of historical radio frequency echo signal data, and constructing a posture convolution kernel according to the position information; Combining the pose convolution kernel with each of the recognition convolution kernels, constructing an initial teacher-apprentice network, wherein the initial teacher-apprentice network includes an initial teacher network and an initial apprentice network; Based on the ResNet framework, the teacher network training data set and the apprentice network training data set are respectively input into the initial teacher network and the initial apprentice network for training to obtain the teacher network and the apprentice network. Meanwhile, during the training process, the knowledge in the initial teacher network is transferred to the initial apprentice network based on a preset migration cross entropy loss function; The apprentice network is used as the inference model.

7. The method for detecting biological substances in a vehicle based on wireless signals as claimed in claim 6, characterized in that: The step of constructing a teacher network training data set according to the plurality of historical image data comprises: Acquire a plurality of frames of image data based on a preset timing cycle through a preset camera in the vehicle, and construct and obtain the plurality of historical image data; extracting a plurality of two-dimensional information from the plurality of historical image data; The plurality of two-dimensional information are combined with the frame rate per second information of the camera to construct a teacher network training data set with a time series.

8. A method for detecting biological substances in a vehicle based on wireless signals as claimed in any one of claims 1 to 7, characterized in that: The radio frequency echo signal is a Wifi signal.

9. A method for detecting biological substances in a vehicle based on wireless signals as claimed in any one of claims 1 to 7, characterized in that: The in-vehicle biological detection method further includes: Obtain real-time vehicle operation information; A safety check of several preset items is performed based on the detection information and the real-time operation information of the vehicle, and a safety warning is issued for each of the preset items based on the detection results.

10. A wireless signal-based in-vehicle biological detection system, characterized in that: It includes an acquisition module, a posture information determination module, an image generation module and an inference module; Wherein, the acquisition module is used to acquire the radio frequency echo signal; The position and posture information determination module is used to determine the position information and posture information of each living thing in the vehicle according to the radio frequency echo signal; The image generation module is used to generate a corresponding Doppler frequency shift heat map according to the radio frequency echo signal, each position information and each posture information; The inference module is used to input the Doppler frequency shift heat map and the radio frequency echo signal into a preset inference model, so that the inference model performs inference according to the Doppler frequency shift heat map and outputs detection information of each organism in the vehicle, wherein the detection information includes classification information, behavior information, posture information, and contour information of the corresponding organism. The inference model is constructed based on a teacher-apprentice network and is obtained through training based on a number of historical image data and a number of historical radio frequency echo signals.

Images