Multi-dimensional child left detection method and device for dual-mode communication

Abstract

This application provides a multi-dimensional method and device for detecting child abandonment using dual-mode communication. It achieves accurate detection of abandoned items through the fusion of radar, visual, and environmental parameters. An early warning mechanism is constructed, combining evidence fusion and hazard assessment to establish a reliable status determination strategy. Communication optimization is introduced, ensuring continuous improvement in response through dual-mode channels and local early warning. This method effectively addresses the shortcomings of traditional technologies in feature recognition, status determination, and early warning response, providing technical support for child safety protection.

Description

Technical Field

[0001] This application relates to the field of data processing, specifically to a multi-dimensional method and device for detecting children's legacy issues in dual-mode communication. Background Technology

[0002] Existing methods for detecting childhood autism have significant shortcomings. Traditional systems perform poorly in multi-sensor data acquisition and preprocessing, failing to effectively and accurately identify vital signs and thus affecting detection results.

[0003] Furthermore, existing technologies suffer from bottlenecks in feature fusion and state determination. Most systems lack robust evidence fusion mechanisms and hazard assessment strategies, resulting in suboptimal recognition accuracy.

[0004] Existing systems have technical shortcomings in early warning response. The lack of in-depth utilization of multiple communication channels makes it difficult to achieve efficient early warning transmission through dual-mode communication, impacting rescue timeliness. Solving these problems is crucial for improving child safety protection capabilities. Summary of the Invention

[0005] To address the problems in existing technologies, this application provides a multi-dimensional method and device for detecting children's abandoned items in dual-mode communication, which can effectively solve the shortcomings of traditional technologies in feature recognition, status determination and early warning response, and provide technical support for children's safety protection.

[0006] To solve at least one of the above problems, this application provides the following technical solution:

[0007] Firstly, this application provides a multi-dimensional method for detecting child legacy issues in dual-mode communication, including:

[0008] After the vehicle is turned off and locked, it enters the legacy detection mode. Based on the millimeter-wave radar sensor deployed in the cabin, the echo signal is collected and the phase change curve is obtained by the distance dimension fast Fourier transform and phase unwrapping. Based on the near-infrared camera, the image frame is collected and the pre-processed image frame is obtained after distortion correction. Based on the temperature sensor and carbon dioxide concentration sensor, the measured values ​​are collected to obtain environmental parameter data.

[0009] The phase change curve is subjected to bandpass filtering and spectral analysis to obtain the vital signs detection results. The preprocessed image frame is input into the occupant detection network to obtain the visual detection results. The vital signs detection results and the visual detection results are converted into evidence and fused to obtain the fusion confidence score. The fusion confidence score is compared with a preset threshold to obtain the child's residual status determination result. The environmental parameter data is compared with a preset threshold to obtain the risk level assessment result.

[0010] Based on the determination of the child's abandoned status and the assessment of the danger level, an early warning message is generated. The early warning message is sent to the cloud service platform via the cellular mobile network channel and to the vehicle owner's mobile terminal via the Bluetooth Low Energy channel. Local early warning measures are triggered according to the assessment of the danger level.

[0011] Furthermore, it also includes: collecting echo signals of frequency-modulated continuous wave mode based on millimeter-wave radar sensors deployed in the top area of ​​the vehicle cabin at a preset frame rate, performing fast Fourier transform on the echo signals along the distance dimension to obtain distance dimension spectrum data, and extracting complex signal sequences of each distance unit corresponding to the front and rear passenger areas in the vehicle cabin from the distance dimension spectrum data;

[0012] Phase calculation is performed on the complex signal sequence to obtain the instantaneous phase value sequence of the echo signal of each range cell. Phase unwrapping processing is performed on the instantaneous phase value sequence to eliminate phase jumps and obtain a continuous phase change curve. The continuous phase change curve is appended with a timestamp and written into the multi-dimensional sensing data buffer as a phase change curve.

[0013] Furthermore, it also includes: acquiring original image frames at a preset frame rate based on a near-infrared camera deployed in the vehicle cabin; performing distortion correction and grayscale normalization processing on the original image frames according to a pre-calibrated camera intrinsic parameter matrix and distortion coefficient to obtain pre-processed image frames; reading original measurement values ​​based on temperature sensors and carbon dioxide concentration sensors distributed in multiple locations in the vehicle cabin at a preset acquisition cycle; and performing validity checks on the original measurement values ​​to remove abnormal data that exceed the preset valid range to obtain valid measurement values.

[0014] The effective measurement values ​​are subjected to sliding window filtering to suppress transient noise interference and obtain filtered measurement values. The preprocessed image frames are appended with timestamps and written into the multi-dimensional perception data buffer. The filtered measurement values ​​are assembled into environmental parameter data and written into the multi-dimensional perception data buffer.

[0015] Furthermore, it also includes: processing the phase change curves through a respiratory bandpass filter and a heartbeat bandpass filter to obtain respiratory signals and heartbeat signals respectively; extracting the peak positions of the respiratory signals and heartbeat signals through spectral analysis to obtain respiratory frequency estimates and heart rate estimates, and assembling them into vital sign detection results; inputting the preprocessed image frames into an occupant detection network to obtain occupant bounding box coordinates; inputting the image regions corresponding to the occupant bounding box coordinates into an age group classification network to obtain age group classification results, and assembling them into visual detection results;

[0016] The vital signs detection results are converted into evidence of life presence and evidence of child characteristics, and the visual detection results are converted into evidence of occupant presence and evidence of child presence. Based on the expected reliability of each evidence source, credibility weights are assigned to the evidence of life presence, the evidence of child characteristics, the evidence of occupant presence, and the evidence of child presence. The fusion confidence level of each piece of evidence is obtained by evidence fusion calculation based on the credibility weights.

[0017] Furthermore, it also includes: comparing the fusion confidence score with a preset confirmation threshold; generating a child legacy confirmation status when the fusion confidence score continuously meets the condition of being higher than the preset confirmation threshold within a preset time window; generating a no-legacy status when the fusion confidence score is lower than the preset confirmation threshold; and using the child legacy confirmation status or the no-legacy status as the child legacy status determination result.

[0018] Temperature and carbon dioxide concentration values ​​are extracted from environmental parameter data. The temperature values ​​are compared with preset temperature thresholds and the carbon dioxide concentration values ​​are compared with preset concentration thresholds. The risk level assessment result is determined based on the degree to which the temperature and carbon dioxide concentration values ​​exceed the corresponding thresholds and the duration of the child's abandoned state determination result.

[0019] Furthermore, it also includes: reading the results of the child's abandoned status determination and the risk level assessment, and encapsulating the vehicle identification, detection time, abandoned status type and risk level, cabin environment parameters and vehicle location information in a preset message format to obtain early warning information;

[0020] The warning information is sent to the cloud service platform via the vehicle communication unit through the cellular mobile network channel. The cloud service platform queries the associated vehicle owner's mobile terminal identifier based on the vehicle identifier in the warning information and pushes the warning information to the vehicle owner's mobile terminal.

[0021] Furthermore, it also includes: broadcasting the warning information to the owner's mobile terminal within the vehicle's near-field range via the vehicle's Bluetooth Low Energy module; after receiving the warning information, the owner's mobile terminal performs local vibration alerts and displays message pop-ups.

[0022] Based on the hazard level assessment results, the local early warning measure triggering rule table is queried to determine the type of local early warning measure to be executed, and the control command corresponding to the type of local early warning measure is sent to the vehicle body controller to execute the audible and visual alarm and ventilation and cooling actions.

[0023] Secondly, this application provides a multi-dimensional child abandonment detection device for dual-mode communication, comprising:

[0024] The environmental monitoring module is used to enter the residual detection working mode after the vehicle is turned off and locked. It collects echo signals based on the millimeter-wave radar sensor deployed in the cabin and obtains the phase change curve by performing a distance-dimensional fast Fourier transform and phase unwrapping. It collects image frames based on the near-infrared camera and obtains pre-processed image frames after distortion correction. It obtains environmental parameter data based on the measured values ​​collected by the temperature sensor and carbon dioxide concentration sensor.

[0025] The image analysis module is used to obtain vital sign detection results by bandpass filtering and spectral analysis of the phase change curve, input the preprocessed image frame into the occupant detection network to obtain visual detection results, convert the vital sign detection results and the visual detection results into evidence and obtain fusion confidence by evidence fusion, compare the fusion confidence with a preset threshold to obtain the child's residual status determination result, and compare the environmental parameter data with a preset threshold to obtain the risk level assessment result.

[0026] An anomaly handling module is used to generate early warning information based on the child's abandoned status determination result and the danger level assessment result, send the early warning information to the cloud service platform via the cellular mobile network channel, send the early warning information to the vehicle owner's mobile terminal via the Bluetooth Low Energy channel, and trigger local early warning measures according to the danger level assessment result.

[0027] Thirdly, this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the multi-dimensional child legacy detection method for dual-mode communication.

[0028] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the multi-dimensional child legacy detection method for dual-mode communication.

[0029] Fifthly, this application provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the multi-dimensional child legacy detection method for dual-mode communication.

[0030] As described above, this application provides a multi-dimensional method and device for detecting child abandonment using dual-mode communication. It achieves accurate detection of abandoned items through the fusion of radar, visual, and environmental parameters. An early warning mechanism is constructed, combining evidence fusion and hazard assessment to establish a reliable status determination strategy. Communication optimization is introduced, ensuring continuous improvement of the response through dual-mode channels and local early warning. This method effectively addresses the shortcomings of traditional technologies in feature recognition, status determination, and early warning response, providing technical support for child safety protection. Attached Figure Description

[0031] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0032] Figure 1 This is a flowchart illustrating the multi-dimensional child legacy detection method for dual-mode communication in an embodiment of this application. Figure 2 This is a structural diagram of a multi-dimensional child legacy detection device for dual-mode communication in an embodiment of this application. Detailed Implementation

[0033] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0034] The acquisition, storage, use, and processing of data in this application all comply with relevant laws and regulations.

[0035] To address the shortcomings of existing technologies, this application provides a multi-dimensional method and device for detecting child abandonment using dual-mode communication. By fusing radar, visual, and environmental parameters, it achieves accurate detection of abandoned items. An early warning mechanism is constructed, combining evidence fusion and hazard assessment to establish a reliable status determination strategy. Communication optimization is introduced, ensuring continuous improvement in response through dual-mode channels and local early warning. This method effectively solves the deficiencies of traditional technologies in feature recognition, status determination, and early warning response, providing technical support for child safety protection.

[0036] To effectively address the shortcomings of traditional technologies in feature recognition, status determination, and early warning response, and to provide technical support for child safety protection, this application provides an embodiment of a multi-dimensional child abandonment detection method based on dual-mode communication. See [link to embodiment]. Figure 1 The multi-dimensional child legacy detection method for dual-mode communication specifically includes the following:

[0037] Step S101: After the vehicle is turned off and locked, the residual detection working mode is entered. The echo signal is collected based on the millimeter-wave radar sensor deployed in the cabin and the phase change curve is obtained by the distance dimension fast Fourier transform and phase unwrapping. The image frame is collected based on the near-infrared camera and the pre-processed image frame is obtained after distortion correction. The environmental parameter data is obtained based on the measurement values ​​collected by the temperature sensor and carbon dioxide concentration sensor.

[0038] In this embodiment, the process of entering the legacy detection working mode is triggered after the vehicle is turned off and all doors have completed the locking action. The vehicle controller reads the ignition switch status and door lock status signals through the body bus. When the ignition switch is in the off position and all door lock signals are valid, a wake-up command is sent to the sensor management module.

[0039] Upon receiving a wake-up command, the sensor management module sequentially activates the millimeter-wave radar sensor, near-infrared camera sensor, and environmental sensor group. The millimeter-wave radar sensor is deployed in the top area of ​​the vehicle cabin, and its antenna beam is pre-calibrated to cover the front and rear passenger areas. The radar sensor operates using a frequency-modulated continuous wave (FM) system, continuously transmitting linear frequency-modulated signals at a preset frame rate and receiving echo signals reflected from targets within the vehicle cabin.

[0040] Based on the echo signals acquired by the aforementioned radar sensors, this embodiment performs range-dimensional Fast Fourier Transform (FFT) processing. The transform process converts the time-domain echo signals to the frequency domain, generating range-dimensional spectral data. This embodiment extracts complex signal sequences corresponding to the distance units of each occupant area from the range-dimensional spectral data. The division of each distance unit is determined based on the seating layout within the vehicle cabin and the radar range resolution.

[0041] After the complex signal sequence is extracted, this embodiment performs phase calculation and phase unwrapping processing. Phase calculation obtains the instantaneous phase value at each sampling time by calculating the arctangent value of the complex signal. Since the phase measurement value is limited to the range of negative to positive pi, the phase value jumps when the cumulative target displacement exceeds half a wavelength. Phase unwrapping processing identifies the jump point by detecting whether the phase difference between adjacent sampling points exceeds a preset threshold, and adds an integer period compensation amount to the phase value after the jump point, thereby obtaining a continuous phase change curve reflecting the small displacement of the target.

[0042] The near-infrared camera sensor is deployed within the vehicle cabin to observe the entire occupant seating area and is equipped with an active near-infrared illuminator to ensure imaging capability under low-light conditions. In this embodiment, raw image frames are read from the near-infrared camera at a preset frame rate. Distortion correction is then performed on the raw image frames based on a pre-calibrated camera intrinsic parameter matrix and distortion coefficients. Distortion correction eliminates geometric distortions introduced by lens optical characteristics. Subsequently, grayscale normalization is performed to unify the image brightness distribution, and a pre-processed image frame is output.

[0043] The environmental sensor group includes temperature sensors and carbon dioxide concentration sensors distributed in multiple locations within the vehicle cabin. Temperature sensors are deployed near the dashboard area, the rear roof area, and the child safety seat installation area to acquire spatial temperature distribution information within the cabin. The carbon dioxide concentration sensor is deployed in the central area of ​​the cabin. In this embodiment, raw measurement values ​​are read from each sensor according to a preset acquisition cycle. The raw measurement values ​​are then subjected to validity checks to remove abnormal data exceeding a preset valid range. After being filtered by a sliding window to suppress transient noise interference, the data is assembled into environmental parameter data.

[0044] In this embodiment, the phase change curve, the preprocessed image frame, and the environmental parameter data are each appended with a timestamp and written into a multi-dimensional perception data buffer for the vital signs extraction module and the occupant detection module in subsequent step S201 to read and call.

[0045] Step S102: The phase change curve is subjected to bandpass filtering and spectral analysis to obtain the vital signs detection result. The preprocessed image frame is input into the occupant detection network to obtain the visual detection result. The vital signs detection result and the visual detection result are converted into evidence and fused to obtain the fusion confidence score. The fusion confidence score is compared with a preset threshold to obtain the child's residual status determination result. The environmental parameter data is compared with a preset threshold to obtain the risk level assessment result.

[0046] In this embodiment, the phase change curve written in step S101 is read from the multi-dimensional sensing data buffer, and bandpass filtering is performed on it to separate the respiratory signal and the heartbeat signal. The passband of the respiratory bandpass filter covers the normal human respiratory frequency range, and the passband of the heartbeat bandpass filter covers the normal human heart rate frequency range. The two filters act independently on the phase change curve, outputting the respiratory signal and the heartbeat signal respectively.

[0047] Based on the respiratory signal and the heartbeat signal, this embodiment performs spectral analysis to extract vital sign parameters. The spectral analysis employs a Fast Fourier Transform (FFT) method to transform the time-domain signal to the frequency domain and locate the spectral peak positions. The spectral peak position of the respiratory signal corresponds to the estimated respiratory rate, and the spectral peak position of the heartbeat signal corresponds to the estimated heart rate. This embodiment also calculates the ratio of spectral peak power to background noise power as a signal quality index, and assembles the estimated respiratory rate, estimated heart rate, and signal quality index into a vital sign detection result.

[0048] The preprocessed image frames are read from the multi-dimensional perceptual data buffer and then input into the occupant detection network. The occupant detection network employs a lightweight target detection architecture, outputting the bounding box coordinates and detection confidence scores of human targets within the vehicle cabin. In this embodiment, the image regions corresponding to the bounding box coordinates are cropped and input into an age group classification network, which outputs the probability distributions for infants, children, adolescents, and adults. This embodiment then assembles the bounding box coordinates and the age group classification results into a visual detection result.

[0049] After the vital signs detection results and the visual detection results are generated, this embodiment converts them into evidence representations. The vital signs detection results are converted into evidence of the presence of life and evidence of child characteristics, wherein the evidence of child characteristics is determined based on whether the respiratory rate and heart rate values ​​fall within the typical range for children. The visual detection results are converted into evidence of the presence of occupants and evidence of the presence of children, wherein the evidence of the presence of children is determined based on the probability of infants and children in the age group classification results.

[0050] Based on the foregoing evidence, this embodiment performs evidence fusion calculation to obtain the fusion confidence score. The fusion process first assigns confidence weights to each evidence source. Visual detection has a reduced weight under low-light conditions, and radar detection has a reduced weight when signal quality indicators are low. The calculation of the fusion confidence score adopts a joint framework of weighted aggregation and conflict adjustment. Specifically, C is equal to the sum of the weighted support scores of each piece of evidence divided by the difference between the normalization factor and the conflict penalty term, where C represents the fusion confidence score. The weighted support score of each piece of evidence is composed of the product of the individual evidence confidence score and its corresponding weight. The normalization factor is used to constrain the output range, and the conflict penalty term reflects the degree of contradiction between different evidence sources.

[0051] The fusion confidence level is compared with a preset confirmation threshold to generate a result indicating whether a child has been abandoned. If the fusion confidence level is continuously higher than the preset confirmation threshold within a preset time window, this embodiment generates a confirmed child abandonment status. If the fusion confidence level is lower than the preset confirmation threshold, this embodiment generates a no-abandonment status. Temporal stability constraints prevent misjudgments caused by transient sensor interference.

[0052] This embodiment reads environmental parameter data from a multi-dimensional sensing data buffer and extracts the temperature and carbon dioxide concentration values. The temperature value is compared with a preset temperature threshold, and the carbon dioxide concentration value is compared with a preset concentration threshold. Based on the degree to which the temperature and carbon dioxide concentration values ​​exceed the corresponding thresholds, and combined with the duration of the child's abandoned state assessment, this embodiment determines the risk level assessment result. The risk level is divided into four levels: low risk, medium risk, high risk, and extremely high risk, for the warning reporting module in subsequent step S103 to read and access.

[0053] Step S103: Generate early warning information based on the child's abandoned status determination result and the danger level assessment result, send the early warning information to the cloud service platform via the cellular mobile network channel, send the early warning information to the vehicle owner's mobile terminal via the Bluetooth Low Energy channel, and trigger local early warning measures according to the danger level assessment result.

[0054] This embodiment reads the child abandonment status determination result and the danger level assessment result generated in step S102 above. When the child abandonment status determination result is a confirmed child abandonment status, the early warning information generation process is initiated. The content fields of the early warning information include vehicle identification, detection time, abandonment status type, danger level, vehicle cabin environmental parameters, and vehicle location information.

[0055] The vehicle identifier is read from the identity storage area of ​​the vehicle controller, the detection time is taken from the system's real-time clock, and the legacy status type and danger level directly reference the aforementioned judgment results. The cabin environment parameters are read from the multi-dimensional perception data buffer, showing the most recent temperature and carbon dioxide concentration values. The vehicle location information is obtained from the vehicle positioning module, showing the current latitude and longitude coordinates. In this embodiment, the above fields are encapsulated according to a preset message format to obtain the warning information.

[0056] Based on the aforementioned warning information, this embodiment performs remote reporting via a cellular mobile network channel. The vehicle-mounted communication unit establishes a data connection with the public mobile communication network and sends the warning information to the cloud service platform via an encrypted channel. After receiving the warning information, the cloud service platform queries the associated vehicle owner's mobile terminal identifier based on the vehicle identifier and pushes a warning notification message to the vehicle owner's mobile terminal.

[0057] The warning information is simultaneously transmitted via Bluetooth Low Energy (BLE) to the vehicle owner's mobile terminal within the vehicle's near-field range. The vehicle's Bluetooth BLE module broadcasts the warning data packet, and the vehicle owner's mobile terminal within the communication coverage area receives the warning data packet and performs a local vibration alert and displays a message pop-up. The Bluetooth BLE channel serves as a backup notification path in areas with poor cellular network coverage.

[0058] This embodiment tracks and confirms the results of the warning information transmission. When cellular transmission fails, it automatically switches to Bluetooth for retry; when Bluetooth transmission fails, it automatically switches back to cellular transmission. The number of retries and the retry interval are dynamically adjusted based on the hazard level assessment results; the higher the hazard level, the higher the retry frequency.

[0059] Based on the aforementioned hazard level assessment results, this embodiment queries the local warning measure triggering rule table to determine the type of warning measure to be executed. Low hazard level triggers slow flashing of hazard lights; medium hazard level triggers rapid flashing of hazard lights and a short horn blast; high hazard level triggers continuous horn blasting and open windows for ventilation; extremely high hazard level triggers all audible and visual alarms and activates the air conditioning system to perform cooling actions.

[0060] In this embodiment, the control command corresponding to the local early warning measure type is sent to the vehicle body controller. After receiving the control command, the vehicle body controller drives the corresponding actuator to complete the audible and visual alarm and ventilation and cooling actions. The local early warning measures continue to be executed while waiting for rescue to arrive, delaying the deterioration of the cabin environment through ventilation and cooling actions. The exit condition for the early warning state is that a door is detected to be open or the vehicle is unlocked or no life is detected in the cabin.

[0061] As described above, the multi-dimensional child abandonment detection method for dual-mode communication provided in this application can achieve accurate detection of abandoned items through the fusion of radar, vision, and environmental parameters. An early warning mechanism is constructed, combining evidence fusion and hazard assessment to establish a reliable status determination strategy. Communication optimization is introduced, ensuring continuous improvement of the response through dual-mode channels and local early warning. This method effectively addresses the shortcomings of traditional technologies in feature recognition, status determination, and early warning response, providing technical assurance for child safety protection.

[0062] In one embodiment of the multi-dimensional child legacy detection method for dual-mode communication in this application, it may further include the following:

[0063] Step S201: Based on the millimeter-wave radar sensor deployed in the top area of ​​the vehicle cabin, the echo signal of the frequency-modulated continuous wave mode is collected at a preset frame rate. The echo signal is subjected to fast Fourier transform along the distance dimension to obtain distance dimension spectrum data. The complex signal sequence of each distance unit corresponding to the front and rear passenger areas in the vehicle cabin is extracted from the distance dimension spectrum data.

[0064] Step S202: Perform phase calculation on the complex signal sequence to obtain the instantaneous phase value sequence of the echo signal of each distance unit, perform phase unwrapping processing on the instantaneous phase value sequence to eliminate phase jumps and obtain a continuous phase change curve, and add a timestamp to the continuous phase change curve and write it into the multi-dimensional sensing data buffer as the phase change curve.

[0065] This embodiment relies on a millimeter-wave radar sensor deployed in the roof area of ​​the vehicle cabin to collect echo signals. The millimeter-wave radar sensor is installed in the central area of ​​the roof lining or near the rearview mirror mount, and its antenna beam is pre-calibrated to cover the spatial range of both the front and rear passenger areas. The radar sensor operates using a frequency-modulated continuous wave (FM) mode, with the transmitted signal frequency increasing linearly over time to form a frequency modulation ramp.

[0066] The millimeter-wave radar sensor continuously acquires echo signals at a preset frame rate. Within each frame acquisition period, the radar transmits a frequency-modulated continuous wave signal and receives echo signals reflected from various targets inside the vehicle cabin. The echo signals are mixed with the transmitted signals to generate beat signals. The frequency of the beat signals is proportional to the target distance; the greater the distance, the higher the beat frequency.

[0067] Based on the aforementioned acquired echo signals, this embodiment performs Fast Fourier Transform (FFT) processing along the range dimension. The transform process converts the time-domain beat signal to the frequency domain, generating range-dimensional spectral data. Each frequency point in the range-dimensional spectral data corresponds to a specific range cell, and the amplitude of that frequency point reflects the reflection intensity of the target within that range cell.

[0068] This embodiment extracts complex signal sequences corresponding to distance units in each occupant area within the vehicle cabin from the distance-dimensional spectrum data. The distance range between the front and rear occupant areas is pre-calibrated based on the vehicle's seating layout, and each area is divided into multiple distance units according to radar distance resolution. This embodiment extracts the complex values ​​of the corresponding distance units from each frame of distance-dimensional spectrum data and organizes them along the time dimension to form a complex signal sequence.

[0069] Based on the complex signal sequence, this embodiment performs phase calculation to obtain an instantaneous phase value sequence. Phase calculation is achieved by calculating the arctangent of the real and imaginary parts of the complex signal, outputting the instantaneous phase value of the range cell echo signal at each sampling time. The instantaneous phase value reflects the fine changes in the distance between the radar and the target on the wavelength scale.

[0070] The instantaneous phase value sequence exhibits a phase jump phenomenon. In this embodiment, phase unwrapping processing is performed to obtain a continuous phase change curve. The phase jump originates from the output of the arctangent function being restricted to the range of negative to positive pi. When the cumulative displacement of the target exceeds half a wavelength, the phase value periodically wraps around. The phase unwrapping processing is achieved by detecting the phase difference between adjacent sampling points point by point. When the absolute value of the phase difference exceeds a preset jump threshold, a phase jump is determined to have occurred, and an integer period compensation amount is added to all phase values ​​after the jump point.

[0071] The continuous phase change curve reflects the minute displacement changes of the target within the corresponding distance unit. Human breathing and heartbeat cause periodic fluctuations in the chest cavity, with displacement amplitudes on the order of millimeters. This displacement change is encoded in the phase change curve. In this embodiment, the continuous phase change curve is appended with a timestamp and written into the multi-dimensional sensing data buffer as a phase change curve for subsequent reading and use by the vital signs extraction module in step S301.

[0072] In one embodiment of the multi-dimensional child legacy detection method for dual-mode communication in this application, it may further include the following:

[0073] Step S301: Based on the near-infrared camera deployed in the cabin, the original image frames are acquired at a preset frame rate. The original image frames are subjected to distortion correction and grayscale normalization processing according to the pre-calibrated camera intrinsic parameter matrix and distortion coefficient to obtain pre-processed image frames. The original measurement values ​​are read based on the temperature sensor and carbon dioxide concentration sensor distributed in multiple locations in the cabin at a preset acquisition cycle. The validity of the original measurement values ​​is checked to remove abnormal data that exceeds the preset valid range to obtain valid measurement values.

[0074] Step S302: Perform sliding window filtering on the valid measurement values ​​to suppress transient noise interference and obtain filtered measurement values. Add timestamps to the preprocessed image frames and write them into the multi-dimensional perception data buffer. Assemble the filtered measurement values ​​into environmental parameter data and write them into the multi-dimensional perception data buffer.

[0075] This embodiment relies on a near-infrared camera deployed within the vehicle cabin for image acquisition. The near-infrared camera is mounted on the top of the cabin or above the dashboard, and its field of view is pre-configured to cover the entire passenger seating area and rear legroom. The near-infrared camera is equipped with an active near-infrared illuminator that emits invisible near-infrared light to illuminate the cabin interior, ensuring imaging capabilities at night or in environments with tinted windows.

[0076] The near-infrared camera continuously acquires raw image frames at a preset frame rate. The frame rate configuration is determined based on the embedded platform's processing capabilities and power consumption constraints, ensuring real-time detection while controlling system power consumption. Each raw image frame contains near-infrared grayscale image information of the scene inside the vehicle cabin.

[0077] Based on the original image frame, this embodiment performs distortion correction processing to eliminate geometric distortion introduced by the lens's optical characteristics. Distortion correction is achieved using a pre-calibrated camera intrinsic parameter matrix and distortion coefficients. The intrinsic parameter matrix describes the camera's focal length and optical center position, while the distortion coefficients describe the degree of radial and tangential distortion. The correction process performs a reverse mapping of the pixel coordinates in the original image frame, restoring the distorted image to an ideal image conforming to the pinhole camera model.

[0078] After distortion correction, this embodiment performs grayscale normalization on the image. Normalization maps image pixel values ​​to a uniform numerical range, eliminating differences in image brightness and contrast under different lighting conditions. After distortion correction and grayscale normalization are completed, a preprocessed image frame is output.

[0079] This embodiment simultaneously collects environmental parameters using temperature sensors and carbon dioxide concentration sensors distributed in multiple locations within the vehicle cabin. Temperature sensors are deployed in the dashboard area, rear roof area, and near the child safety seat installation area to obtain spatial temperature distribution information within the cabin. Carbon dioxide concentration sensors are deployed in the central area of ​​the cabin to detect changes in carbon dioxide concentration produced by human respiration.

[0080] The temperature sensor and carbon dioxide concentration sensor read the raw measurement values ​​according to a preset acquisition cycle. The acquisition cycle is configured according to the rate of change of each parameter. Since temperature changes relatively slowly, the acquisition cycle can be appropriately extended, while carbon dioxide concentration responds quickly to respiratory activity, so the acquisition cycle is shortened accordingly.

[0081] This embodiment performs a validity check on the original measured values ​​to remove abnormal data. The validity check compares the measured values ​​of each sensor with a preset valid range. Measurement values ​​exceeding the upper or lower limit of the valid range are identified as abnormal data and removed. Abnormal data may originate from sensor malfunctions or transient electromagnetic interference. After removal, valid measured values ​​are retained.

[0082] Based on the valid measurements, this embodiment performs sliding window filtering to suppress transient noise interference. The filtering process sets a fixed-length sliding window in the time dimension, and calculates a weighted average of the measurements at each sampling point within the window as the filtered output for the current moment. The filtering process smooths the high-frequency noise components in the measurement sequence, outputting the filtered measurement value.

[0083] In this embodiment, the preprocessed image frames are appended with timestamps and then written into the multi-dimensional sensing data buffer. The filtered measurement values ​​are assembled into environmental parameter data according to the structure of temperature and carbon dioxide concentration values, and are also written into the multi-dimensional sensing data buffer for the occupant detection module and environmental monitoring module in subsequent step S401 to read and call.

[0084] In one embodiment of the multi-dimensional child legacy detection method for dual-mode communication in this application, it may further include the following:

[0085] Step S401: The phase change curve is processed by the respiratory band bandpass filter and the heartbeat band bandpass filter respectively to obtain the respiratory signal and the heartbeat signal. The respiratory signal and the heartbeat signal are subjected to spectrum analysis to extract the peak position of the spectrum to obtain the respiratory frequency estimate and the heart rate estimate, and assembled into the vital signs detection result. The preprocessed image frame is input into the occupant detection network to obtain the occupant bounding box coordinates. The image region corresponding to the occupant bounding box coordinates is input into the age group classification network to obtain the age group classification result and assembled into the visual detection result.

[0086] Step S402: Convert the vital signs detection results into evidence of life presence and evidence of child characteristics, convert the visual detection results into evidence of occupant presence and evidence of child presence, assign credibility weights to the evidence of life presence, the evidence of child characteristics, the evidence of occupant presence, and the evidence of child presence based on the expected reliability of each evidence source, and calculate the fusion confidence level of each piece of evidence through evidence fusion according to the credibility weights.

[0087] In this embodiment, the phase change curve written in step S202 is read from the multi-dimensional sensing data buffer and subjected to dual-channel bandpass filtering. The lower passband limit of the respiratory bandpass filter corresponds to a respiratory rate of ten times per minute, and the upper passband limit corresponds to a respiratory rate of forty times per minute. The lower passband limit of the heartbeat bandpass filter corresponds to a heart rate of forty times per minute, and the upper passband limit corresponds to a heart rate of one hundred and fifty times per minute.

[0088] The phase change curves are processed by a respiratory bandpass filter and a heartbeat bandpass filter, respectively, to output respiratory and heartbeat signals. The respiratory bandpass filter suppresses the DC component and high-frequency noise in the phase change curves, while retaining the periodic phase fluctuations caused by respiratory activity. The heartbeat bandpass filter suppresses the respiratory frequency component and higher-frequency noise, while retaining the weak phase fluctuations caused by heartbeat activity.

[0089] Based on the respiratory signal and the heartbeat signal, this embodiment performs spectral analysis to extract vital sign parameters. The spectral analysis performs a Fast Fourier Transform on each signal, converting the time-domain waveform to a frequency-domain representation. In this embodiment, the peak position of the spectrum with the largest amplitude is located in the frequency domain; the peak position of the respiratory signal corresponds to the estimated respiratory rate, and the peak position of the heartbeat signal corresponds to the estimated heart rate.

[0090] This embodiment calculates the signal-to-noise ratio (SNR) of each spectral peak as a signal quality indicator. The SNR is characterized by the ratio of peak power to background noise power; a higher SNR indicates clearer and more discernible vital signs. This embodiment assembles the estimated respiratory rate, estimated heart rate, and signal quality indicator into a vital signs detection result.

[0091] The preprocessed image frames are read from the multi-dimensional perceptual data buffer and then input into the occupant detection network. The occupant detection network employs a lightweight target detection architecture for embedded platforms, and the target categories include human bodies, faces, child safety seats, and infant carriers. The network outputs the occupant bounding box coordinates and detection confidence scores for each detected target.

[0092] In this embodiment, the image region corresponding to the occupant bounding box coordinates is cropped and input into an age group classification network. The age group classification network performs age feature analysis on the cropped image region and outputs the probability distributions for each category: infant, child, adolescent, and adult. In this embodiment, the occupant bounding box coordinates, detection confidence, and age group classification results are assembled into a visual detection result.

[0093] Based on the vital signs detection results, this embodiment converts them into an evidentiary representation. Evidence of life presence is assessed based on whether the signal quality index exceeds a preset threshold to determine its support level. Evidence of pediatric characteristics is assessed based on whether the estimated respiratory rate and heart rate fall within the typical physiological parameter range for children; children typically have higher respiratory rates and heart rates than adults.

[0094] Based on the visual detection results, this embodiment converts them into an evidence representation form. The support level for evidence of occupant presence is determined based on the detection confidence level. The support level for evidence of child presence is determined based on the sum of the probabilities of the infant and child categories in the age group classification results.

[0095] This embodiment assigns credibility weights to each piece of evidence based on the expected reliability of each evidence source. Visual detection receives a lower credibility weight under low-light conditions, determined by whether the average brightness value of the preprocessed image frame is below a preset threshold. Radar detection receives a lower credibility weight when signal quality indicators are low. The credibility weights reflect the credibility of each evidence source under the current environmental conditions.

[0096] Based on the aforementioned evidence and credibility weights, this embodiment performs evidence fusion calculation. The fusion calculation weights and aggregates the support levels of each piece of evidence according to their corresponding credibility weights, while introducing a conflict adjustment term to handle contradictions between different evidence sources. The fusion calculation outputs a fusion confidence score for the hypothesis that the child exists, which is then read and used by the state determination module in subsequent step S501.

[0097] In one embodiment of the multi-dimensional child legacy detection method for dual-mode communication in this application, it may further include the following:

[0098] Step S501: Compare the fusion confidence with the preset confirmation threshold. When the fusion confidence continuously meets the condition of being higher than the preset confirmation threshold within a preset time window, a child legacy confirmation status is generated. When the fusion confidence is lower than the preset confirmation threshold, a no-legacy status is generated. The child legacy confirmation status or the no-legacy status is used as the child legacy status determination result.

[0099] Step S502: Extract temperature and carbon dioxide concentration values ​​from environmental parameter data, compare the temperature value with a preset temperature threshold and compare the carbon dioxide concentration value with a preset concentration threshold, and determine the risk level assessment result based on the degree to which the temperature value and the carbon dioxide concentration value exceed the corresponding threshold and the duration of the child's abandoned state determination result.

[0100] In this embodiment, the fusion confidence score output in step S402 is read and compared with a preset confirmation threshold. The preset confirmation threshold is stored in the legacy detection configuration file of the vehicle terminal, and its value is determined based on the balance requirements between detection sensitivity and false alarm rate.

[0101] When the fusion confidence level exceeds the preset confirmation threshold, this embodiment initiates the temporal stability verification process. Temporal stability verification requires the fusion confidence level to remain continuously above the preset confirmation threshold within a preset duration window. The preset duration window covers multiple detection cycles to avoid single-frame misjudgments triggering error warnings due to transient sensor interference.

[0102] When the fusion confidence score continuously meets the condition of being higher than the preset confirmation threshold within a preset time window, this embodiment generates a child abandonment confirmation status. The child abandonment confirmation status indicates that the determination that a child has been abandoned in the vehicle cabin is valid, and the system enters the early warning response process.

[0103] When the fusion confidence level is lower than the preset confirmation threshold, this embodiment generates a "no leftover" status. The "no leftover" status indicates that no reliable evidence of a child's presence in the vehicle cabin was found during the current detection cycle. This embodiment writes either the "child leftover confirmation" status or the "no leftover" status as the child leftover status determination result into the status determination result register.

[0104] In this embodiment, the environmental parameter data written in step S302 is read from the multi-dimensional sensing data buffer, and the temperature value and carbon dioxide concentration value are extracted from it. The temperature value is taken from the highest value of the measurement results of multiple temperature sensors in the vehicle cabin, reflecting the worst thermal environment conditions in the vehicle cabin.

[0105] The temperature value is compared with a preset temperature threshold. The preset temperature threshold is divided into multiple levels, including a warning temperature threshold, a danger temperature threshold, and an extreme temperature threshold. Temperature values ​​exceeding different threshold levels correspond to different degrees of heat stress risk.

[0106] The carbon dioxide concentration value is compared with a preset concentration threshold. A continuous increase in carbon dioxide concentration indicates respiratory activity and insufficient ventilation in the vehicle cabin. When the concentration exceeds the threshold, the air quality in the vehicle cabin poses a threat to human health.

[0107] This embodiment determines the risk level assessment result based on the degree to which the temperature and carbon dioxide concentration values ​​exceed the corresponding thresholds, combined with the duration of the child's abandoned state. The risk level is divided into four levels: low risk, medium risk, high risk, and very high risk.

[0108] Low hazard level corresponds to a situation where the cabin temperature is within a safe range and the follow-up confirmation time is short. Medium hazard level corresponds to a situation where the cabin temperature is close to the warning threshold or the follow-up confirmation time exceeds the initial time limit. High hazard level corresponds to a situation where the cabin temperature exceeds the warning threshold or the follow-up confirmation time exceeds the emergency time limit. Very high hazard level corresponds to a situation where the cabin temperature reaches the danger threshold.

[0109] In this embodiment, the hazard level assessment result is written into the environmental assessment result register for subsequent reading and calling by the early warning reporting module and the local early warning module in step S601.

[0110] In one embodiment of the multi-dimensional child legacy detection method for dual-mode communication in this application, it may further include the following:

[0111] Step S601: Read the results of the child's abandoned status determination and the risk level assessment, and encapsulate the vehicle identification, detection time, abandoned status type, risk level, cabin environment parameters, and vehicle location information into a preset message format to obtain the warning information;

[0112] Step S602: The warning information is sent to the cloud service platform via the vehicle communication unit through the cellular mobile network channel. The cloud service platform queries the associated vehicle owner mobile terminal identifier based on the vehicle identifier in the warning information and pushes the warning information to the vehicle owner mobile terminal.

[0113] In this embodiment, the status determination result of the child being left behind, generated in step S501, is read from the status determination result register, and the hazard level assessment result generated in step S502, generated from the environmental assessment result register, is read from the environmental assessment result register. When the status determination result of the child being left behind is a confirmed case of child being left behind, this embodiment initiates the early warning information encapsulation process.

[0114] The warning information includes six fields: vehicle identification, detection time, residual status type, hazard level, cabin environment parameters, and vehicle location information. Each field is obtained from different data sources and organized according to a preset message format.

[0115] The vehicle identifier is read from the identity storage area of ​​the on-board controller; this identifier is a unique identification code written to the vehicle at the factory. The detection time is obtained from the system's real-time clock module, with the current time stamp accurate to the second.

[0116] The type of abandoned status is directly referenced from the status marker in the child's abandoned status determination result. The risk level is directly referenced from the level marker in the risk level assessment result, with values ​​ranging from low risk, medium risk, high risk, and extremely high risk.

[0117] The cabin environment parameters are read from the most recently written temperature and carbon dioxide concentration values ​​from the multi-dimensional sensing data buffer. The vehicle location information is obtained from the onboard positioning module, which provides the current latitude and longitude coordinates and positioning accuracy.

[0118] This embodiment encapsulates the above six fields into a pre-defined message format to obtain the warning information. The message format defines the data type, byte length, and arrangement order of each field to ensure that the receiving end can correctly parse the message content.

[0119] Based on the aforementioned warning information, this embodiment performs remote reporting via a cellular mobile network channel. The vehicle-mounted communication unit has a built-in cellular mobile communication module, supporting access to public mobile communication networks to establish data connections.

[0120] In this embodiment, the warning information is encrypted and then sent to the cloud service platform via a cellular mobile network channel. The encryption process uses a pre-negotiated key to protect the message content and prevent information leakage during transmission.

[0121] After receiving the warning information, the cloud service platform performs message parsing and vehicle owner association queries. Based on the vehicle identifier in the warning information, the cloud service platform queries the vehicle registration database to obtain the vehicle owner's mobile terminal identifier associated with that vehicle.

[0122] The cloud service platform pushes a warning notification message to the vehicle owner's mobile terminal. The message includes key information such as the type of residual status, the danger level, the cabin temperature, and the vehicle's location. Upon receiving the push, the vehicle owner's mobile terminal displays a pop-up message and provides an audio alert. When the danger level is high or extremely high, the cloud service platform simultaneously initiates a live agent follow-up process, where an agent proactively contacts the vehicle owner to confirm the situation. The Bluetooth transmission of the warning information and the triggering of local warning measures are executed in subsequent step S701.

[0123] In one embodiment of the multi-dimensional child legacy detection method for dual-mode communication in this application, it may further include the following:

[0124] Step S701: Broadcast the warning information to the owner's mobile terminal within the vehicle's near-field range via the vehicle's Bluetooth Low Energy module. After receiving the warning information, the owner's mobile terminal performs local vibration alerts and displays a message pop-up.

[0125] Step S702: Based on the hazard level assessment results, query the local early warning measure triggering rule table to determine the type of local early warning measure to be executed, and send the control command corresponding to the type of local early warning measure to the vehicle body controller to execute the audible and visual alarm and ventilation and cooling actions.

[0126] In this embodiment, the warning information generated in step S601 is transmitted via near-field broadcast using the vehicle-mounted Bluetooth Low Energy module. The vehicle-mounted Bluetooth Low Energy module is deployed within the vehicle control unit and supports the Bluetooth Low Energy broadcast protocol.

[0127] The warning information is compressed and format converted before being encapsulated into a Bluetooth broadcast data packet. The payload length of the broadcast data packet is constrained by the Bluetooth Low Energy protocol. In this embodiment, the key fields in the warning information are simplified and encoded, retaining only the three core contents: legacy status type, hazard level, and vehicle identification.

[0128] The vehicle-mounted Bluetooth Low Energy module periodically broadcasts the data packets to areas within the vehicle's near-field range. The broadcast period and transmission power are configured based on power consumption constraints and coverage requirements, with the near-field range typically covering an area of ​​tens of meters around the vehicle.

[0129] The owner's mobile terminal, located within the vehicle's near-field range, receives the broadcast data packets via Bluetooth Low Energy scanning. The owner's mobile terminal has a pre-installed vehicle-specific application that continuously listens for Bluetooth broadcast signals matching the vehicle's identifier in the background.

[0130] After receiving the warning information, the vehicle owner's mobile terminal executes a local alert. The local vibration alert is achieved by activating the mobile terminal's vibration motor, with different intensity and duration settings configured according to the level of danger. A message pop-up displays the key content of the warning information in a prominent interface on the mobile terminal screen.

[0131] The Bluetooth Low Energy channel serves as a backup notification path in areas with poor cellular network coverage. When the driver is near the vehicle but the cellular signal is weak, the Bluetooth channel ensures that warning messages are delivered promptly.

[0132] In this embodiment, the hazard level assessment result generated in step S502 is read from the environmental assessment result register, and the local early warning measure triggering rule table is queried to determine the type of local early warning measure to be executed. The triggering rule table is indexed by hazard level and records the combination of early warning measures corresponding to each level.

[0133] The local warning measures for a low-risk level are slow flashing hazard lights. For a medium-risk level, the local warning measures are a combination of rapid flashing hazard lights and short horn blasts. For a high-risk level, the local warning measures are a combination of continuous horn blasts and open windows for ventilation. For an extremely high-risk level, the local warning measures are a combination of all audible and visual alarms and air conditioning system cooling.

[0134] This embodiment converts the local warning measure type into a corresponding control command. The control command includes an actuator identifier and action parameters. The actuator identifier distinguishes between the light controller, horn controller, window motor, and air conditioning controller.

[0135] The control commands are sent to the body controller via the vehicle bus. Upon receiving the control commands, the body controller drives the corresponding actuators to complete the audible and visual alarm and ventilation / cooling actions. The audible and visual alarm attracts the attention of people around the vehicle through visual and auditory channels, while ventilation / cooling slows down the deterioration of the cabin environment by opening windows or turning on the air conditioning, thus buying time for rescue of the abandoned children.

[0136] To effectively address the shortcomings of traditional technologies in feature recognition, state determination, and early warning response, and to provide technical support for child safety protection, this application provides an embodiment of a multi-dimensional child abandonment detection device for dual-mode communication, which implements all or part of the aforementioned multi-dimensional child abandonment detection method for dual-mode communication. See [link to embodiment]. Figure 2 The multi-dimensional child abandonment detection device for dual-mode communication specifically includes the following components:

[0137] The environmental monitoring module 10 is used to enter the residual detection working mode after the vehicle is turned off and locked. It collects echo signals based on the millimeter-wave radar sensor deployed in the vehicle cabin and obtains the phase change curve by performing a distance-dimensional fast Fourier transform and phase unwrapping. It collects image frames based on the near-infrared camera and obtains pre-processed image frames by distortion correction. It obtains environmental parameter data based on the measured values ​​collected by the temperature sensor and carbon dioxide concentration sensor.

[0138] Image analysis module 20 is used to obtain vital sign detection results by bandpass filtering and spectrum analysis of the phase change curve, input the preprocessed image frame into the occupant detection network to obtain visual detection results, convert the vital sign detection results and the visual detection results into evidence and obtain fusion confidence by evidence fusion, compare the fusion confidence with a preset threshold to obtain the child's residual status determination result, and compare the environmental parameter data with a preset threshold to obtain the risk level assessment result.

[0139] The abnormal handling module 30 is used to generate early warning information based on the child's abandoned status determination result and the danger level assessment result, send the early warning information to the cloud service platform via the cellular mobile network channel, send the early warning information to the vehicle owner's mobile terminal via the Bluetooth Low Energy channel, and trigger local early warning measures according to the danger level assessment result.

[0140] As described above, the multi-dimensional child abandonment detection device for dual-mode communication provided in this application can achieve accurate detection of abandoned items through the fusion of radar, vision, and environmental parameters. An early warning mechanism is constructed, combining evidence fusion and hazard assessment to establish a reliable status determination strategy. Communication optimization is introduced, ensuring continuous improvement of the response through dual-mode channels and local early warning. This method effectively solves the shortcomings of traditional technologies in feature recognition, status determination, and early warning response, providing technical assurance for child safety protection.

[0141] This invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the multi-dimensional child legacy detection method for dual-mode communication.

[0142] This invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described multi-dimensional child legacy detection method for dual-mode communication.

[0143] This invention also provides a computer program product, which includes a computer program that, when executed by a processor, implements the above-described multi-dimensional child legacy detection method for dual-mode communication.

[0144] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0145] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0146] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0147] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0148] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A multi-dimension child left detection method for dual-mode communication, characterized in that, The method includes: After the vehicle is turned off and locked, it enters the legacy detection mode. Based on the millimeter-wave radar sensor deployed in the cabin, the echo signal is collected and the phase change curve is obtained by the distance dimension fast Fourier transform and phase unwrapping. Based on the near-infrared camera, the image frame is collected and the pre-processed image frame is obtained after distortion correction. Based on the temperature sensor and carbon dioxide concentration sensor, the measured values ​​are collected to obtain environmental parameter data. The phase change curve is subjected to bandpass filtering and spectral analysis to obtain the vital signs detection results. The preprocessed image frame is input into the occupant detection network to obtain the visual detection results. The vital signs detection results and the visual detection results are converted into evidence and fused to obtain the fusion confidence score. The fusion confidence score is compared with a preset threshold to obtain the child's residual status determination result. The environmental parameter data is compared with a preset threshold to obtain the risk level assessment result. Based on the determination of the child's abandoned status and the assessment of the danger level, an early warning message is generated. The early warning message is sent to the cloud service platform via the cellular mobile network channel and to the vehicle owner's mobile terminal via the Bluetooth Low Energy channel. Local early warning measures are triggered according to the assessment of the danger level.

2. The multi-dimension child left detection method for dual-mode communication according to claim 1, wherein, The process involves entering a residual detection mode after the vehicle is turned off and locked. This mode utilizes millimeter-wave radar sensors deployed within the vehicle cabin to collect echo signals, which are then processed through a range-dimensional fast Fourier transform and phase unwrapping to obtain phase change curves. This includes: Based on the millimeter-wave radar sensor deployed in the top area of ​​the vehicle cabin, the echo signal of the frequency-modulated continuous wave mode is collected at a preset frame rate. The echo signal is subjected to fast Fourier transform along the distance dimension to obtain distance dimension spectrum data. The complex signal sequence corresponding to each distance unit of the front passenger area and the rear passenger area in the vehicle cabin is extracted from the distance dimension spectrum data. Phase calculation is performed on the complex signal sequence to obtain the instantaneous phase value sequence of the echo signal of each range cell. Phase unwrapping processing is performed on the instantaneous phase value sequence to eliminate phase jumps and obtain a continuous phase change curve. The continuous phase change curve is appended with a timestamp and written into the multi-dimensional sensing data buffer as a phase change curve.

3. The multi-dimension child left detection method for dual-mode communication according to claim 1, wherein, The process involves acquiring image frames using a near-infrared camera and obtaining preprocessed image frames after distortion correction. Environmental parameter data is then obtained based on measurements from a temperature sensor and a carbon dioxide concentration sensor, including: Based on the near-infrared camera deployed in the cabin, the original image frames are acquired at a preset frame rate. The original image frames are then subjected to distortion correction and grayscale normalization processing according to the pre-calibrated camera intrinsic parameter matrix and distortion coefficient to obtain pre-processed image frames. The original measurement values ​​are read by temperature sensors and carbon dioxide concentration sensors distributed in multiple locations in the cabin at a preset acquisition cycle. The validity of the original measurement values ​​is then checked to remove abnormal data that exceeds the preset valid range to obtain valid measurement values. The effective measurement values ​​are subjected to sliding window filtering to suppress transient noise interference and obtain filtered measurement values. The preprocessed image frames are appended with timestamps and written into the multi-dimensional perception data buffer. The filtered measurement values ​​are assembled into environmental parameter data and written into the multi-dimensional perception data buffer.

4. The multi-dimension child left detection method for dual-mode communication according to claim 1, wherein, The process of obtaining vital sign detection results by bandpass filtering and spectral analysis of the phase change curve, inputting the preprocessed image frame into the occupant detection network to obtain visual detection results, and converting the vital sign detection results and visual detection results into evidence and obtaining fusion confidence through evidence fusion includes: The phase change curves are processed by a respiratory bandpass filter and a heartbeat bandpass filter to obtain respiratory and heartbeat signals, respectively. The respiratory and heartbeat signals are then analyzed by spectrum analysis to extract the peak positions of the spectrum to obtain respiratory frequency estimates and heart rate estimates, which are then assembled into vital sign detection results. The preprocessed image frames are input into the occupant detection network to obtain occupant bounding box coordinates. The image regions corresponding to the occupant bounding box coordinates are input into the age group classification network to obtain age group classification results, which are then assembled into visual detection results. The vital signs detection results are converted into evidence of life presence and evidence of child characteristics, and the visual detection results are converted into evidence of occupant presence and evidence of child presence. Based on the expected reliability of each evidence source, credibility weights are assigned to the evidence of life presence, the evidence of child characteristics, the evidence of occupant presence, and the evidence of child presence. The fusion confidence level of each piece of evidence is obtained by evidence fusion calculation based on the credibility weights.

5. The multi-dimension child left detection method for dual-mode communication according to claim 1, wherein, The step of comparing the fusion confidence level with a preset threshold to obtain the child's abandoned status determination result, and comparing the environmental parameter data with a preset threshold to obtain the risk level assessment result, includes: The fusion confidence score is compared with a preset confirmation threshold. When the fusion confidence score continuously meets the condition of being higher than the preset confirmation threshold within a preset time window, a child legacy confirmation status is generated. When the fusion confidence score is lower than the preset confirmation threshold, a no-legacy status is generated. The child legacy confirmation status or the no-legacy status is used as the child legacy status determination result. Temperature and carbon dioxide concentration values ​​are extracted from environmental parameter data. The temperature values ​​are compared with preset temperature thresholds and the carbon dioxide concentration values ​​are compared with preset concentration thresholds. The risk level assessment result is determined based on the degree to which the temperature and carbon dioxide concentration values ​​exceed the corresponding thresholds and the duration of the child's abandoned state determination result.

6. The multi-dimensional child abandonment detection method for dual-mode communication according to claim 1, characterized in that, The process of generating early warning information based on the child's abandoned status determination result and the risk level assessment result, and sending the early warning information to the cloud service platform via a cellular mobile network channel, includes: Read the results of the child's abandoned status assessment and the risk level assessment, and encapsulate the vehicle identification, detection time, abandoned status type and risk level, cabin environmental parameters and vehicle location information in a preset message format to obtain the warning information; The warning information is sent to the cloud service platform via the vehicle communication unit through the cellular mobile network channel. The cloud service platform queries the associated vehicle owner's mobile terminal identifier based on the vehicle identifier in the warning information and pushes the warning information to the vehicle owner's mobile terminal.

7. The multi-dimensional child legacy detection method for dual-mode communication according to claim 1, characterized in that, The step of sending the warning information to the vehicle owner's mobile terminal via Bluetooth Low Energy channel and triggering local warning measures based on the hazard level assessment result includes: The warning information is broadcast to the owner's mobile terminal within the vehicle's near-field range via the vehicle's Bluetooth Low Energy module. After receiving the warning information, the owner's mobile terminal performs a local vibration alert and displays a message pop-up. Based on the hazard level assessment results, the local early warning measure triggering rule table is queried to determine the type of local early warning measure to be executed, and the control command corresponding to the type of local early warning measure is sent to the vehicle body controller to execute the audible and visual alarm and ventilation and cooling actions.

8. A multi-dimensional child abandonment detection device for dual-mode communication, characterized in that, The device includes: The environmental monitoring module is used to enter the residual detection working mode after the vehicle is turned off and locked. It collects echo signals based on the millimeter-wave radar sensor deployed in the cabin and obtains the phase change curve by performing a distance-dimensional fast Fourier transform and phase unwrapping. It collects image frames based on the near-infrared camera and obtains pre-processed image frames after distortion correction. It obtains environmental parameter data based on the measured values ​​collected by the temperature sensor and carbon dioxide concentration sensor. The image analysis module is used to obtain vital sign detection results by bandpass filtering and spectral analysis of the phase change curve, input the preprocessed image frame into the occupant detection network to obtain visual detection results, convert the vital sign detection results and the visual detection results into evidence and obtain fusion confidence by evidence fusion, compare the fusion confidence with a preset threshold to obtain the child's residual status determination result, and compare the environmental parameter data with a preset threshold to obtain the risk level assessment result. An anomaly handling module is used to generate early warning information based on the child's abandoned status determination result and the danger level assessment result, send the early warning information to the cloud service platform via the cellular mobile network channel, send the early warning information to the vehicle owner's mobile terminal via the Bluetooth Low Energy channel, and trigger local early warning measures according to the danger level assessment result.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the multi-dimensional child legacy detection method for dual-mode communication as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the steps of the multi-dimensional child legacy detection method for dual-mode communication as described in any one of claims 1 to 7.

Images