A method and system for detecting a child left in a vehicle

Abstract

The present application relates to the technical field of automobile safety prevention and control, and particularly relates to a method and system for detecting children left in a vehicle. The method for detecting children left in a vehicle comprises the following steps: S3: generating a thermal effect map according to the extracted information related to the movement of a living target, and determining whether a passenger is left in the vehicle; S6: obtaining the size of a human body limb according to the information of the pixel coordinate points of the human body skeleton, fitting the volume of the passenger by the size of the human body limb, and thus completing the judgment of the passenger category; and S7: according to the judgment results of steps S3 and S6, the CPD system can send an alarm information. According to the common judgment results of the millimeter wave radar and the camera, the CPD system sends an alarm information, compared with the prior art in which the millimeter wave radar or the camera is used alone as the direct detection device of the CPD system, the false alarm and the missed alarm risks are greatly reduced.

Description

Technical Field

[0001] This invention relates to the field of automotive safety and security technology, specifically to a method and system for detecting children left in vehicles. Background Technology

[0002] Currently, safety issues arising from parents leaving children alone in enclosed vehicles due to negligence are common. Vehicles exposed to sunlight experience rapid increases in interior temperature, potentially leading to heatstroke, unconsciousness, or even death in children. Similar cases of child deaths have been reported both domestically and internationally in recent years. While cases of children dying from heatstroke in vehicles are less frequent than those from traffic accidents, these entirely preventable deaths deserve special attention.

[0003] Euro NCAP is the European New Car Assessment Program, which aims to assess and compare the safety performance of vehicles. It includes provisions for Child Presence Detection (CPD), requiring the vehicle's CPD system to be able to detect the presence of a child in the vehicle, issue an alert to the owner or a third-party service provider, and take appropriate and effective intervention measures if the alert is ignored.

[0004] In existing technologies, such as Chinese patent CN110576819A, a method for preventing and monitoring the dangers of leaving young children in cars is disclosed. This method uses millimeter-wave radar to detect the cardiopulmonary activity of living beings to detect children left behind. However, since the signal received by the millimeter-wave radar receiver is usually a superposition of signals from multiple target objects, including living beings inside the car, the edges of the car cabin, and objects inside the vehicle interior, the information generated by the child's breathing and heartbeat is not the primary feedback to the radar. Directly judging whether it is a child based on the above information has a high risk of misjudgment and missed detection. Another example is Chinese patent CN113978354A, which discloses a method, device, equipment, and storage medium for vehicle control. It uses images obtained by an in-vehicle camera to determine whether there is a child in the car. However, in practice, in-vehicle facilities such as seat backs, headrests, and armrests interfere with the camera's line of sight. Especially for younger infants, blankets or other coverings are usually added, making it impossible to accurately judge the presence of a child based on the images obtained by the camera, which also has a high risk of misjudgment and missed detection. Therefore, there is a need to provide a method and system for detecting children left in vehicles that combines the advantages of millimeter-wave radar and cameras. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a method and system for detecting children left in vehicles, thereby solving the problem of high risks of false positives and false negatives in existing technologies.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: a method for detecting children left in a vehicle, comprising the following steps:

[0007] S1: The millimeter-wave radar of the CPD system emits millimeter waves at a specific carrier frequency and acquires the scattering point echo of the detected living target;

[0008] The carrier frequency emitted by the millimeter-wave radar is waveform And the scattered point echo Sp2t of a living target detected by radar can be expressed as:

[0009]

[0010]

[0011] in, Indicates in The amplitude value of the echo signal at time t, where c is the speed of light in a vacuum. Indicates the distance difference between the radar and a living target. It represents the imaginary unit of complex numbers and is used to write complex radar signals.

[0012] S2: Obtain the corresponding Doppler frequency shift based on the echo from the scattering point, perform data processing and analysis on the collected micro-Doppler signals, and extract information related to the motion of the living target;

[0013] The calculation process for the Doppler frequency shift is as follows:

[0014] The phase difference between the echo signal and the transmitted signal is:

[0015]

[0016] right By differentiating the time step, we can obtain the Doppler frequency shift corresponding to the scattering point. :

[0017]

[0018] in, It represents a tiny change in phase difference and is used to calculate the Doppler frequency shift;

[0019] S3: Based on the extracted information related to the movement of the living target, generate a thermal effect map to determine whether there are any occupants left inside the vehicle;

[0020] S4: Based on the judgment result of S3, the camera makes further judgments;

[0021] S5: The CPD system's camera captures the visual features of the occupants, and uses the OpenPose model based on the VGG-19 convolutional neural network framework to train images to obtain the pixel coordinates of human skeletons.

[0022] S6: Obtain the human limb size based on the pixel coordinate information of the human skeleton, fit the occupant's volume by the human limb size, and thus complete the occupant category determination.

[0023] S7: Based on the judgment results of steps S3 and S6, the CPD system can issue an alarm message.

[0024] The beneficial effects of this solution are as follows: This solution uses millimeter-wave radar to determine if any occupants are left inside the vehicle, and then uses a camera to further classify the occupants. Based on the combined judgment results of the millimeter-wave radar and the camera, the CPD system issues an alarm. Compared to existing technologies that use only millimeter-wave radar or a camera as the direct detection device for the CPD system, this significantly reduces the risk of false alarms and missed alarms. Specifically, the radar echo signal is decomposed, filtered, and screened using singular value decomposition to obtain the micro-Doppler signal of the desired object. The generated thermal image can accurately identify whether any occupants are left behind. Combined with the visual features acquired by the camera, the OpenPose model, based on the VGG-19 convolutional neural network framework, is used to train images to obtain the pixel coordinates of human skeletons. The volume of the occupant is then fitted using human limb dimensions to complete the logical judgment of the passenger category inside the vehicle.

[0025] Since the CPD system primarily detects children inside the vehicle, especially in scenarios involving children in child seats and covered with blankets, it requires high penetration capabilities from the sensors. Therefore, millimeter-wave radar was chosen as the sensing tool instead of lidar, which offers high precision but weaker penetration. Millimeter-wave radar is used for liveness detection to determine if an occupant has been left inside the vehicle, offering higher accuracy. Furthermore, millimeter-wave radar can be discreetly installed inside the vehicle, without affecting the interior or exterior, and can operate 24 / 7, unaffected by inclement weather.

[0026] Millimeter-wave radar uses echo imaging to detect the state of objects by emitting electromagnetic waves. If there is relative motion between the transmitter and the target, the frequency of the information received by the receiver from the transmitter will differ from the frequency of the transmitted information. When the target moves closer to the transmitter, the frequency of the reflected signal will be higher than the frequency of the transmitted signal, and vice versa. By utilizing the periodic movement characteristics of the human body, such as breathing and heartbeat, the time-frequency domain micro-Doppler effect generates a thermal image to identify living beings. This method is used solely to determine if an occupant is left inside the vehicle, and its accuracy is high. Compared to existing technologies that directly use the aforementioned breathing and heartbeat characteristics to determine whether an occupant is an adult or a child, this method reduces the risk of false alarms.

[0027] Further, in step S5, obtaining the pixel coordinates of human skeletons includes a first algorithm branch and a second algorithm branch. The first algorithm branch is for training the confidence of skeleton points, and the output is a set of coordinates and confidence scores of human skeleton points, representing the probability that each skeleton point in the image training set is a key point of the human skeleton. The first stage of the first algorithm branch will generate a set of detection confidence maps.

[0028]

[0029] The second algorithm branch trains the affinity between bone points, and the output is the compatibility between bone points. The first stage of the second algorithm branch generates a set of affinity vectors.

[0030]

[0031] The first and second algorithm branches undergo multiple iterations of training, using a loss function to calculate the loss value and provide feedback to the previous network structure. Subsequent stages iteratively update the confidence and fit values ​​from the previous stage, as shown in the following formula:

[0032]

[0033]

[0034] in, and represents the confidence and fit of the skeletal points obtained after t iterations, respectively; F represents the visual features extracted by the VGG19 network. and The function symbol is defined, and its superscript t represents the function obtained after the t-th iteration update;

[0035] After multiple iterations, the network parameters are continuously updated until the accuracy standard is met, thus obtaining the final set of skeleton point confidence scores:

[0036]

[0037] And the set of skeletal point fits used for limb connections:

[0038]

[0039] in, Represents the J elements in the confidence set. The set represents the C elements in the fit set. The set of bone point confidence and the set of bone point fit together constitute the complete human bone pixel coordinate information.

[0040] The beneficial effects of this solution are as follows: Based on the acquisition of the aforementioned human skeleton pixel coordinate information, on the one hand, the algorithm's multiple iterations ensure the accuracy of the final acquired information; on the other hand, it can mitigate the interference of other facilities inside the vehicle on the camera's line of sight during actual operation, allowing occluded skeleton points to be inferred from the skeleton points of other complete limbs. By simulating limb size and structure through human skeleton pixel coordinates, the type of occupant remaining in the vehicle can be determined more stably, avoiding the significant impact of occlusion on existing conventional camera recognition methods such as facial recognition. The OpenPose algorithm, with its bottom-up approach, first detects key points of human limbs and then maps these key points to different limb information and corresponding human bodies, representing the set of skeleton point confidence and the set of skeleton point affinity, respectively. Compared with existing recognition algorithms, it is more effective in recognizing and detecting single and multiple individuals, meeting the coverage requirements of real-world in-vehicle scenarios.

[0041] Furthermore, in step S7, the logic for determining the alarm information issued by the CPD system is as follows:

[0042] The millimeter-wave radar detected a living creature, and the camera identified it as a child, triggering an alarm.

[0043] The millimeter-wave radar detected a living creature, and the camera identified it as an adult, so no alarm was triggered.

[0044] The millimeter-wave radar detected a living object, but the camera determined it was not a child, triggering an alarm.

[0045] The millimeter-wave radar did not detect any living creature, but the camera identified it as a child and triggered an alarm.

[0046] Furthermore, the alarm information is divided into three stages: initial alarm, enhanced alarm, and mandatory intervention; specifically:

[0047] Initial alarm: If a child is detected left behind within 10 seconds of the vehicle being locked, an initial alarm will be triggered, which will be triggered by flashing hazard lights and sounding the horn for 5 seconds. During the response, the CPD system will remain in monitoring mode.

[0048] Enhanced Alarm: If a child is still detected within 60 seconds after the initial alarm is triggered, an enhanced alarm will be triggered, responding with stronger visual and auditory alarms as well as a pop-up notification on the mobile app; the enhanced alarm will last for 10 seconds and will cycle every minute up to 20 minutes.

[0049] Mandatory Intervention: If a child is still detected in the car after the enhanced alarm is triggered, mandatory intervention measures will be triggered. This is also a mandatory means to ensure the safety of children. The response method is to automatically turn on the car's air conditioning and adjust the temperature inside the car to a normal level.

[0050] The beneficial effects of this solution are: setting alarm information of different intensities to ensure the effectiveness of the alarm, and being able to take over the vehicle's system during the forced intervention phase, ensuring the safety of left-behind children by turning on the air conditioning and other means.

[0051] Furthermore, a vehicle in-car child detection system, applied to any of the above-mentioned methods for detecting children left in vehicles, uses millimeter-wave radar and a camera as direct detection devices for the vehicle CPD system, based on the existing vehicle CPD system. The millimeter-wave radar is installed in the middle of the rear roof of the vehicle, and the camera is installed in the middle of the front roof of the vehicle.

[0052] The beneficial effects of this solution are as follows: By using millimeter-wave radar and cameras as direct detection devices for the vehicle's CPD system, the in-vehicle child detection system can detect children left in the vehicle using the methods described above. The millimeter-wave radar installed in the middle of the rear roof can detect human breathing and heartbeat to determine whether there are living beings in the vehicle. The camera located in the center of the front roof can capture the visual characteristics of the occupants to determine the type of occupants. The combination of the two functions enables the detection of children left in the vehicle and minimizes false alarms while ensuring the safety of children.

[0053] Furthermore, the in-vehicle child detection system can be verified through testing scenarios based on Euro-NCAP regulations, other testing scenarios, and a matrix of reverse testing scenarios.

[0054] The beneficial effects of this solution are: the system, through all the test scenarios set, not only meets the requirements of Euro-NCAP regulations, but also exceeds them, comprehensively covering various scenarios and eliminating false alarms and omissions.

[0055] Furthermore, in the other test scenarios, the child was still left alone in different locations inside the vehicle, including the child lying flat on the back seat, the child standing or squatting in the back footwell, and the child standing by the back window.

[0056] The beneficial effects of this solution are: by testing in other set test scenarios, the system can identify more real-life situations of children in various scenarios, ensuring the comprehensiveness of the system's recognition.

[0057] Furthermore, in the reverse test scenario, the adult and child are located in different positions inside the car, or the adult is holding the child in different positions inside the car. Specifically, in the scenario where the adult and child are located in different positions inside the car, if the child is located in the left or right position of the second row, then the adult is located in the middle of the second row; if the child is located in the middle of the second row, then the adult is located in the front passenger seat.

[0058] The beneficial effects of this solution are: by testing the system in the set reverse test scenario, it can avoid false alarms when children and adults are both in the car at the same time, thus avoiding causing resentment among passengers. Attached Figure Description

[0059] Figure 1 This is a simplified flowchart of the method for detecting children left in a vehicle according to Embodiment 1 of the present invention;

[0060] Figure 2 This is an example of a skeleton point connection diagram in Embodiment 1 of the present invention;

[0061] Figure 3 This is a logical relationship diagram of alarm information in Embodiment 1 of the present invention;

[0062] Figure 4 This is a schematic diagram of the system structure in Embodiment 1 of the present invention;

[0063] Figure 5 This is a waveform example of the various information points acquired by CANoe in Embodiment 2 of the present invention.

[0064] Figure 6 This is another waveform example of the time points of various information obtained by CANoe in Embodiment 2 of the present invention. Detailed Implementation

[0065] The following detailed description illustrates the specific implementation method:

[0066] Example 1

[0067] Example 1 is basically as shown in the appendix. Figure 1-4 As shown, Figure 1 As shown, a method for detecting children left in a vehicle includes the following steps:

[0068] S1: The CPD system millimeter-wave radar emits millimeter waves at a specific carrier frequency and acquires the scattering point echo of the detected living target;

[0069] Specifically, the carrier frequency emitted by the millimeter-wave radar is waveform And the scattered point echo of living targets detected by radar It can be represented as:

[0070]

[0071]

[0072] in, Indicates in The amplitude value of the echo signal at time t, where c is the speed of light in a vacuum. Indicates the distance difference between the radar and a living target. It represents the imaginary unit of complex numbers and is used to write complex radar signals.

[0073] S2: Obtain the corresponding Doppler frequency shift based on the echo from the scattering point, perform data processing and analysis on the collected micro-Doppler signals, and extract information related to the motion of the living target;

[0074] Specifically, the Doppler frequency shift calculation process involves the phase difference between the echo signal and the transmitted signal as follows:

[0075]

[0076] Differentiating the derivative with respect to time t yields the Doppler frequency shift corresponding to the scattering point. :

[0077]

[0078] in, It represents a tiny change in phase difference and is used to calculate the Doppler frequency shift.

[0079] S3: Based on the extracted target motion-related information, generate a thermal effect map to determine whether there are any occupants left inside the vehicle;

[0080] Specifically, in the process of generating the thermal effect map, the data is converted into the form of pixels, each pixel representing a specific Doppler frequency shift, and the brightness or color of the pixel indicates the amount or intensity of the Doppler frequency shift.

[0081] S4: Based on the judgment result of step S3, the camera makes further judgments;

[0082] S5: The CPD system camera captures the visual features of the occupants and uses the OpenPose model based on the VGG-19 convolutional neural network framework to train images to obtain the pixel coordinates of human skeletons from bottom to top.

[0083] It adopts a bottom-up approach, which first detects key points of human limbs and then maps these key points to different limb information and corresponding human bodies, representing the set of skeletal point confidence and the set of skeletal point affinity, respectively. Compared with the top-down human posture recognition algorithm, it can effectively identify and detect single and multiple people, and meet the coverage of real-world in-vehicle scenarios.

[0084] Specifically, obtaining the pixel coordinates of human skeletons involves two algorithm branches. One branch is keypoint (skeleton point) confidence training, which outputs a set of coordinates and confidence scores for each human skeleton point, representing the probability that each skeleton point in the image training set is a human skeleton keypoint. The first stage of this branch generates a set of detection confidence maps.

[0085]

[0086] Another branch is keypoint (skeletal point) affinity training, which outputs the affinity between skeletal points. Connecting the skeletal keypoints with the highest affinity forms a human limb structure diagram. The first stage of this branch generates a set of affinity vectors:

[0087]

[0088] The two branches mentioned above undergo multiple iterations of training, using a loss function to calculate the loss value and provide feedback to the previous network structure. Subsequent stages iteratively update the confidence and fit values ​​from the previous stage, as shown in the following formula:

[0089]

[0090]

[0091] in, and represents the confidence and fit of the skeletal points obtained after t iterations; F represents the visual features extracted by the VGG19 network. and The function symbol is defined, and its superscript t represents the function obtained after the t-th iteration update;

[0092] After multiple iterations, the network parameters are continuously updated until the accuracy standard is met, thus obtaining the final set of skeleton point confidence scores:

[0093]

[0094] And the set of skeletal point fits used for limb connections:

[0095]

[0096] in, Represents the J elements in the confidence set. The set represents the C elements in the fit set. The set of bone point confidence and the set of bone point fit together constitute the complete human bone pixel coordinate information.

[0097] S6: Obtain the human limb size based on the pixel coordinate information of the human skeleton, fit the occupant's volume based on the human limb size, and thus complete the occupant category determination.

[0098] Specifically, the dozens of identified skeletal pixel coordinates are connected and combined to outline a complete human limb structure diagram, such as... Figure 2The image shown is an example of a human limb structure diagram drawn by skeletal points when a real child is located in the middle of the second row of a vehicle, as shown by the camera in this embodiment. In actual practice, due to the interference of in-vehicle facilities such as seat backs, headrests, and armrests on the OMS camera's line of sight, some skeletal points of the occupants may be obscured. In this case, the obscured skeletal points can be obtained by reasoning from the skeletal points of other complete limbs.

[0099] S7: Based on the judgment results of steps S3 and S6, the CPD system issues an alarm message.

[0100] Specifically, such as Figure 3 As shown, the judgment logic for the CPD system to issue alarm information is as follows:

[0101] The millimeter-wave radar detected a living creature, and the camera identified it as a child, triggering an alarm.

[0102] The millimeter-wave radar detected a living creature, and the camera identified it as an adult, so no alarm was triggered.

[0103] The millimeter-wave radar detects a living object, but the camera determines it is not a child, triggering an alarm. In this case, it may be due to a pet leaving behind the object, or the visual features of a young child may be completely obstructed when the child seat is installed rear-facing, in which case an alarm should still be triggered.

[0104] The millimeter-wave radar did not detect any living creature, but the camera identified it as a child and triggered an alarm.

[0105] The above alert information is divided into three stages: initial alert, enhanced alert, and mandatory intervention.

[0106] Initial Alarm: If a child is detected left behind within 10 seconds of the vehicle being locked, an initial alarm is triggered, which is triggered by 5 seconds of continuous hazard lights and horn blaring. During the response, the CPD system remains in monitoring mode.

[0107] Enhanced Alarm: If a child is still detected within 60 seconds of the initial alarm being triggered, an enhanced alarm will be activated, responding with stronger visual and auditory alarms as well as a pop-up notification in the mobile app. The enhanced alarm lasts for 10 seconds and cycles every minute up to 20 minutes.

[0108] Mandatory Intervention: If a child is still detected in the car after the enhanced alarm is triggered, mandatory intervention measures will be triggered. This is also a mandatory means to ensure the safety of children. The response method is to automatically turn on the car's air conditioning and adjust the temperature inside the car to a normal level.

[0109] like Figure 4As shown, a vehicle in-car child detection system, based on the existing CPD system, uses millimeter-wave radar and a camera as direct detection devices for the vehicle's CPD system, enabling the system to adopt the aforementioned method for detecting children left behind in vehicles.

[0110] By installing a millimeter-wave radar in the middle of the rear roof of the vehicle, the millimeter waves emitted can penetrate non-metallic media to reach and reflect the surface of living organisms. Even a stationary or sleeping human body will experience chest rise and fall over time through breathing and heartbeat, which is periodic over a period of time, thus generating a time-frequency domain micro-Doppler effect and producing a thermal effect map. The purpose is to determine whether there are any occupants left in the vehicle. It should be noted that in this embodiment, the millimeter-wave radar does not set thresholds for heartbeat, breathing rate, etc. to distinguish between adults and children, so as to reduce the risk of misjudgment due to similar vital signs of occupants.

[0111] By installing a camera in the front roof of the vehicle to capture the visual characteristics of the occupants, the aim is to determine the type of occupant. Working together with millimeter-wave radar, the CPD (Car Safety Device) function is realized. Based on the millimeter-wave radar's determination of whether an occupant has been left behind, the camera then captures images, enabling the camera to accurately identify children in the vehicle while minimizing false alarms.

[0112] Example 2

[0113] Based on Embodiment 1, this embodiment provides a testing method for the aforementioned in-vehicle child detection system to ensure the system's reliability. The method provides a matrix of tests covering Euro-NCAP regulatory scenarios, other scenarios, and reverse scenarios. The in-vehicle child detection system must be tested and verified through this method across all scenarios. The testing method includes the following steps:

[0114] A1. Check the vehicle status, confirm that the vehicle battery has sufficient power, and ensure that the in-vehicle CPD system is in normal working condition, including the millimeter-wave radar and OMS camera being able to continuously monitor after the vehicle is locked; to ensure the safety of the vehicle and the personal safety of the test subjects throughout the entire test process, the vehicle will be in P gear during the test.

[0115] A2, an external CANoe device is connected to the vehicle to read various information, recording messages for each test scenario and collecting key information during the test process, including vehicle locking time, door open / close status, whether the millimeter-wave radar identifies living beings, camera identification of occupant type and location, alarm trigger time, CPD system status, etc.; a computer and vehicle OBD interface are connected to record video from the camera in each test scenario using the MobaX software, facilitating the storage of video evidence and subsequent system optimization and updates; such as... Figure 5 , Figure 6As shown, this embodiment displays an example of time-point waveforms of various information acquired by CANoe during the system testing process.

[0116] A3. Place the test child and child seat in the corresponding position inside the vehicle for the test scenario. Fasten the child's seatbelt. The seat for younger children should be placed rear-facing, and the seat for older children should be placed forward-facing. Depending on the test scenario requirements, children may need to be covered with a blanket in some scenarios. For test children who are sleeping, the blanket should cover them below the shoulders, arms, and feet. For children who are not sleeping, the blanket should cover them below the chest, feet, but arms should be exposed.

[0117] A4. The driver opens the driver's side door, enters the vehicle, closes the door, and starts the vehicle to simulate the driving process. For safety reasons, the vehicle remains stationary during the actual test.

[0118] A5, the driver turns off the vehicle, opens the driver's side door and gets out of the vehicle, and then closes the driver's side door;

[0119] A6: The driver locks the vehicle and maintains a certain distance from it, waiting for the alarm to be triggered; the vehicle is locked at the zero point of the timer, at which point CANoe begins recording messages and the OMS camera begins recording video.

[0120] A7, initial alarm triggered, record the initial alarm trigger time and duration; after the initial alarm ends, continue data collection until the enhanced alarm is triggered. After several enhanced alarm cycles, intervention is triggered. The entire process lasts 20 minutes; at this time, stop CANoe message recording and OMS camera video recording, save the obtained information, and the test process for this scenario ends.

[0121] For the A8, the driver can deactivate the alarm by unlocking or opening the door and proceed to the next test scenario. The above steps are repeated until all tests based on Euro-NCAP regulations are completed.

[0122] The above steps are a detailed testing process based on Euro-NCAP regulatory scenarios. An example of a Euro-NCAP regulatory scenario is shown in Table 1 below. For other testing scenarios outside of the regulations, the tests are conducted according to Table 2 below. Unlike Euro-NCAP regulatory requirements, other testing scenarios do not require the installation of child seats, and the child's state is only static. Therefore, in step A3, the child being tested must lie flat on the rear seat, with their head facing left and right respectively, and stand (or squat) in the rear footwell and stand by the rear window.

[0123]

[0124] Table 1

[0125]

[0126] Table 2

[0127] During the reverse scenario test, the adult and child must complete the test scenario together in the car, and both the adult and the child must be stationary. Therefore, in step A3, the adult and the child must be in different positions in the car or the adult must be holding the child. The actual reverse test scenarios are shown in Table 3 below. In these scenarios, the child is assumed to be in a safe state, and the expected test result is that the CPD system does not issue an alarm.

[0128]

[0129] Table 3

[0130] The above descriptions are merely embodiments of the present invention, and common knowledge such as specific technical solutions and / or characteristics are not described in detail here. It should be noted that those skilled in the art can make various modifications and improvements without departing from the technical solution of the present invention, and these should also be considered within the scope of protection of the present invention. These modifications and improvements will not affect the effectiveness of the implementation of the present invention or the practicality of the patent. The scope of protection claimed in this application should be determined by the content of its claims, and the specific embodiments described in the specification can be used to interpret the content of the claims.

Claims

1. A method for detecting children left in a vehicle, characterized in that, Includes the following steps: S1: The millimeter-wave radar of the CPD system emits millimeter waves at a specific carrier frequency and acquires the scattering point echo of the detected living target; The carrier frequency emitted by the millimeter-wave radar is waveform And the scattered point echo of living targets detected by radar It can be represented as: in, Indicates in The amplitude value of the echo signal at time t, where c is the speed of light in a vacuum. Indicates the distance difference between the radar and a living target. It represents the imaginary unit of complex numbers and is used to write complex radar signals. S2: Obtain the corresponding Doppler frequency shift based on the echo from the scattering point, process and analyze the collected micro-Doppler signals, and extract information related to the motion of the living target. The calculation process for the Doppler frequency shift is as follows: The phase difference between the echo signal and the transmitted signal is: right By differentiating the time step, we can obtain the Doppler frequency shift corresponding to the scattering point. : in, It represents a tiny change in phase difference and is used to calculate the Doppler frequency shift; S3: Based on the extracted information related to the movement of living targets, generate a thermal effect map to determine whether there are any occupants left inside the vehicle; S4: Based on the judgment result of S3, the camera makes further judgments; S5: The CPD system's camera captures the visual features of the occupants, and uses the OpenPose model based on the VGG-19 convolutional neural network framework to train images to obtain the pixel coordinates of human skeletons. S6: Obtain the human limb size based on the pixel coordinate information of the human skeleton, fit the occupant's volume by the human limb size, and thus complete the occupant category determination. S7: Based on the judgment results of steps S3 and S6, the CPD system can issue an alarm message.

2. The method for detecting children left in a vehicle according to claim 1, characterized in that, In step S5, obtaining the pixel coordinates of human skeletons includes a first algorithm branch and a second algorithm branch. The first algorithm branch is for training the confidence of skeleton points, and the output is a set of coordinates and confidence scores of human skeleton points, representing the probability that each skeleton point in the image training set is a key point of the human skeleton. The first stage of the first algorithm branch will generate a set of detection confidence maps. The second algorithm branch trains the affinity between bone points, and the output is the compatibility between bone points. The first stage of the second algorithm branch generates a set of affinity vectors. The first and second algorithm branches undergo multiple iterations of training, using a loss function to calculate the loss value and provide feedback to the previous network structure. Subsequent stages iteratively update the confidence and fit values ​​from the previous stage, as shown in the following formula: in, and represents the confidence and fit of the skeletal points obtained after t iterations, respectively; F represents the visual features extracted by the VGG19 network. and The function symbol is defined, and its superscript t represents the function obtained after the t-th iteration update; After multiple iterations, the network parameters are continuously updated until the accuracy standard is met, thus obtaining the final set of skeleton point confidence scores: And the set of skeletal point fits used for limb connections: in, Represents the J elements in the confidence set. The set represents the C elements in the fit set. The set of bone point confidence and the set of bone point fit together constitute the complete human bone pixel coordinate information.

3. The method for detecting children left in a vehicle according to claim 1, characterized in that, In step S7, the logic for determining the alarm information issued by the CPD system is as follows: The millimeter-wave radar detected a living creature, and the camera identified it as a child, triggering an alarm. The millimeter-wave radar detected a living creature, and the camera identified it as an adult, so no alarm was triggered. The millimeter-wave radar detected a living object, but the camera determined it was not a child, triggering an alarm. The millimeter-wave radar did not detect any living creature, but the camera identified it as a child and triggered an alarm.

4. The method for detecting children left in a vehicle according to claim 3, characterized in that, The alarm information is divided into three stages: initial alarm, enhanced alarm, and mandatory intervention; specifically: Initial alarm: If a child is detected left behind within 10 seconds of the vehicle being locked, an initial alarm will be triggered, which will be triggered by flashing hazard lights and sounding the horn for 5 seconds. During the response, the CPD system will remain in monitoring mode. Enhanced Alarm: If a child is still detected within 60 seconds after the initial alarm is triggered, an enhanced alarm will be triggered, responding with stronger visual and auditory alarms as well as a pop-up notification on the mobile app; the enhanced alarm will last for 10 seconds and will cycle every minute up to 20 minutes. Mandatory Intervention: If a child is still detected in the car after the enhanced alarm is triggered, mandatory intervention measures will be triggered. This is also a mandatory means to ensure the safety of children. The response method is to automatically turn on the car's air conditioning and adjust the temperature inside the car to a normal level.

5. A child detection system left in a vehicle, characterized in that, The method for detecting children left in a vehicle as described in any one of claims 1-4 uses millimeter-wave radar and a camera as direct detection devices for the vehicle's CPD system, based on the existing vehicle CPD system. The millimeter-wave radar is installed in the middle of the rear roof of the vehicle, and the camera is installed in the middle of the front roof of the vehicle.

6. A child detection system left in a vehicle according to claim 5, characterized in that, The system can be verified through testing scenarios based on Euro-NCAP regulations, other testing scenarios, and a matrix of reverse testing scenarios.

7. A child detection system left in a vehicle according to claim 6, characterized in that, In the other test scenarios, the child was still left alone in different locations inside the car, including lying flat on the back seat, standing or squatting in the back footwell, and standing by the back window.

8. A child detection system left in a vehicle according to claim 6, characterized in that, In the reverse test scenario, the adult and child are located in different positions inside the car, or the adult is holding the child in different positions inside the car. Specifically, in the scenario where the adult and child are located in different positions inside the car, if the child is located in the left or right position of the second row, the adult is located in the middle of the second row; if the child is located in the middle of the second row, the adult is located in the front passenger seat.

Images