Early warning method and device for falling of child in vehicle and vehicle

Abstract

The invention discloses an early warning method and device for falling of a child in a vehicle and the vehicle, and relates to the technical field of vehicle safety. The method comprises the following steps: acquiring an image of a child in a vehicle by using a depth camera device, analyzing the image of the child, and identifying skeleton key points of the child; determining action data of the child according to the skeleton key points; identifying whether the action of the child belongs to a dangerous action according to the action data; and in response to the fact that the action of the child belongs to the dangerous action, early warning of falling of the child is carried out. According to the embodiment, the safety state of the child in the vehicle can be monitored in real time, the accuracy and timeliness of dangerous action recognition of the child are improved, and the attention of a driver is prevented from being distracted.

Description

Technical Field

[0001] The present invention relates to the field of vehicle safety technology, and in particular to a method and device for warning a child falling in a vehicle. Background Art

[0002] Families with children often find themselves driving with them. Children are typically seated in the back seat of a vehicle. To prevent them from falling, drivers must constantly monitor their condition and take appropriate measures. This can lead to distracted driving and potentially traffic accidents. Failure to promptly detect a child's imminent fall can put the child in danger.

[0003] Therefore, there is an urgent need for a method that can promptly identify situations where children are about to fall in a vehicle and provide an early warning. Summary of the Invention

[0004] In view of this, an embodiment of the present invention provides a method and device for warning children falling in a vehicle, and a vehicle, which can realize real-time monitoring of the safety status of children in the vehicle, improve the accuracy and timeliness of identifying dangerous actions of children, and avoid distracting the driver's attention.

[0005] To achieve the above objectives, according to one aspect of an embodiment of the present invention, a method for warning a child falling in a vehicle is provided, comprising:

[0006] Using a depth camera device to capture images of children in the vehicle, and analyzing the images of the children to identify key points of the children's skeleton;

[0007] Determine the motion data of the child based on the skeleton key points;

[0008] Identify whether the child's action is dangerous based on the action data;

[0009] In response to the above-mentioned child's action being a dangerous action, a child fall warning is issued.

[0010] Optionally, analyzing the child image to identify key skeletal points of the child includes:

[0011] The child images are analyzed using a pre-trained key point recognition model to identify the skeletal key points in the child images.

[0012] Optionally, determining the motion data of the child based on the skeleton key points includes:

[0013] Determine the two-dimensional coordinates of the above-mentioned skeletal key points in a preset two-dimensional coordinate system of the child image;

[0014] In combination with the imaging parameters of the depth camera, the two-dimensional coordinates of the key points of the skeleton are converted into three-dimensional coordinates in a preset camera coordinate system;

[0015] The motion data of the child is determined based on the three-dimensional coordinates of the skeleton key points.

[0016] Optionally, the above-mentioned camera parameters include: focal length;

[0017] The above-mentioned method of converting the two-dimensional coordinates of the above-mentioned skeleton key points into three-dimensional coordinates in a preset camera coordinate system in combination with the camera parameters of the above-mentioned depth camera device includes:

[0018] Determining the depth value of the pixel point corresponding to the two-dimensional coordinate in the child image;

[0019] According to the depth value and the focal length, the horizontal and vertical coordinates in the two-dimensional coordinates are converted into the horizontal and vertical coordinates in the three-dimensional coordinates in the camera coordinate system, and the depth value is used as the vertical coordinate in the three-dimensional coordinates.

[0020] Optionally, the motion data includes movement acceleration;

[0021] The above-mentioned determination of the child's motion data based on the three-dimensional coordinates of the above-mentioned skeletal key points includes:

[0022] Determine the target three-dimensional coordinates of the child's torso center of gravity based on the three-dimensional coordinates of the key skeletal points.

[0023] The movement acceleration of the child is calculated based on the target three-dimensional coordinates of the center of gravity of the torso corresponding to the plurality of continuously captured images of the child.

[0024] Optionally, the step of identifying whether the child's action is a dangerous action based on the action data includes:

[0025] Determining whether the movement acceleration is greater than a preset acceleration;

[0026] In response to the movement acceleration being greater than the preset acceleration, determining that the child's action is a dangerous action;

[0027] In response to the movement acceleration being less than or equal to the preset acceleration, it is determined that the child's action is not a dangerous action.

[0028] Optionally, the motion data includes shoulder joint abduction angle and knee joint angle;

[0029] The above-mentioned skeleton key points include elbow key points, shoulder key points, ankle key points, knee key points and hip key points;

[0030] The above-mentioned determination of the child's motion data based on the three-dimensional coordinates of the above-mentioned skeletal key points includes:

[0031] Determine the upper arm vector of the child according to the three-dimensional coordinates corresponding to the elbow key point and the shoulder key point;

[0032] Determining the direction of the child's trunk axis according to the shoulder key point and the hip key point;

[0033] Calculate the shoulder joint abduction angle based on the trunk axis direction through the upper arm vector;

[0034] Determining a calf vector of the child based on the three-dimensional coordinates corresponding to the ankle key point and the knee key point, and determining a thigh vector of the child based on the three-dimensional coordinates corresponding to the knee key point and the hip key point;

[0035] The knee joint angle is calculated based on the calf vector and the thigh vector.

[0036] Optionally, the step of identifying whether the child's action is a dangerous action based on the action data includes:

[0037] Determining whether the shoulder joint abduction angle is greater than a preset first angle threshold, and whether the knee joint angle is greater than a preset second angle threshold;

[0038] In response to the shoulder joint abduction angle being greater than a preset first angle threshold and the knee joint angle being greater than a preset second angle threshold, determining that the child's action is a dangerous action;

[0039] In response to the shoulder joint abduction angle being less than or equal to the preset first angle threshold, or the knee joint angle being less than or equal to the preset second angle threshold, it is determined that the child's action is not a dangerous action.

[0040] To achieve the above objectives, according to another aspect of an embodiment of the present invention, a child fall warning device in a vehicle is provided, comprising:

[0041] an analysis module, configured to capture an image of a child in a vehicle using a depth camera device, analyze the image of the child, and identify key points of the child's skeleton;

[0042] A determination module, configured to determine the motion data of the child based on the skeleton key points;

[0043] an identification module, configured to identify whether the child's action is dangerous based on the action data;

[0044] The early warning module is used to issue an early warning of a child falling in response to the above-mentioned child's action being a dangerous action.

[0045] To achieve the above-mentioned objective, according to another aspect of an embodiment of the present invention, a vehicle is provided, comprising the child fall warning device in the vehicle according to an embodiment of the present invention.

[0046] One embodiment of the above invention has the following advantages or beneficial effects: by analyzing the images of children in the vehicle captured by the depth camera device, the child's movement data is determined based on the above-mentioned skeletal key points, and then it is judged whether the child's movement is a dangerous movement. When the child's movement is identified as a dangerous movement, a child fall warning is issued. The child's image captured in real time by the depth camera device can be used to effectively identify the child's dangerous movement, realizing real-time monitoring of the safety status of children in the vehicle, which can improve the accuracy and timeliness of the identification of children's dangerous movements, improve the safety of children traveling in the vehicle, and at the same time, avoid distracting the driver's attention.

[0047] The further effects of the above-mentioned non-conventional optional manner will be described below in conjunction with specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The accompanying drawings are provided for a better understanding of the present invention and are not intended to limit the present invention.

[0049] Figure 1 is a flow chart of a method for warning a child falling in a vehicle according to an embodiment of the present invention;

[0050] Figure 2 is a schematic diagram of a process for determining child motion data according to an embodiment of the present invention;

[0051] Figure 3 is an example diagram of a two-dimensional coordinate system for a child image according to an embodiment of the present invention;

[0052] Figure 4 is a schematic diagram of a process for determining child motion data according to another embodiment of the present invention;

[0053] Figure 5 is a flow chart of a method for warning a child falling in a vehicle according to another embodiment of the present invention;

[0054] Figure 6 is a flow chart of a method for warning a child falling in a vehicle according to yet another embodiment of the present invention;

[0055] Figure 7 is a schematic diagram of main modules of a child fall warning device in a vehicle according to an embodiment of the present invention;

[0056] Figure 8 is an exemplary system architecture diagram in which embodiments of the present invention may be applied;

[0057] Figure 9 It is a structural diagram of a computer system suitable for implementing the in-vehicle child fall warning method or in-vehicle child fall warning device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0058] The following description of exemplary embodiments of the present invention is made in conjunction with the accompanying drawings, in which various details of the embodiments of the present invention are included to facilitate understanding. These details should be considered as merely exemplary. Therefore, it should be appreciated by those skilled in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0059] It should be pointed out that, in the absence of conflict, the embodiments of the present invention and the technical features therein may be combined with each other.

[0060] It should be noted that the collection, collection, updating, analysis, processing, use, transmission, and storage of user personal information involved in the technical solutions disclosed herein all comply with relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. Necessary measures are taken with respect to user personal information to prevent unauthorized access to user personal information data and to safeguard the security of user personal information, network security, and national security.

[0061] Figure 1 FIG. 1 is a schematic diagram of the main steps of the method for warning a child falling in a vehicle according to an embodiment of the present invention. Figure 1 As shown, the child fall warning method in a vehicle according to an embodiment of the present invention mainly includes the following steps S101 to S104:

[0062] Step S101, using a depth camera device to capture an image of a child in a vehicle, and analyzing the image of the child to identify key points of the child's skeleton;

[0063] A depth camera is a camera device that can obtain the depth value of an object. It can be installed at any location in the vehicle where it can capture images of children, such as the roof of the vehicle, the back of the front seat, etc.

[0064] Skeletal key points refer to representative points on a child's torso. By detecting the skeletal key points, the child's movements can be identified. For example, the skeletal key points may include wrist key points, elbow key points, shoulder key points, neck key points, ankle key points, knee key points and / or hip key points, etc., but are not limited to these.

[0065] Step S102, determining the motion data of the child based on the skeleton key points;

[0066] By analyzing one or more key points of the skeleton, the child's motion data is calculated. The motion data may include joint angles, speeds, accelerations, etc., but is not limited to these.

[0067] Step S103, identifying whether the child's action is a dangerous action based on the action data;

[0068] The range of the child's motion data during dangerous actions can be determined based on the child's historical motion data. When the child's motion data is within this range, the child may be in danger of falling from the seat; when the child's motion data is not within this data range, there is basically no danger of the child falling from the seat. Therefore, it is possible to identify whether the child's action is a dangerous action based on the child's motion data.

[0069] Alternatively, a pre-trained machine learning model can be used to identify whether a child's movements are dangerous. The pre-trained machine learning model can be a normal movement recognition model for identifying normal movements or a dangerous movement recognition model for identifying dangerous movements. Specifically, movement data can be used as input to the normal movement recognition model or the dangerous movement recognition model to identify whether the child's movements are normal or dangerous.

[0070] The normal action recognition model or the dangerous action recognition model can be trained using historical action data of children that have been labeled to indicate normal or dangerous actions. The pre-trained machine learning model can be a hidden Markov model, a Gaussian mixture model, or a neural network model, among others, without limitation.

[0071] Step S104: In response to the child's dangerous action, a child falling warning is issued.

[0072] Once a child's movements are identified as dangerous, a child fall warning is immediately issued to alert the driver or other passengers to pay attention to the child and take appropriate measures. Child fall warning methods can include audio and visual warnings, tactile warnings, and information prompts, such as beeps, voice prompts, seat vibrations, or warning messages displayed on the central control screen, so that the driver and other passengers are aware of the child's situation in a timely manner.

[0073] In an optional embodiment, the above-mentioned analyzing the above-mentioned child image and identifying the child's skeletal key points include: analyzing the above-mentioned child image through a pre-trained key point recognition model, and identifying the above-mentioned skeletal key points in the above-mentioned child image.

[0074] Among them, the training process of the key point recognition model may include: obtaining images with a specific range of key points of the child's skeleton marked, and iteratively training the big data model as a training set to obtain the key point recognition model.

[0075] During the use of the key point recognition model, the collected child images can be used as input, and the key point recognition model can be used to identify skeletal key points, so that the child images collected by the depth camera device can be analyzed in real time, and skeletal key points can be accurately, efficiently and quickly identified in the child images. There is no need for the driver or other passengers to pay attention to the child's status at all times, avoiding distracting the driver's attention while the vehicle is driving.

[0076] In an optional embodiment, as Figure 2 As shown, the above-mentioned determination of the motion data of the child based on the above-mentioned skeleton key points includes the following steps S201 to S203:

[0077] Step S201, determining the two-dimensional coordinates of the above-mentioned skeleton key points in a preset two-dimensional coordinate system of the child image;

[0078] Step S202, combining the camera parameters of the depth camera device, converting the two-dimensional coordinates of the skeleton key points into three-dimensional coordinates in a preset camera coordinate system;

[0079] Step S203: determining the motion data of the child according to the three-dimensional coordinates of the skeleton key points.

[0080] In order to accurately determine the child's motion data through the skeleton key points, the position of the skeleton key points in the child's image can be determined, the child's motion can be restored, and the child's motion data in the real scene can be determined.

[0081] For child images, a two-dimensional coordinate system of the child images is constructed, and then the two-dimensional coordinates of one or more skeletal key points in the two-dimensional coordinate system of the child images can be determined.

[0082] When constructing a two-dimensional coordinate system for a child image, any pixel point on the child image can be used as the coordinate origin. Preferably, the pixel point at the corner of the child image can be used as the coordinate origin, and the two perpendicular sides of the child image can be used as the x-axis and y-axis respectively. For example, Figure 3 As shown, the pixel point at the lower left corner of the child image can be used as the coordinate origin to determine the coordinates of the shoulder key point I (m i ,n i ), coordinates of elbow key point H (m h ,n h ), coordinates of hip key point L (m l ,n l ), coordinates of knee key point K (m k ,n k ) and the coordinates of the ankle key point J (m j ,n j ).

[0083] The camera coordinate system refers to a three-dimensional coordinate system established with the depth camera as the center. Generally, the origin is located at the optical center of the depth camera, so that the z-axis of the camera coordinate system coincides with the optical axis of the depth camera. The two perpendicular edges in the child image can be used as the x-axis and y-axis, respectively. It should be noted that, generally, the x-axis and y-axis of the camera coordinate system coincide with the corresponding edges of the x-axis and y-axis of the child image's two-dimensional coordinate system.

[0084] After determining the camera coordinate system, the two-dimensional coordinates of the skeletal key points in the two-dimensional coordinate system of the child image can be converted into three-dimensional coordinates in the camera coordinate system, so as to determine the child's motion data through the three-dimensional coordinates of the skeletal key points and realize accurate calculation of the child's motion data.

[0085] The camera parameters of the depth camera device may include focal length.

[0086] The above-mentioned camera parameters may also include principal point coordinates. The principal point coordinates represent the coordinates of the intersection of the optical axis of the depth camera device and the child image plane in a preset child image two-dimensional coordinate system. They are generally camera parameters determined during the depth camera device calibration process and can be directly obtained.

[0087] Furthermore, the above-mentioned method combines the camera parameters of the above-mentioned depth camera device to convert the two-dimensional coordinates of the above-mentioned skeletal key points into three-dimensional coordinates in a preset camera coordinate system, including: determining the depth value of the pixel point corresponding to the above-mentioned two-dimensional coordinate in the above-mentioned child image; according to the above-mentioned depth value and the above-mentioned focal length, converting the horizontal coordinate and vertical coordinate in the above-mentioned two-dimensional coordinate into the horizontal coordinate and vertical coordinate in the three-dimensional coordinate system in the above-mentioned camera coordinate system, and using the above-mentioned depth value as the vertical coordinate in the above-mentioned three-dimensional coordinate.

[0088] Specifically, the two-dimensional coordinates of the skeleton key points in the two-dimensional coordinate system of the child image are (m, n), which can be converted into three-dimensional coordinates (o, p, q) in the camera coordinate system by the following formula:

[0089]

[0090] q=d

[0091] Among them, d represents the depth value of the pixel point corresponding to the two-dimensional coordinate of the skeleton key point; f represents the focal length; m0 represents the abscissa of the principal point coordinate; n0 represents the ordinate of the principal point coordinate.

[0092] In an optional embodiment, the above-mentioned motion data may include movement acceleration.

[0093] The above-mentioned determination of the motion data of the above-mentioned child based on the three-dimensional coordinates of the above-mentioned skeletal key points includes: determining the target three-dimensional coordinates corresponding to the center of gravity of the child's trunk based on the three-dimensional coordinates of the above-mentioned skeletal key points; and calculating the movement acceleration of the above-mentioned child based on the target three-dimensional coordinates of the center of gravity of the trunk corresponding to multiple continuously captured images of the child.

[0094] Specifically, the midpoint of the line connecting the 3D coordinates of any two skeletal keypoints can be used as the target 3D coordinate of the torso center of gravity. For example, when the skeletal keypoints include a shoulder keypoint and a hip keypoint, the midpoint of the line connecting the 3D coordinates of the shoulder keypoint and the 3D coordinates of the hip keypoint can be used as the target 3D coordinate of the torso center of gravity.

[0095] The movement acceleration of the child is calculated based on the target three-dimensional coordinates of the center of gravity of the torso corresponding to multiple consecutive images of the child. Assume that the target three-dimensional coordinates at time t are (u, v, w); the target three-dimensional coordinates at time t-Δt are (u0, v0, w0); and the target three-dimensional coordinates at time t+Δt are (u1, v1, w1). Among them, Δt represents the time interval between consecutive shots of the child's images. Specifically,

[0096] For the horizontal coordinate u in the target three-dimensional coordinates (u, v, w), the velocity in the x direction is calculated using the following formula:

[0097]

[0098] Among them, v u (t) represents the velocity of the center of gravity of the trunk in the x direction at time t.

[0099] According to the above formula, the velocity sequence of the center of gravity of the trunk at different times in the x direction can be calculated {…v u (t-Δt), v u (t), v u (t+Δt)…}, and then calculate the acceleration in the x direction using the following formula:

[0100]

[0101] Among them, a u (t) represents the acceleration of the center of gravity of the trunk in the x direction at time t.

[0102] Similarly, the same method is used to calculate the acceleration a of the center of gravity of the trunk in the y direction at time t v (t) and the acceleration a in the z direction w (t). Then, the acceleration vector of the center of gravity of the torso can be determined

[0103] According to the above-mentioned acceleration vector of the trunk center of gravity, the acceleration of the trunk center of gravity at time t is calculated by the following formula:

[0104]

[0105] Among them, a(t) represents the acceleration of the center of gravity of the torso at time t.

[0106] In an optional embodiment, the above-mentioned identification of whether the child's action is a dangerous action based on the above-mentioned action data includes: judging whether the above-mentioned movement acceleration is greater than the preset acceleration; in response to the above-mentioned movement acceleration being greater than the above-mentioned preset acceleration, determining that the child's action is a dangerous action; in response to the above-mentioned movement acceleration being less than or equal to the above-mentioned preset acceleration, determining that the child's action is not a dangerous action.

[0107] A preset acceleration can be set in advance, and whether the child's action is a dangerous action is determined based on the comparison result of the movement acceleration of the center of gravity of the torso and the preset acceleration.

[0108] For example, based on a child's historical acceleration, it can be determined that the acceleration of a child's movements during normal activities is generally less than 0.8g, while the acceleration of a child's movements during dangerous activities is generally greater than 1.5g. Therefore, the preset acceleration can be set to 1.5g. Once the acceleration of the torso center of gravity is detected to be greater than 1.5g, the child's movement can be determined to be dangerous; once the acceleration of the torso center of gravity is detected to be less than or equal to 1.5g, the child's movement can be determined to be not dangerous.

[0109] In an optional embodiment, the motion data includes a shoulder abduction angle and a knee joint angle. In this case, the skeletal key points may include an elbow key point, a shoulder key point, an ankle key point, a knee key point, and a hip key point. The child's shoulder abduction angle and knee joint angle can be determined using the skeletal key points.

[0110] Specifically, if Figure 4 As shown, the above-mentioned determination of the motion data of the child according to the three-dimensional coordinates of the above-mentioned skeleton key points includes the following steps S401 to S404:

[0111] Step S401, determining the upper arm vector of the child based on the three-dimensional coordinates corresponding to the elbow key point and the shoulder key point;

[0112] Step S402, determining the trunk axis direction of the child according to the shoulder key point and the hip key point, and calculating the shoulder joint abduction angle according to the upper arm vector based on the trunk axis direction;

[0113] The shoulder abduction angle can be calculated by the three-dimensional coordinates (x h ,y h ,z h ), the three-dimensional coordinates of the shoulder key point I (x i ,y i ,z i ) and the three-dimensional coordinates of the hip key point L (x l ,y l ,z l Specifically, we can first calculate the shoulder key point I based on its three-dimensional coordinates (x i ,y i ,z i ) and the three-dimensional coordinates of the hip key point L (x l ,y l ,z l ) determines the trunk vector indicating the direction of the trunk axis Normalized vector As shown below:

[0114]

[0115] Then, the three-dimensional coordinates (x h ,y h ,z h ) and the three-dimensional coordinates of the shoulder key point I (x i ,y i ,z i ) can determine that the arm vector can be Based on the above trunk axis direction, the shoulder joint abduction angle can be calculated using the above upper arm vector using the following formula.

[0116]

[0117] Step S403: determining the child's calf vector based on the three-dimensional coordinates corresponding to the ankle key point and the knee key point, and determining the child's thigh vector based on the three-dimensional coordinates corresponding to the knee key point and the hip key point;

[0118] Step S404, calculating the knee joint angle according to the calf vector and the thigh vector;

[0119] The knee joint angle can be expressed by the three-dimensional coordinates (x j ,y j ,z j ), the three-dimensional coordinates of the knee key point K (x k ,y k ,z k ) and the three-dimensional coordinates of the hip key point L (x l ,yl ,z l Specifically, according to the three-dimensional coordinates (x j ,y j ,z j ) and the three-dimensional coordinates of the knee key point K (x k ,y k ,z k ) can determine the calf vector According to the three-dimensional coordinates of the knee key point K (x k ,y k ,z k ) and the three-dimensional coordinates of the hip key point L (x l ,y l ,z l ) can determine the thigh vector The knee joint angle is the calf vector and thigh vector The angle β can be calculated by the following formula.

[0120]

[0121] By using precisely quantifiable shoulder abduction and knee angles as motion data to determine whether a child's movements are dangerous, the system can objectively determine whether a child's movements are dangerous, eliminating the influence of subjective factors of the driver or other passengers. Furthermore, the system can analyze the child's shoulder abduction and knee angles in real time using collected images of the child, allowing for timely detection of any dangerous movements. Furthermore, the comprehensive use of shoulder abduction and knee angles as motion data avoids focusing solely on a single joint on the child's torso while ignoring dangerous movements of the torso as a whole.

[0122] In an optional embodiment, the above-mentioned identification of whether the child's action is a dangerous action based on the above-mentioned action data includes: judging whether the above-mentioned shoulder joint abduction angle is greater than a preset first angle threshold, and whether the above-mentioned knee joint angle is greater than a preset second angle threshold; in response to the above-mentioned shoulder joint abduction angle being greater than the preset first angle threshold and the above-mentioned knee joint angle being greater than the preset second angle threshold, determining that the child's action is a dangerous action; in response to the above-mentioned shoulder joint abduction angle being less than or equal to the above-mentioned preset first angle threshold, or the above-mentioned knee joint angle being less than or equal to the above-mentioned preset second angle threshold, determining that the child's action is not a dangerous action.

[0123] A preset first angle threshold associated with the shoulder abduction angle and a preset second angle threshold associated with the knee joint angle may be preset. As an example, based on the child's historical shoulder abduction angle and historical knee joint angle, the preset first angle threshold may be preset to 45° and the preset second angle threshold may be preset to 160°. In this case, if the shoulder abduction angle is greater than 45° and the knee joint angle is greater than 160°, the child's action is determined to be dangerous. If the shoulder abduction angle is less than or equal to 45°, or the knee joint angle is less than or equal to 160°, the child's action is determined not to be dangerous.

[0124] Furthermore, in order to avoid frequent false alarms, a warning condition can be pre-set for the number of consecutive recognition frames of child images in which the shoulder abduction angle is greater than the preset first angle threshold and the knee joint angle is greater than the preset second angle threshold. Specifically, the warning frame number threshold can be pre-set, and when the number of consecutive recognition frames of child images in which the shoulder abduction angle is greater than the preset first angle threshold and the knee joint angle is greater than the preset second angle threshold reaches the warning frame number threshold, a child fall warning is issued; if the number of consecutive recognition frames of child images in which the shoulder abduction angle is greater than the preset first angle threshold and the knee joint angle is greater than the preset second angle threshold does not reach the warning frame number threshold, the child fall warning can be temporarily not issued. As an example, the warning frame number threshold can be set to 3 frames. When the shoulder joint abduction angle is greater than the preset first angle threshold and the knee joint angle is greater than the preset second angle threshold in 3 consecutive frames of child images, a child fall warning is issued; if the shoulder joint abduction angle is greater than the preset first angle threshold and the knee joint angle is greater than the preset second angle threshold in only two consecutive frames of child images, and the shoulder joint abduction angle in the third frame of the child image is less than or equal to the preset first angle threshold or the knee joint angle is less than or equal to the preset second angle threshold, no child fall warning is issued.

[0125] According to the method for warning children falling in vehicles in an embodiment of the present invention, the images of children in the vehicle captured by the depth camera device are analyzed, the movement data of the children are determined based on the above-mentioned skeletal key points, and then it is judged whether the children's movements are dangerous movements. When the children's movements are identified as dangerous movements, a child fall warning is performed. The images of children captured in real time by the depth camera device can be used to effectively identify the children's dangerous movements, thereby realizing real-time monitoring of the safety status of children in the vehicle, improving the accuracy and timeliness of identifying children's dangerous movements, and improving the safety of children traveling in vehicles. At the same time, it can also avoid distracting the driver's attention.

[0126] The following describes a method for warning a child falling in a vehicle through a specific embodiment.

[0127] like Figure 5As shown, the child fall warning method in a vehicle according to an embodiment of the present invention may include the following steps S501 to S508:

[0128] Step S501: using a depth camera to capture an image of a child in a vehicle, and analyzing the image of the child using a pre-trained key point recognition model to identify the child's skeletal key points, wherein the skeletal key points include shoulder key points and hip key points;

[0129] Step S502, determining the two-dimensional coordinates of the shoulder key point and the hip key point in the two-dimensional coordinate system of the child image;

[0130] Step S503: Determine the focal length of the depth camera device, the coordinates of the principal point, and the depth values ​​of the two-dimensional coordinates of the shoulder key point and the two-dimensional coordinates of the hip key point, and convert the two-dimensional coordinates of the shoulder key point and the two-dimensional coordinates of the hip key point into three-dimensional coordinates in the coordinate system of the camera device;

[0131] Step S504: taking the midpoint of the line connecting the three-dimensional coordinates of the shoulder key point and the three-dimensional coordinates of the hip key point as the target three-dimensional coordinates of the center of gravity of the trunk;

[0132] Step S505, calculating the movement acceleration of the child based on the target three-dimensional coordinates of the center of gravity of the torso corresponding to the plurality of consecutively captured images of the child;

[0133] Step S506, determining whether the movement acceleration is greater than a preset acceleration; in response to the movement acceleration being greater than the preset acceleration, executing step S507; in response to the movement acceleration being less than or equal to the preset acceleration, executing step S508;

[0134] Step S507: determining that the child's action is dangerous and issuing a child fall warning;

[0135] Step S508: Determine that the child's action is not a dangerous action, and end the current process.

[0136] According to the method for warning a child falling in a vehicle according to an embodiment of the present invention, the image of the child in the vehicle captured by the depth camera device is analyzed, the movement acceleration of the child is determined based on the above-mentioned skeletal key points, and then it is judged whether the child's action is a dangerous action. When the child's action is identified as a dangerous action, a child fall warning is performed. The child's images continuously captured by the depth camera device can be used to identify the child's movement acceleration, effectively identify the child's dangerous actions, and realize real-time monitoring of the safety status of children in the vehicle. The accuracy and timeliness of the identification of children's dangerous actions can be improved, and the safety of children traveling in the vehicle is improved. At the same time, it can also avoid distracting the driver's attention.

[0137] The following describes a method for warning a child falling in a vehicle through another specific embodiment.

[0138] like Figure 6 As shown, the child fall warning method in a vehicle according to an embodiment of the present invention may include the following steps S601 to S608:

[0139] Step S601, using a depth camera device to capture an image of a child in a vehicle, and analyzing the image of the child using a pre-trained key point recognition model to identify the child's skeletal key points; wherein the skeletal key points include elbow key points, shoulder key points, ankle key points, knee key points, and hip key points;

[0140] Step S602, respectively determining the two-dimensional coordinates of the elbow key point, the shoulder key point, the ankle key point, the knee key point, and the hip key point in the two-dimensional coordinate system of the child image;

[0141] Step S603, determining the focal length of the depth camera device, the principal point coordinates, and the depth value of the two-dimensional coordinates of each skeleton key point, and converting the two-dimensional coordinates of each skeleton key point into three-dimensional coordinates in the camera device coordinate system;

[0142] Step S604: determining the upper arm vector of the child using the three-dimensional coordinates corresponding to the elbow key point and the shoulder key point, determining the direction of the child's trunk axis based on the shoulder key point and the hip key point, and calculating the shoulder joint abduction angle based on the upper arm vector and the trunk axis direction;

[0143] Step S605: determining the child's calf vector based on the three-dimensional coordinates corresponding to the ankle key point and the knee key point, and determining the child's thigh vector based on the three-dimensional coordinates corresponding to the knee key point and the hip key point; and calculating the knee joint angle based on the calf vector and the thigh vector.

[0144] Step S606, determining whether the shoulder joint abduction angle is greater than a preset first angle threshold, and whether the knee joint angle is greater than a preset second angle threshold; in response to the shoulder joint abduction angle being greater than the preset first angle threshold and the knee joint angle being greater than the preset second angle threshold, executing step S607; in response to the shoulder joint abduction angle being less than or equal to the preset first angle threshold, or the knee joint angle being less than or equal to the preset second angle threshold, executing step S608;

[0145] Step S607: determining that the child's action is dangerous and issuing a child fall warning;

[0146] Step S608: Determine that the child's action is not a dangerous action, and end the current process.

[0147] According to the method for warning children falling in vehicles according to an embodiment of the present invention, the image of the child in the vehicle captured by the depth camera device is analyzed, the shoulder abduction angle and knee joint angle of the child are determined according to the above-mentioned skeletal key points, and then it is judged whether the child's action is a dangerous action. When the child's action is identified as a dangerous action, a child fall warning is performed. The image of the child captured in real time by the depth camera device can be used to effectively identify the child's dangerous actions, thereby realizing real-time monitoring of the safety status of children in the vehicle, which can improve the accuracy and timeliness of the identification of children's dangerous actions, improve the safety of children traveling in the vehicle, and at the same time, avoid distracting the driver's attention.

[0148] Figure 7 FIG is a schematic diagram of the main modules of the child fall warning device in a vehicle according to an embodiment of the present invention. Figure 7 As shown, the child fall warning device 700 in a vehicle of an embodiment of the present invention includes: an analysis module 701, which is used to collect images of children in the vehicle using a depth camera device, and analyze the above-mentioned child images to identify the child's skeletal key points; a determination module 702, which is used to determine the motion data of the above-mentioned child based on the above-mentioned skeletal key points; an identification module 703, which is used to identify whether the above-mentioned child's motion is a dangerous motion based on the above-mentioned motion data; and a warning module 704, which is used to issue a child fall warning in response to the above-mentioned child's motion being a dangerous motion.

[0149] In an optional embodiment of the present invention, the above-mentioned analysis module 701 is further used to: analyze the above-mentioned child image through a pre-trained key point recognition model, and identify the above-mentioned skeletal key points in the above-mentioned child image.

[0150] In an optional embodiment of the present invention, the above-mentioned determination module 702 is also used to: determine the two-dimensional coordinates of the above-mentioned skeletal key points in a preset two-dimensional coordinate system of the child image; combine the camera parameters of the above-mentioned depth camera device to convert the two-dimensional coordinates of the above-mentioned skeletal key points into three-dimensional coordinates in a preset camera device coordinate system; and determine the motion data of the above-mentioned child based on the three-dimensional coordinates of the above-mentioned skeletal key points.

[0151] In an optional embodiment of the present invention, the camera parameters include focal length. The determination module 702 is further configured to determine a depth value of a pixel point in the child image corresponding to the two-dimensional coordinates; and, based on the depth value and the focal length, convert the horizontal and vertical coordinates of the two-dimensional coordinates into horizontal and vertical coordinates of three-dimensional coordinates in the camera device coordinate system, respectively, and use the depth value as the vertical coordinate of the three-dimensional coordinates.

[0152] In an optional embodiment of the present invention, the above-mentioned determination module 702 is also used to: determine the target three-dimensional coordinates corresponding to the center of gravity of the child's trunk based on the three-dimensional coordinates of the above-mentioned skeletal key points; and calculate the movement acceleration of the child based on the target three-dimensional coordinates of the center of gravity of the trunk corresponding to multiple continuously captured images of the child.

[0153] In an optional embodiment of the present invention, the above-mentioned identification module 703 is also used to: determine whether the above-mentioned movement acceleration is greater than the preset acceleration; in response to the above-mentioned movement acceleration being greater than the above-mentioned preset acceleration, determine that the above-mentioned child's action is a dangerous action; in response to the above-mentioned movement acceleration being less than or equal to the above-mentioned preset acceleration, determine that the above-mentioned child's action is not a dangerous action.

[0154] In an optional embodiment of the present invention, the above-mentioned motion data includes a shoulder joint abduction angle and a knee joint angle; the above-mentioned skeletal key points include an elbow key point, a shoulder key point, an ankle key point, a knee key point and a hip key point.

[0155] The above-mentioned determination module 702 is also used to: determine the upper arm vector of the above-mentioned child according to the three-dimensional coordinates corresponding to the above-mentioned elbow key point and the above-mentioned shoulder key point; determine the trunk axis direction of the above-mentioned child according to the above-mentioned shoulder key point and the above-mentioned hip key point, and calculate the above-mentioned shoulder joint abduction angle according to the above-mentioned upper arm vector based on the above-mentioned trunk axis direction; determine the calf vector of the above-mentioned child according to the three-dimensional coordinates corresponding to the above-mentioned ankle key point and the above-mentioned knee key point, and determine the thigh vector of the above-mentioned child according to the three-dimensional coordinates corresponding to the above-mentioned knee key point and the above-mentioned hip key point; calculate the above-mentioned knee joint angle according to the above-mentioned calf vector and the above-mentioned thigh vector.

[0156] In an optional embodiment of the present invention, the above-mentioned identification module 703 is also used to: determine whether the above-mentioned shoulder joint abduction angle is greater than a preset first angle threshold, and whether the above-mentioned knee joint angle is greater than a preset second angle threshold; in response to the above-mentioned shoulder joint abduction angle being greater than the preset first angle threshold and the above-mentioned knee joint angle being greater than the preset second angle threshold, determine that the above-mentioned child's action is a dangerous action; in response to the above-mentioned shoulder joint abduction angle being less than or equal to the above-mentioned preset first angle threshold, or the above-mentioned knee joint angle being less than or equal to the above-mentioned preset second angle threshold, determine that the above-mentioned child's action is not a dangerous action.

[0157] According to the child fall warning device in the vehicle of the embodiment of the present invention, by analyzing the image of the child in the vehicle captured by the depth camera device, the child's movement data is determined according to the above-mentioned skeletal key points, and then it is judged whether the child's movement is a dangerous movement. When the child's movement is identified as a dangerous movement, a child fall warning is issued. The child's image captured in real time by the depth camera device can be used to effectively identify the child's dangerous movement, thereby realizing real-time monitoring of the safety status of children in the vehicle, which can improve the accuracy and timeliness of the identification of children's dangerous movements, avoid distracting the driver's attention, and at the same time, improve the safety of children traveling in the vehicle.

[0158] The following describes the technical scenarios to which the technical solutions provided by the embodiments of the present invention are applicable based on the system architecture on which the technical solutions provided by the embodiments of the present invention rely.

[0159] Figure 8 FIG. 8 shows an exemplary system architecture 800 to which the in-vehicle child fall warning method or in-vehicle child fall warning device according to an embodiment of the present invention can be applied. Figure 8 As shown, the vehicle system architecture 800 may include various systems, such as a driving control system 801, a power system 802, a sensor system 803, a control system 804, one or more peripheral devices 805, a power supply 806, a computer system 807, and a user interface 808. The vehicle control method provided in the embodiment of the present invention may be implemented by interacting with each of the above systems, or by controlling the above systems through external devices or by operating the above systems through a robot driving the vehicle. Optionally, the vehicle system architecture 800 may include more or fewer systems, and each system may include multiple components. In addition, each system and component of the vehicle system architecture 800 may be interconnected by wire or wirelessly.

[0160] The vehicle system architecture 800 includes a driving control system 801, which can be in a fully or partially autonomous driving mode or controlled by the driver's operation of the steering wheel, clutch, accelerator, etc. For example, the driving control system 801 can automatically control the vehicle's driving based on control signals or control instructions without human interaction or through interaction with external devices or a robot driving the vehicle.

[0161] The power system 802 may include components that provide power and movement for the vehicle. For example, the power system 802 may include an engine, an energy source, a transmission, wheels, tires, etc. The engine may be an internal combustion engine, an electric motor, an air compression engine, or a combination of other types of engines, such as a hybrid engine consisting of a gasoline engine and an electric motor, or a hybrid engine consisting of an internal combustion engine and an air compression engine. The engine converts the energy source into mechanical energy and provides it to the transmission. Examples of energy sources may include gasoline, diesel, other petroleum-based fuels, propane, other compressed gas-based fuels, ethanol, solar panels, batteries, and other sources of electricity. The energy source may also provide energy to other systems of the vehicle. In addition, the transmission may include a gearbox, a differential, a drive shaft, a clutch, etc.

[0162] The sensor system 803 may include sensors for sensing the surrounding environment of the vehicle (such as sensors for sensing whether there are targets around, etc.). For example, a positioning system (the positioning system may be a global positioning system (GPS) system, or a BeiDou system or other positioning systems), a radar sensor, an ultrasonic sensor, a laser rangefinder, an inertial measurement unit (IMU), and an image sensor. The positioning system can be used to locate the geographic location of the vehicle. The IMU is used to sense the position and orientation changes of the vehicle based on inertial acceleration. In one embodiment, the IMU may be a combination of an accelerometer and a gyroscope. The radar sensor may use millimeter wave signals to sense objects in the surrounding environment of the vehicle. In some embodiments, in addition to sensing objects, the radar sensor may also be used to sense the speed and / or direction of travel of the object.

[0163] Image sensors can be used to detect information and objects inside or outside a vehicle. To detect environmental information and objects outside the vehicle, image sensors can be placed at appropriate locations outside the vehicle. To detect occupants inside the vehicle, image sensors can be placed at appropriate locations inside the vehicle. Image sensors can be either still or video cameras. Furthermore, image sensors can include depth sensors.

[0164] The control system 804 may include software systems for implementing vehicle driving control, such as a system for analyzing the vehicle's surroundings, a system for pre-tensioning seatbelts, a system for route planning, a system for avoiding obstacles, and a vision system for image analysis. The control system 804 may also include hardware systems such as a throttle, a steering wheel system, a seatbelt system, an airbag system, and peripheral devices (such as projection equipment and displays). Furthermore, the control system 804 may include additional or alternative components beyond those shown and described. Alternatively, some of the components shown above may be reduced.

[0165] Furthermore, the control system 804 may also include a child fall warning device in a vehicle, which uses a depth camera device to capture images of children in the vehicle, and analyzes the above-mentioned child images to identify the child's skeletal key points; determines the above-mentioned child's motion data based on the above-mentioned skeletal key points; identifies whether the above-mentioned child's motion is a dangerous motion based on the above-mentioned motion data; and issues a child fall warning in response to the above-mentioned child's motion being a dangerous motion.

[0166] In addition, the control system 804 can also interact with external sensors, other autonomous driving devices, other computer systems, or users through peripheral devices 805. Peripheral devices 805 may include wireless communication systems, onboard computers, microphones and / or speakers, cameras, and projectors.

[0167] In some embodiments, peripheral devices 805 provide a means for a user of control system 804 to interact with a user interface. For example, an onboard computer can provide information to the user of the vehicle. The user interface can also operate the onboard computer to receive user input. The onboard computer can be operated via a touch screen. In other cases, peripheral devices can provide a means for communicating with other devices located within the vehicle. For example, a microphone can receive audio (e.g., voice commands or other audio input) from the user of the control system. Similarly, a speaker can output audio to the user of the control system.

[0168] A wireless communication system can communicate wirelessly with one or more devices directly or via a communication network. For example, a wireless communication system can communicate using a cellular network, WiFi, or wireless local area network (WLAN), or can directly communicate with devices using infrared links, Bluetooth, or ZigBee. Other wireless protocols, such as various autonomous driving communication systems, are also used.

[0169] The power supply 806 can provide power to various components of the vehicle. The power supply 806 can be a rechargeable lithium-ion or lead-acid battery.

[0170] Some or all of the functions implementing the child fall warning system in the vehicle are controlled by computer system 807. Computer system 807 may include at least one processor that executes instructions stored in a non-transitory computer-readable medium such as a memory. Computer system 807 provides the control system with executable code that implements active safety function adjustments.

[0171] The processor can be any conventional processor, such as a commercially available central processing unit (CPU). Alternatively, the processor can be a dedicated device such as an application specific integrated circuit (ASIC) or other hardware-based processor. Those skilled in the art will appreciate that the processor, computer, or memory can actually include multiple processors, computers, or memories that may or may not be stored in the same physical housing. For example, the memory can be a hard drive or other storage medium located in a housing different from the computer. Therefore, references to a processor or computer will be understood to include references to a collection of processors or computers or memories that may or may not operate in parallel. Different from using a single processor to perform the steps described herein, some components such as a steering assembly and a deceleration assembly can each have their own processor, which only performs determinations related to the functions specific to the component.

[0172] The user interface 808 is used to provide information to or receive information from a user of the vehicle. Optionally, the user interface 808 may include one or more input / output devices within the set of peripheral devices 805, such as a wireless communication system, an onboard computer, a microphone, and a speaker.

[0173] It should be understood that the above components are only examples. In actual applications, components in the above modules or systems may be added or deleted according to actual needs. Figure 8 It should not be understood as limiting the embodiments of the present application.

[0174] Reference below Figure 9 , which shows a structural diagram of a computer system 900 suitable for implementing the in-vehicle child fall warning method or in-vehicle child fall warning device according to an embodiment of the present invention. Figure 9 The terminal device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0175] like Figure 9 As shown, the computer system 900 includes a central processing unit (CPU) 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage unit 908 into a random access memory (RAM) 903. Various programs and data required for the operation of the computer system 900 are also stored in the RAM 903. The CPU 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0176] The following components are connected to the I / O interface 905: an input section 906 including a keyboard, a mouse, and the like; an output section 907 including devices such as a cathode ray tube (CRT), a liquid crystal display (LCD), and a speaker; a storage section 908 including a hard disk and the like; and a communication section 909 including a network interface card such as a LAN card or a modem. The communication section 909 performs communication processing via a network such as the Internet. A drive 910 is also connected to the I / O interface 905 as needed. A removable medium 911, such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, is installed in the drive 910 as needed, so that computer programs read therefrom can be installed into the storage section 908 as needed.

[0177] In particular, according to the embodiments disclosed in the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, the embodiments disclosed in the present invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program comprising program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 909, and / or installed from a removable medium 911. When the computer program is executed by the central processing unit (CPU) 901, the above-mentioned functions defined in the system of the present invention are performed.

[0178] It should be noted that the computer-readable medium described in the present invention can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media can include, but are not limited to, an electrical connection having one or more conductors, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In the present invention, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. This propagated data signal can take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. Program code embodied on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wireline, optical fiber cable, RF, or any suitable combination thereof.

[0179] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the above-mentioned module, program segment, or a part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0180] The modules involved in the embodiments of the present invention may be implemented in software or hardware. The modules described may also be provided in a processor. For example, they may be described as follows: a processor including an analysis module, a determination module, an identification module, and an early warning module. The names of these modules do not, in some cases, constitute limitations on the modules themselves. For example, the analysis module may also be described as a module that "captures images of children in a vehicle using a depth camera device, analyzes the images of the children, and identifies key skeletal points of the children."

[0181] As another aspect, the present invention further provides a computer-readable medium, which may be included in the device described in the above embodiment; or it may exist independently and not be assembled into the device. The computer-readable medium carries one or more programs, and when the one or more programs are executed by the device, the device includes: using a depth camera device to capture an image of a child in a vehicle, and analyzing the image of the child to identify the child's skeletal key points; determining the child's motion data based on the skeletal key points; identifying whether the child's motion is a dangerous motion based on the motion data; and issuing a child fall warning in response to the child's motion being a dangerous motion.

[0182] According to the technical solution of the embodiment of the present invention, by analyzing the images of children in the vehicle captured by the depth camera device, the child's movement data is determined based on the above-mentioned skeletal key points, and then it is judged whether the child's movement is a dangerous movement. When the child's movement is identified as a dangerous movement, a child fall warning is issued. The child's image captured in real time by the depth camera device can be used to effectively identify the child's dangerous movement, realizing real-time monitoring of the safety status of children in the vehicle, which can improve the accuracy and timeliness of the identification of children's dangerous movements, avoid distracting the driver's attention, and at the same time, improve the safety of children traveling in vehicles.

[0183] The above specific embodiments do not limit the scope of protection of the present invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may occur depending on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention are intended to be included within the scope of protection of the present invention.

Claims

1. A method for warning a child falling in a vehicle, characterized in that: include: Using a depth camera device to capture an image of a child in a vehicle, and analyzing the image of the child to identify key points of the child's skeleton; Determining the motion data of the child according to the skeleton key points; identifying whether the child's action is a dangerous action based on the action data; In response to the child's action being a dangerous action, a child fall warning is issued.

2. The method for warning a child falling in a vehicle according to claim 1, characterized in that: Analyzing the child image to identify key skeletal points of the child includes: The child image is analyzed using a pre-trained key point recognition model to identify the skeletal key points in the child image.

3. The method for warning a child falling in a vehicle according to claim 1, characterized in that: Determining the motion data of the child according to the skeleton key points includes: Determining the two-dimensional coordinates of the skeleton key points in a preset two-dimensional coordinate system of the child image; In combination with the imaging parameters of the depth camera device, the two-dimensional coordinates of the skeleton key points are converted into three-dimensional coordinates in a preset camera device coordinate system; The motion data of the child is determined according to the three-dimensional coordinates of the skeleton key points.

4. The method for warning a child falling in a vehicle according to claim 3, characterized in that: The camera parameters include: focal length; The step of converting the two-dimensional coordinates of the skeleton key points into three-dimensional coordinates in a preset camera coordinate system in combination with the camera parameters of the depth camera device includes: Determining a depth value of a pixel point in the child image corresponding to the two-dimensional coordinate; According to the depth value and the focal length, the horizontal coordinate and the vertical coordinate in the two-dimensional coordinate are respectively converted into the horizontal coordinate and the vertical coordinate in the three-dimensional coordinate system of the camera device, and the depth value is used as the vertical coordinate in the three-dimensional coordinate.

5. The method for warning a child falling in a vehicle according to claim 3, characterized in that: The motion data includes movement acceleration; The determining of the motion data of the child according to the three-dimensional coordinates of the skeleton key points includes: Determining the target three-dimensional coordinates of the child's torso center of gravity according to the three-dimensional coordinates of the skeleton key points; The movement acceleration of the child is calculated based on the target three-dimensional coordinates of the center of gravity of the torso corresponding to the plurality of continuously captured images of the child.

6. The method for warning a child falling in a vehicle according to claim 5, characterized in that: The identifying, based on the motion data, whether the child's motion is a dangerous motion includes: Determining whether the movement acceleration is greater than a preset acceleration; In response to the movement acceleration being greater than the preset acceleration, determining that the child's action is a dangerous action; In response to the movement acceleration being less than or equal to the preset acceleration, it is determined that the action of the child is not a dangerous action.

7. The method for warning a child falling in a vehicle according to claim 3, characterized in that: The motion data includes shoulder joint abduction angle and knee joint angle; The skeleton key points include elbow key points, shoulder key points, ankle key points, knee key points and hip key points; The determining of the motion data of the child according to the three-dimensional coordinates of the skeleton key points includes: Determine the upper arm vector of the child according to the three-dimensional coordinates corresponding to the elbow key point and the shoulder key point respectively; Determining the child's trunk axis direction according to the shoulder key point and the hip key point; Calculating the shoulder joint abduction angle based on the trunk axis direction through the upper arm vector; Determining a calf vector of the child based on the three-dimensional coordinates corresponding to the ankle key point and the knee key point, and determining a thigh vector of the child based on the three-dimensional coordinates corresponding to the knee key point and the hip key point; The knee joint angle is calculated according to the shank vector and the thigh vector.

8. The method for warning a child falling in a vehicle according to claim 7, characterized in that: The identifying, based on the motion data, whether the child's motion is a dangerous motion includes: Determining whether the shoulder joint abduction angle is greater than a preset first angle threshold, and whether the knee joint angle is greater than a preset second angle threshold; In response to the shoulder joint abduction angle being greater than the preset first angle threshold and the knee joint angle being greater than the preset second angle threshold, determining that the child's action is a dangerous action; In response to the shoulder joint abduction angle being less than or equal to the preset first angle threshold, or the knee joint angle being less than or equal to the preset second angle threshold, it is determined that the child's action is not a dangerous action.

9. A child fall warning device in a vehicle, characterized in that: include: An analysis module, configured to capture an image of a child in a vehicle using a depth camera device, analyze the image of the child, and identify key points of the child's skeleton; A determination module, configured to determine the motion data of the child based on the skeleton key points; an identification module, configured to identify whether the child's action is a dangerous action based on the action data; The early warning module is used to issue an early warning of a child falling in response to the child's action being a dangerous action.

10. A vehicle, characterized in that: The device comprises the child fall warning device in a vehicle as claimed in claim 9.

Images