Full lying posture determination method and device, electronic equipment, storage medium and vehicle

Abstract

The invention relates to a full-lying posture determination method and device, electronic equipment, a storage medium and a vehicle, and the method comprises the steps: carrying out the recognition of a target human body in a to-be-recognized human body image based on a human body key point detection network, and obtaining a plurality of crotch key points and the confidence degrees of the crotch key points, the to-be-recognized human body image is obtained according to an image of a cabin internal environment shot by a camera in the vehicle; based on the confidence coefficients of the plurality of crotch key points, determining the crotch key points of which the confidence coefficients are greater than or equal to a corresponding crotch confidence coefficient threshold in the plurality of crotch key points as at least one credible crotch key point; under the condition that the at least one credible crotch key point and the crotch point distribution domain in the full-lying posture meet a preset relation, it is determined that the target human body is in the full-lying posture; wherein the preset relation is used for indicating that an overlapping area exists between the area with at least one credible crotch key point as the vertex and the crotch point distribution area in the full-lying posture.

Description

Technical Field

[0001] The present application relates to the field of automobile technology, and in particular to a method, device, electronic device, storage medium and vehicle for determining a fully reclining posture. Background Art

[0002] In the interior environment of the cabin, the torso length of the human body in the image is often used to assist in judging the attributes of the human body, such as judging whether the human body is an adult or a child. However, in the interior environment of the vehicle cabin, due to the installation position and camera angle of the camera, as well as the different distances between the human body and the camera, for the same human body in a sitting position and a lying position (including a fully lying position and a half-lying position), the torso length of the human body in the images taken separately (the projection distance from the top of the head to the hips on the camera imaging screen) is quite different. Taking the human body in the front driver's seat as an example, the camera is installed above the middle rearview mirror in the car, and the camera angle is from front to back and tilted downward at a small angle. When the human body changes from a sitting position to a lying position, the torso length of the human body in the image will become shorter, so it is very easy to misidentify an adult as a child. Therefore, it is necessary to identify the human posture in the interior environment of the cabin.

[0003] Existing human posture recognition algorithms include algorithms based on depth maps. Since the depth map includes the depth information of the human body (i.e., the distance information of various parts of the human body from the camera), and the characteristics of the depth information of the human body are obviously different when the human body is in a sitting position or a lying position, human posture recognition can be performed based on the depth map algorithm.

[0004] However, algorithms based on depth maps, on the one hand, require image acquisition devices that can obtain depth information, which has high requirements and high costs. On the other hand, algorithms based on depth images have a large amount of data processing, a slow recognition speed, and sometimes inaccurate depth recognition and low recognition efficiency. Summary of the invention

[0005] The present application provides a method, device, electronic device, storage medium and vehicle for determining a fully reclining posture, which can accurately identify whether a human body in a captured cabin environment image is in a fully reclining posture.

[0006] In the first aspect, an embodiment of the present application provides a method for determining a full-lying posture, including: identifying a target human body in a human body image to be identified based on a human body key point detection network, obtaining multiple crotch key points and confidence levels of the multiple crotch key points, wherein the human body image to be identified is obtained based on an image of the interior environment of a cabin taken by a camera in a vehicle; based on the confidence levels of the multiple crotch key points, determining a crotch key point among the multiple crotch key points whose confidence level is greater than or equal to a corresponding crotch confidence threshold as at least one credible crotch key point; determining that the target human body is in a full-lying posture when at least one credible crotch key point and a crotch point distribution domain in a full-lying posture satisfy a preset relationship; wherein the preset relationship is used to indicate that an area with at least one credible crotch key point as a vertex has an overlapping area with the crotch point distribution domain in a full-lying posture.

[0007] In some embodiments of the present application, when at least one credible crotch key point and the crotch point distribution domain in the full-lying posture satisfy a preset relationship, it is determined that the target human body is in the full-lying posture, including: when the point indicated by the weighted average of the coordinates of at least one credible crotch key point is located in the crotch point distribution domain in the full-lying posture, it is determined that the target human body is in the full-lying posture.

[0008] In some embodiments of the present application, when at least one credible crotch key point and the crotch point distribution domain in the full-lying posture satisfy a preset relationship, it is determined that the target human body is in the full-lying posture, including: when there is a crotch key point located in the crotch point distribution domain in the full-lying posture among at least one credible crotch key point, it is determined that the target human body is in the full-lying posture.

[0009] In some embodiments of the present application, when at least one credible crotch key point satisfies a preset relationship with the crotch point distribution domain in a fully lying posture, before determining that the target human body is in a fully lying posture, the method also includes: when the target torso length is less than or equal to a torso length threshold, determining whether the coordinates of at least one credible crotch key point satisfy a preset relationship with the crotch point distribution domain; wherein the target torso length is determined based on a mapping relationship between the torso key points and the torso length of the target human body, the torso key points are obtained by identifying the target human body in the human body image to be identified through a human key point detection network, and the torso length mapping relationship is used to indicate the mapping relationship between the torso key points and the target torso length.

[0010] In some embodiments of the present application, when at least one credible crotch key point and the crotch point distribution domain in the fully lying posture satisfy a preset relationship, it is determined that the target human body is in a fully lying posture, including: when the coordinates of at least one credible crotch key point and the crotch point distribution domain satisfy a preset relationship, based on the mapping relationship between the lower limb key points and the lower limb length of the target human body, the lower limb length mapping relationship is used to indicate the mapping relationship between the lower limb key points and the lower limb length; based on the torso key points and the torso length mapping relationship of the target human body, the target torso length of the target human body is determined, and the torso length mapping relationship is used to indicate the mapping relationship between the torso key points and the target torso length; the ratio of the lower limb length to the target torso length is determined as the target ratio; when the target ratio is greater than or equal to the ratio threshold, it is determined that the target human body is in a fully lying posture; wherein the lower limb key points and the torso key points are obtained by identifying the target human body in the human body image to be identified through a human body key point detection network.

[0011] In some embodiments of the present application, when at least one credible crotch key point and the crotch point distribution domain in the fully lying posture satisfy a preset relationship, after determining that the target human body is in the fully lying posture, the method also includes: based on the target correction coefficient, correcting the target torso length to obtain a corrected target torso length; wherein the target correction coefficient is determined based on multiple torso correction coefficients, the target correction coefficient is greater than or equal to the minimum value of the multiple torso correction coefficients, and less than or equal to the maximum value of the multiple torso correction coefficients; each torso correction coefficient is the ratio of the torso length of a human body in a sitting position to the torso length in a fully lying posture, or each correction coefficient is the ratio of the torso length of a human body in a fully lying posture to the torso length in a sitting position; wherein the target torso length is determined based on the mapping relationship between the torso key points and the torso length of the target human body, the torso key points are obtained by identifying the target human body in the human body image to be identified through a human key point detection network, and the torso length mapping relationship is used to indicate the mapping relationship between the torso key points and the target torso length.

[0012] In the second aspect, an embodiment of the present application provides a device for determining a full-lying posture, including: an identification module, which is used to identify a target human body in a human body image to be identified based on a human body key point detection network, and obtain multiple crotch key points and confidence levels of the multiple crotch key points, wherein the human body image to be identified is obtained based on an image of the internal environment of a cabin taken by a camera in a vehicle; a determination module, which is used to determine, based on the confidence levels of multiple crotch key points, a crotch key point among the multiple crotch key points whose confidence level is greater than or equal to a corresponding crotch confidence threshold as at least one credible crotch key point; the determination module is also used to determine that the target human body is in a full-lying posture when at least one credible crotch key point satisfies a preset relationship with a crotch point distribution domain in a full-lying posture; wherein the preset relationship is used to indicate that there is an overlapping area between an area with at least one credible crotch key point as a vertex and the crotch point distribution domain in a full-lying posture.

[0013] In some embodiments of the present application, the determination module is specifically used to determine that the target human body is in a fully lying posture when the point indicated by the weighted average value of the coordinates of at least one credible crotch key point is located in the crotch point distribution domain in the fully lying posture.

[0014] In some embodiments of the present application, the determination module is specifically used to determine that the target human body is in a fully lying posture when there is a crotch key point located in the crotch point distribution domain in the fully lying posture among at least one credible crotch key point.

[0015] In some embodiments of the present application, the determination module is also used to determine whether the coordinates of at least one credible crotch key point satisfy a preset relationship with the crotch point distribution domain in the fully lying posture before determining that the target human body is in a fully lying posture, when at least one credible crotch key point satisfies a preset relationship with the crotch point distribution domain in the fully lying posture, and when the target torso length is less than or equal to a torso length threshold; wherein the target torso length is determined based on a mapping relationship between the torso key points and the torso length of the target human body, the torso key points are obtained by identifying the target human body in the human body image to be identified through a human key point detection network, and the torso length mapping relationship is used to indicate the mapping relationship between the torso key points and the target torso length.

[0016] In some embodiments of the present application, the determination module is specifically used to determine the lower limb length of the target human body based on the mapping relationship between the lower limb key points and the lower limb length of the target human body, when the coordinates of at least one credible crotch key point and the crotch point distribution domain satisfy a preset relationship, and the lower limb length mapping relationship is used to indicate the mapping relationship between the lower limb key points and the lower limb length; determine the target torso length of the target human body based on the mapping relationship between the torso key points and the torso length of the target human body, and the torso length mapping relationship is used to indicate the mapping relationship between the torso key points and the target torso length; determine the ratio of the lower limb length to the target torso length as the target ratio; when the target ratio is greater than or equal to the ratio threshold, determine that the target human body is in a fully lying position; wherein the lower limb key points and the torso key points are obtained by identifying the target human body in the human body image to be identified through a human body key point detection network.

[0017] In some embodiments of the present application, the device also includes: a correction module, which is used to correct the target torso length based on the target correction coefficient to obtain a corrected target torso length after determining that the target human body is in a fully lying posture when at least one credible crotch key point and the crotch point distribution domain in the fully lying posture satisfy a preset relationship; wherein the target correction coefficient is determined based on multiple torso correction coefficients, and the target correction coefficient is greater than or equal to the minimum value of the multiple torso correction coefficients, and less than or equal to the maximum value of the multiple torso correction coefficients; each torso correction coefficient is the ratio of the torso length of a human body in a sitting position to the torso length in a fully lying position, or each correction coefficient is the ratio of the torso length of a human body in a fully lying position to the torso length in a sitting position; wherein the target torso length is determined based on the mapping relationship between the torso key points and the torso length of the target human body, the torso key points are obtained by identifying the target human body in the human body image to be identified through a human key point detection network, and the torso length mapping relationship is used to indicate the mapping relationship between the torso key points and the target torso length.

[0018] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a processor, wherein the processor is used to execute a computer program stored in a memory, wherein when the computer program is executed by the processor, the steps of any one of the methods for determining a fully lying posture provided in the first aspect are implemented.

[0019] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of any one of the methods for determining a fully lying posture provided in the first aspect are implemented.

[0020] In a fifth aspect, an embodiment of the present application provides a vehicle, comprising: a fully reclining posture determination device as described in the second aspect, or an electronic device as described in the third aspect, or a computer-readable storage medium as described in the fourth aspect.

[0021] In a sixth aspect, an embodiment of the present application provides a computer program product, wherein the computer program product includes a computer program or instructions. When the computer program product runs on a processor, the processor executes the computer program or instructions to implement the steps of the method for determining a fully lying posture as described in the first aspect.

[0022] In the seventh aspect, an embodiment of the present application provides a chip, which includes a processor, a memory and a communication interface, the communication interface is coupled to the processor, the memory is used to store programs or instructions that can be run on the processor, and the processor is used to execute the program or instructions to implement the steps of the full-lying posture determination method as described in the first aspect.

[0023] Compared with the prior art, the technical solution provided by the embodiment of the present application has the following advantages: in the embodiment of the present application, a target human body in a human body image to be identified (the human body image to be identified is obtained by an image of the interior environment of the cabin taken by a camera in the vehicle) is identified based on a human body key point detection network to obtain a plurality of crotch key points and the confidence levels of the plurality of crotch key points; based on the confidence levels of the plurality of crotch key points, a crotch key point among the plurality of crotch key points whose confidence level is greater than or equal to a corresponding crotch confidence level threshold is determined as at least one credible crotch key point; when at least one credible crotch key point and a crotch point distribution domain in a fully lying posture satisfy a preset relationship, it is determined that the target human body is in a fully lying posture; wherein the preset relationship is used to indicate that an area with at least one credible crotch key point as a vertex has an overlapping area with the crotch point distribution domain in a fully lying posture. In this way, this scheme determines whether the target human body in the cabin interior environment is in a fully lying posture by judging whether the coordinates of the key points among multiple crotch key points whose confidence is greater than or equal to the crotch confidence threshold and whether the crotch point distribution domain in the fully lying posture meets the preset conditions. Therefore, this scheme does not need to judge the human body posture in combination with the depth information of the human body, and there is no need to install an image acquisition device that can collect depth information in the cabin. The requirements for the image acquisition device are low and the cost is low. By using an ordinary image acquisition device to collect a two-dimensional image of the cabin interior environment, it can be accurately identified whether the human body in the cabin is in a lying position. Compared with the depth image including depth information, the processing data is small, the processing speed is fast, the recognition accuracy is high, and the recognition efficiency is high. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0025] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0026] Figure 1 A schematic diagram of key points of the human body provided for this application;

[0027] Figure 2 A schematic diagram of a process for determining a fully lying posture provided in this application;

[0028] Figure 3 A schematic diagram of a human body key point recognition process provided in this application;

[0029] Figure 4 A flowchart of another method for determining a fully lying posture provided by the present application;

[0030] Figure 5 A schematic diagram of the structure of a fully lying posture determination device provided in this application;

[0031] Figure 6 A schematic diagram of the hardware structure of an electronic device provided in this application. DETAILED DESCRIPTION

[0032] In order to more clearly understand the above-mentioned purposes, features and advantages of the present application, the scheme of the present application will be further described below. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0033] In the following description, many specific details are set forth to facilitate a full understanding of the present application, but the present application may also be implemented in other ways different from those described herein; obviously, the embodiments in the specification are only part of the embodiments of the present application, rather than all of the embodiments.

[0034] The terms "first", "second", etc. in the specification and claims of this application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first", "second", etc. are generally of the same type, and the number of objects is not limited. For example, the first object can be one or more.

[0035] First, some nouns or terms involved in the claims and description of the present invention are explained below.

[0036] Cabin: The interior space of a car, where the driver and passengers sit.

[0037] Lying posture: In the cabin, some seats have a large adjustable backrest range and can be laid flat. At this time, the posture of a person lying on the seat is defined as lying posture, or simply lying posture. Generally, it refers to the front row seats. The embodiments of this application are described using the front row seats as an example. The lying posture is further divided into a semi-lying posture and a full-lying posture.

[0038] Semi-reclining posture: The person lies on the seat, but the lower body is in the same normal sitting position, except that the upper body lies flat and the calves are not placed on the seat.

[0039] Fully reclining posture: The person lies on the seat, with the whole body completely flat, the body roughly in a straight line, and the calves placed on the seat.

[0040] Human key point detection network: A deep learning network built for detection scenarios. It is usually composed of a backbone network, a neck network, and a head network. It is used to detect whether there are human key points and return the coordinates and confidence of the human key points.

[0041] Trusted crotch key points: There are three crotch key points, namely the right crotch key point indicated by mark "16", the lower crotch key point indicated by mark "17", and the left crotch key point indicated by mark "18". For a crotch key point, a crotch key point is a trusted crotch point if and only if the confidence of the crotch key point is greater than or equal to the crotch confidence threshold.

[0042] Crotch point distribution domain in full-lying posture: Through the analysis of a large amount of data, in the full-lying posture, since the entire human body moves backward, the distribution range of the crotch points is concentrated in a special closed domain, which is obviously isolated from the crotch point distribution in other postures. This closed domain is called the crotch point distribution domain in the full-lying posture, and the crotch point falls into it if and only if the human body is in the full-lying posture.

[0043] like Figure 1As shown, the mark "1" indicates the key point of the top of the head, the mark "2" indicates the key point of the right ear, the mark "3" indicates the key point of the left ear, the mark "4" indicates the key point of the chin, the mark "5" indicates the key point of the neck, the mark "6" indicates the key point of the right shoulder, the mark "7" indicates the key point of the right elbow, the mark "8" indicates the key point of the right wrist, the mark "9" indicates the key point of the right finger, the mark "10" indicates the key point of the left shoulder, and the mark "11" indicates the key point of the left elbow. The key points marked “12” indicate the left wrist, “13” indicate the left finger, “14” indicate the chest, “15” indicate the waist, “19” indicate the right knee, “20” indicate the right ankle, “21” indicate the right toe, “22” indicate the left knee, “23” indicate the left ankle, and “24” indicate the left toe.

[0044] Torso length: The straight-line distance from the top of the head to the hips, often used to assist in judging human attributes, is calculated from the coordinates of the key points (top of the head key point, crotch key point, right crotch key point and lower crotch key point) obtained by the human key point detection network.

[0045] Torso correction coefficient: Due to the installation position of the camera, the shooting angle, and the different distances between the human body and the camera, the torso length of the same person is very different when sitting and lying down, and needs to be corrected by a coefficient, which is called the torso correction coefficient.

[0046] An embodiment of the present application proposes a method for determining a fully lying posture, which identifies a target human body in a human body image to be identified (the human body image to be identified is obtained by an image of the interior environment of a cabin taken by a camera in a vehicle) based on a human body key point detection network, and obtains multiple crotch key points and confidence levels of the multiple crotch key points; based on the confidence levels of the multiple crotch key points, a crotch key point among the multiple crotch key points whose confidence level is greater than or equal to a corresponding crotch confidence level threshold is determined as at least one credible crotch key point; when at least one credible crotch key point and a crotch point distribution domain in a fully lying posture satisfy a preset relationship, it is determined that the target human body is in a fully lying posture; wherein the preset relationship is used to indicate that an area with at least one credible crotch key point as a vertex has an overlapping area with the crotch point distribution domain in a fully lying posture. In this way, this scheme determines whether the target human body in the cabin interior environment is in a fully lying posture by judging whether the coordinates of the key points among multiple crotch key points whose confidence is greater than or equal to the crotch confidence threshold and whether the crotch point distribution domain in the fully lying posture meets the preset conditions. Therefore, this scheme does not need to judge the human body posture in combination with the depth information of the human body, and there is no need to install an image acquisition device that can collect depth information in the cabin. The requirements for the image acquisition device are low and the cost is low. By using an ordinary image acquisition device to collect a two-dimensional image of the cabin interior environment, it can be accurately identified whether the human body in the cabin is in a lying position. Compared with the depth image including depth information, the processing data is small, the processing speed is fast, the recognition accuracy is high, and the recognition efficiency is high.

[0047] The present application is applied to the scenario of human body lying posture recognition in the cockpit environment. The electronic device in the embodiment of the present application can be a vehicle-mounted terminal, or a mobile phone, notebook, computer, etc. that is connected to the vehicle for communication. The specific details can be determined according to actual conditions and are not limited here.

[0048] The technical solution of the present application is explained in detail below through several specific embodiments.

[0049] Figure 2 A schematic diagram of a process for determining a fully lying posture provided in this application, such as Figure 2 As shown, the method for determining the fully lying posture may include the following steps 101 to 103.

[0050] 101. Identify a target human body in the human body image to be identified based on a human body key point detection network to obtain a plurality of crotch key points and confidence levels of the plurality of crotch key points.

[0051] The multiple crotch key points may include a left crotch key point, a right crotch key point and a lower crotch key point, and may also include other crotch key points, which may be determined according to actual conditions and are not limited here. In the following embodiments, the multiple crotch key points including the left crotch key point, the right crotch key point and the lower crotch key point are used as an example for explanation.

[0052] The human body image to be identified is obtained based on an image of the interior environment of the cabin taken by a camera in the vehicle.

[0053] In some embodiments of the present application, Figure 3 As shown, when identifying the key points of the human body, the human body image to be identified (the image of the interior environment of the cabin taken by the camera in the vehicle) is firstly identified through the human body detection network to obtain the human body external frame, and then the human body image to be identified corresponding to the human body external frame is extracted from the human body image to be identified, and the human body key point detection network is used to detect the human body key points of the human body image to be identified to obtain the coordinates and confidence of the human body key points. Among them, before the human body image to be identified is identified through the human body detection network, the human body image to be identified needs to be preprocessed, and after the human body image to be identified is identified through the human body detection network, the human body recognition result needs to be post-processed; similarly, before the human body key point detection network is used to detect the human body key points of the human body image to be identified, the human body image to be identified needs to be preprocessed, and after the human body key point detection network is used to detect the human body key points of the human body image to be identified, the human body key point recognition result needs to be post-processed; the specific preprocessing and post-processing processes can refer to the relevant technology, which will not be repeated here.

[0054] In some embodiments of the present application, the process of training a human body detection network and a human body key point detection network includes: collecting a large amount of diverse image data in the cockpit environment, annotating the coordinate information of the human body external bounding box for each image, and then obtaining the first sample training data based on each image and the coordinate information of the human body external bounding box in the image; based on the first training sample data, training a basic network composed of a backbone network, a shoulder network, and a head network to train a human body detection network. Based on each external bounding box, at least one human body image is separated from each image, and for each human body image, the coordinate information of the human body key points in each human body image is annotated, and then based on each human body image and the coordinate information of the human body key points in each human body image, obtaining the second sample training data. Based on the second training sample data, training a basic network composed of a backbone network, a shoulder network, and a head network to train a human body key point detection network.

[0055] 102. Based on the confidences of the multiple crotch key points, a crotch key point whose confidence is greater than or equal to a corresponding crotch confidence threshold among the multiple crotch key points is determined as at least one credible crotch key point.

[0056] It can be understood that the crotch key point whose confidence is greater than or equal to the corresponding crotch confidence threshold is a credible crotch key point. The at least one credible crotch key point is a credible key point, for example, if the confidence of the left crotch key point is greater than or equal to the corresponding crotch confidence threshold, the left crotch key point is a credible crotch key point; if the confidence of the right crotch key point is greater than or equal to the corresponding crotch key point threshold, the right crotch key point is a credible crotch key point; if the confidence of the lower crotch key point is less than the corresponding crotch confidence threshold, the lower crotch key point is not a credible crotch key point. In other words, the at least one credible crotch key point includes the left crotch key point and the right crotch key point.

[0057] The crotch confidence thresholds corresponding to each crotch key point may be equal or unequal, which may be determined according to actual conditions. For example, the crotch confidence threshold corresponding to the left crotch key point may be equal or unequal to the crotch key point threshold corresponding to the right crotch key point, which may be determined according to actual conditions.

[0058] 103. When at least one credible crotch key point and the crotch point distribution domain in the fully lying posture satisfy a preset relationship, it is determined that the target human body is in the fully lying posture.

[0059] The preset relationship is used to indicate that there is an overlapping area between a region with at least one credible crotch key point as a vertex and the crotch point distribution domain in a fully lying posture.

[0060] In some embodiments of the present application, the preset relationship may be that some or all of the at least one trusted crotch key point are located in the crotch point distribution domain in the fully lying posture, that is, there is a crotch key point in the at least one trusted crotch key point that is located in the crotch point distribution domain in the fully lying posture. In this case, there is an overlapping area between the area where at least one trusted crotch key point is the vertex and the crotch point distribution domain in the fully lying posture.

[0061] In some embodiments of the present application, the preset relationship may be that the point indicated by the weighted average of the coordinates of the at least one credible crotch key point is located in the crotch point distribution domain in the fully lying posture. In this case, there is an overlapping area between the area where at least one credible crotch key point is the vertex and the crotch point distribution domain in the fully lying posture.

[0062] In some embodiments of the present application, when the more credible crotch key points there are among the left crotch key points, the right crotch key points and the lower crotch key points, and when the coordinates of all credible crotch key points satisfy a preset relationship with the distribution domain of the lower crotch points in the fully lying posture, the higher the accuracy of using the crotch key points to judge whether the target human body is in the fully lying posture.

[0063] In the embodiment of the present application, whether the target human body in the interior environment of the cabin is in a fully lying posture is judged by judging the coordinates of the key points among multiple crotch key points whose confidence is greater than or equal to the crotch confidence threshold and whether the crotch point distribution domain in the fully lying posture meets the preset conditions. Therefore, this scheme does not need to judge the posture of the human body in combination with the depth information of the human body, and there is no need to install an image acquisition device that can collect depth information in the cabin. The requirements for the image acquisition device are low and the cost is low. By using an ordinary image acquisition device to collect a two-dimensional image of the interior environment of the cabin, it is possible to accurately identify whether the human body in the cabin is in a lying position. Compared with the depth image including depth information, the processing data is small, the processing speed is fast, the recognition accuracy is high, and the recognition efficiency is high.

[0064] In some embodiments of the present application, the above step 103 can be specifically implemented through the following step 103a.

[0065] 103a. When the point indicated by the weighted average value of the coordinates of at least one credible crotch key point is located in the crotch point distribution domain in the fully lying posture, it is determined that the target human body is in the fully lying posture.

[0066] Among them, the weighted average of the coordinates of each key point in at least one credible crotch key point can be obtained by weighted summing up the coordinates of the at least one key point based on the weight corresponding to each key point in the at least one credible crotch key point, and the sum of the weights corresponding to each key point in the at least one key point is 1.

[0067] In some embodiments of the present application, the weights corresponding to each key point in the at least one key point may be equal or unequal, which may be determined based on actual conditions and is not limited here.

[0068] Exemplarily, the preset relationship may be that the point indicated by the arithmetic mean of the coordinates of the at least one credible crotch key point is located in the crotch point distribution domain in the fully lying posture.

[0069] Exemplarily, the weighted average is the arithmetic mean. If at least one credible crotch key point includes a left crotch key point and a right crotch key point, wherein the coordinates of the left crotch key point are (a, b) and the coordinates of the right crotch key point are (c, d), then the weighted average of the coordinates of the at least one credible crotch key point is The weighted average of the coordinates of the at least one credible crotch key point indicates a point, i.e., a coordinate Indicated point.

[0070] In an embodiment of the present application, when the point indicated by the weighted average value of the coordinates of at least one credible crotch key point is located in the crotch point distribution domain in the fully lying posture, it can be accurately and quickly determined that the target human body is in the fully lying posture, which can improve the efficiency of judging whether the target human body is in the fully lying posture.

[0071] In some embodiments of the present application, the above step 103 can be specifically implemented through the following step 103b.

[0072] 103b. When there is a crotch key point located in the crotch point distribution domain in the fully lying posture among at least one credible crotch key point, it is determined that the target human body is in the fully lying posture.

[0073] It can be understood that if some or all of at least one credible crotch key point are located in the crotch point distribution domain in the fully lying posture, then at least one credible crotch key point exists in the crotch point distribution domain in the fully lying posture.

[0074] In an embodiment of the present application, when there is a crotch key point located in the crotch point distribution domain in the full-lying posture among at least one credible crotch key point, it can be accurately and quickly determined that the target human body is in the full-lying posture, which can improve the efficiency of judging whether the target human body is in the full-lying posture.

[0075] In some embodiments of the present application, the method for determining a fully lying posture provided in the embodiments of the present application may further include the following step 104.

[0076] 104. Determine the target torso length of the target human body based on the top of the head key point, the left crotch key point, the right crotch key point and the lower crotch key point.

[0077] In some embodiments of the present application, the torso length mapping relationship is used to indicate that the straight-line distance between the point corresponding to the average coordinates of the valid key points (key points with confidence greater than a certain threshold) among the left crotch key point, the right crotch key point and the lower crotch key point and the top of the head key point is determined as the target torso length.

[0078] In an embodiment of the present application, after the above-mentioned step 101, the target torso length is calculated, which is conducive to determining whether to subsequently judge whether the human body is in a fully lying posture, and / or, is conducive to determining whether the human body is in a fully lying posture, and / or, is conducive to correcting the target torso length when it is determined that the human body is in a fully lying posture, which can improve processing efficiency.

[0079] In some embodiments of the present application, before the above step 103, the method for determining the fully lying posture provided by the embodiments of the present application may further include the following step 105.

[0080] 105. When the target torso length is less than or equal to the torso length threshold, determine whether the coordinates of at least one credible crotch key point satisfy a preset relationship with the crotch point distribution domain.

[0081] The target torso length is determined based on a mapping relationship between the torso key points and the torso length of the target human body, the torso key points are obtained by identifying the target human body in the human body image to be identified through a human key point detection network, and the torso length mapping relationship is used to indicate a mapping relationship between the torso key points and the target torso length. For a specific description of the torso length mapping relationship and determining the target torso length, reference can be made to the relevant description in the above step 104, which will not be repeated here.

[0082] The trunk length threshold can be determined according to actual conditions and is not limited here. For example, the trunk length threshold can be a critical value of the trunk length of an adult and a child.

[0083] In the embodiment of the present application, when the target torso length is greater than the torso length threshold, it is not necessary to determine whether the target human body is in a fully lying posture, and it can be determined that the human body is an adult; when the target torso length is less than or equal to the torso length threshold, it is impossible to determine whether the target human body is an adult or a child, so it is necessary to determine whether the target human body is in a fully lying posture to assist in determining whether the target human body is an adult or a child. Therefore, by judging whether the target torso length is less than or equal to the torso length threshold, it is possible to avoid judging whether the target human body is in a fully lying posture in all cases. For example, it is not necessary to judge whether the target human body is in a fully lying posture when the target torso length is greater than the torso length threshold, which can save calculation amount and improve judgment efficiency.

[0084] In some embodiments of the present application, the above step 103 can be specifically implemented by the following steps 103c to 103e.

[0085] 103c. When the coordinates of at least one credible crotch key point and the crotch point distribution domain satisfy a preset relationship, the length of the target body's lower limbs is determined based on a mapping relationship between the target body's lower limb key points and the lower limb length.

[0086] The lower limb length mapping relationship is used to indicate the mapping relationship between the lower limb key points and the lower limb length.

[0087] 103d. Determine the target torso length of the target human body based on the mapping relationship between the torso key points and the torso length of the target human body.

[0088] The torso length mapping relationship is used to indicate the mapping relationship between the torso key points and the target torso length.

[0089] Among them, the lower limb key points and torso key points are obtained by identifying the target human body in the human body image through the human body key point detection network.

[0090] Among them, the mapping relationship of the lower limb length and the mapping relationship of the trunk length can be determined according to actual conditions and are not limited here.

[0091] In some embodiments of the present application, the lower limb key points include: a left crotch key point, a right crotch key point, a lower crotch key point, a left knee key point, and a right knee key point. The length of the lower limb of the target human body can be determined based on the left crotch key point, the right crotch key point, the lower crotch key point, the left knee key point, and the right knee key point.

[0092] In some embodiments of the present application, the distance between the position corresponding to the average coordinates of the coordinates of the valid key points (key points with confidence greater than a certain threshold) among the left crotch key point, the right crotch key point and the lower crotch key point, and the position corresponding to the average coordinates of the coordinates of the valid key points among the left knee key point and the right knee key point can be determined as the length of the lower limb.

[0093] For the specific description of determining the target torso length, reference may be made to the relevant description in the above step 104, which will not be repeated here.

[0094] 104e. The ratio of the lower limb length to the target trunk length is determined as the target ratio.

[0095] 104f. When the target ratio is greater than or equal to the ratio threshold, it is determined that the target human body is in a fully lying position.

[0096] The ratio threshold can be determined according to actual conditions and is not limited here.

[0097] When the target ratio is less than the ratio threshold, it is determined that the target human body is not in a lying position.

[0098] It can be understood that when a human body is in a sitting position (the lower limbs cannot be stretched out), the ratio of the lower limb length to the target torso length is smaller than the ratio of the lower limb length to the target torso length when the human body is in a fully lying position (the lower limbs can be stretched out). Therefore, when the target ratio is greater than or equal to the ratio threshold, it is determined that the target human body is in a fully lying position. In this way, it can be more accurately determined that the current human body is in a fully lying position.

[0099] In some embodiments of the present application, before the above step 103, the method for determining the fully lying posture provided by the embodiments of the present application may further include the following step 106.

[0100] 106. Based on the target correction coefficient, the target torso length is corrected to obtain a corrected target torso length.

[0101] The target correction coefficient is determined based on a plurality of torso correction coefficients, and the target correction coefficient is greater than or equal to a minimum value among the plurality of torso correction coefficients, and less than or equal to a maximum value among the plurality of torso correction coefficients.

[0102] Each torso correction factor is the ratio of the torso length of a person in a sitting position to the torso length of a person in a fully reclining position, or each correction factor is the ratio of the torso length of a person in a fully reclining position to the torso length of a person in a sitting position.

[0103] The target torso length is determined based on the mapping relationship between the torso key points and the torso length of the target human body, the torso key points are obtained by identifying the target human body in the human body image to be identified through a human key point detection network, and the torso length mapping relationship is used to indicate the mapping relationship between the torso key points and the target torso length. For the description of the torso length mapping relationship and the determination process of the target torso length, reference can be made to the relevant description in the above step 104, which will not be repeated here.

[0104] Among them, the target correction coefficient can be the average value of multiple torso correction coefficients, the minimum value of multiple torso correction coefficients, the maximum value of multiple torso correction coefficients, or the expected value after normal distribution fitting of multiple torso correction coefficients.

[0105] In the embodiment of the present application, the target torso length is corrected based on the target correction coefficient to obtain a corrected target torso length, which is helpful for judging human attributes based on the corrected target torso length and can improve the accuracy of judging human attributes.

[0106] Exemplarily, the human key point detection network is used to identify the target human body in the human body image to be identified. The coordinate positions of the left crotch key point, the right crotch key point, and the lower crotch key point are simply recorded as m, n, and t, respectively. The confidence levels of the left crotch key point, the right crotch key point, and the lower crotch key point are simply recorded as a, b, and c, respectively. The crotch confidence threshold is recorded as T, and the distribution domain of the lower crotch points in the fully lying posture is recorded as S.

[0107] To determine the reliable crotch key points:

[0108] If a ≥ T, the left crotch key point is a credible crotch key point;

[0109] If b ≥ T, the right crotch key point is a credible crotch key point;

[0110] If c ≥ T, the lower crotch key point is a credible crotch key point.

[0111] Determine whether there is a reliable crotch key point:

[0112] If there is no credible crotch key point, the current information is not clear enough and the process terminates;

[0113] If there are credible crotch key points, calculate the arithmetic mean of the coordinate positions of all credible crotch key points.

[0114] Assuming that the left crotch key point, the right crotch key point, and the lower crotch key point are all credible crotch key points, the weighted average value of the coordinates of the credible crotch key points is x=(m+n+t) / 3.

[0115] Determine whether the target person is in a fully lying position:

[0116] If x falls inside S, the target body is in a fully lying position and the next step of torso length correction is required, otherwise no processing is performed.

[0117] In the embodiments of the present application, it is very important to distinguish between sitting posture and lying posture in the interior environment of the cabin, which will directly affect the accuracy of judging the attributes of the human body. However, in the past, there was a lack of such a method for distinguishing. The present invention proposes a fully lying posture judgment scheme based on the position of a credible crotch key point. Starting from a large amount of data, it is found that when the human body is in a fully lying posture, the distribution range of the crotch key points is concentrated in a special closed domain, and there is a clear isolation from the distribution of the crotch key points in other postures. Therefore, it can be used to distinguish between the fully lying posture and other postures. Furthermore, the present invention introduces the concept of a credible crotch key point, which is adopted if and only if the crotch key point is credible, which improves the reliability of the result.

[0118] In the embodiment of the present application, by accurately identifying whether the human body in the image in the cabin environment is in a fully reclining posture, the human body in a fully reclining posture is separated from the image, which greatly reduces the false detection of adults being identified as children in a fully reclining posture, and greatly improves the accuracy of the results of judging the attributes of the human body inside the cabin.

[0119] The present application also provides a fully lying posture determination device, Figure 5 A schematic diagram of the structure of a fully lying posture determination device provided in this application, such as Figure 5As shown, the fully lying posture determination device includes: an identification module 501, which is used to identify a target human body in a human body image to be identified based on a human body key point detection network, and obtain multiple crotch key points and the confidence levels of the multiple crotch key points, wherein the human body image to be identified is obtained based on an image of the internal environment of the cabin taken by a camera in the vehicle; a determination module 502, which is used to determine, based on the confidence levels of the multiple crotch key points, a crotch key point among the multiple crotch key points whose confidence level is greater than or equal to the corresponding crotch confidence level threshold as at least one credible crotch key point; the determination module 502 is also used to determine that the target human body is in a fully lying posture when at least one credible crotch key point satisfies a preset relationship with a crotch point distribution domain in a fully lying posture; wherein the preset relationship is used to indicate that an area with at least one credible crotch key point as a vertex has an overlapping area with the crotch point distribution domain in a fully lying posture.

[0120] In some embodiments of the present application, the determination module 502 is specifically used to determine that the target human body is in a fully lying posture when the point indicated by the weighted average value of the coordinates of at least one credible crotch key point is located in the crotch point distribution domain in the fully lying posture.

[0121] In some embodiments of the present application, the determination module 502 is specifically used to determine that the target human body is in a fully lying posture when there is a crotch key point located in the crotch point distribution domain in the fully lying posture among at least one credible crotch key point.

[0122] In some embodiments of the present application, the determination module 502 is also used to determine whether the coordinates of at least one credible crotch key point satisfy a preset relationship with the crotch point distribution domain in the fully lying posture before determining that the target human body is in the fully lying posture, when at least one credible crotch key point satisfies a preset relationship with the crotch point distribution domain in the fully lying posture, and when the target torso length is less than or equal to a torso length threshold; wherein the target torso length is determined based on a mapping relationship between the torso key points and the torso length of the target human body, the torso key points are obtained by identifying the target human body in the human body image to be identified through a human key point detection network, and the torso length mapping relationship is used to indicate the mapping relationship between the torso key points and the target torso length.

[0123] In some embodiments of the present application, the determination module 502 is specifically used to determine the lower limb length of the target human body based on the mapping relationship between the lower limb key points and the lower limb length of the target human body, when the coordinates of at least one credible crotch key point and the crotch point distribution domain satisfy a preset relationship, and the lower limb length mapping relationship is used to indicate the mapping relationship between the lower limb key points and the lower limb length; determine the target torso length of the target human body based on the mapping relationship between the torso key points and the torso length of the target human body, and the torso length mapping relationship is used to indicate the mapping relationship between the torso key points and the target torso length; determine the ratio of the lower limb length to the target torso length as the target ratio; when the target ratio is greater than or equal to the ratio threshold, determine that the target human body is in a fully lying position; wherein the lower limb key points and the torso key points are obtained by identifying the target human body in the human body image to be identified through a human body key point detection network.

[0124] In some embodiments of the present application, the device also includes: a correction module, which is used to correct the target torso length based on the target correction coefficient to obtain a corrected target torso length after determining that the target human body is in a fully lying posture when at least one credible crotch key point and the crotch point distribution domain in the fully lying posture satisfy a preset relationship; wherein the target correction coefficient is determined based on multiple torso correction coefficients, and the target correction coefficient is greater than or equal to the minimum value of the multiple torso correction coefficients, and less than or equal to the maximum value of the multiple torso correction coefficients; each torso correction coefficient is the ratio of the torso length of a human body in a sitting position to the torso length in a fully lying position, or each correction coefficient is the ratio of the torso length of a human body in a fully lying position to the torso length in a sitting position; wherein the target torso length is determined based on the mapping relationship between the torso key points and the torso length of the target human body, the torso key points are obtained by identifying the target human body in the human body image to be identified through a human key point detection network, and the torso length mapping relationship is used to indicate the mapping relationship between the torso key points and the target torso length.

[0125] It should be noted that the above-mentioned fully lying posture determination device can be the electronic device in the above-mentioned method embodiment of the present application, or it can be a functional module and / or functional entity in the electronic device that can realize the function of the device embodiment, and the embodiment of the present application is not limited.

[0126] In the embodiments of the present application, each module can implement the method for determining the fully lying posture provided in the above method embodiments, and can achieve the same technical effect. To avoid repetition, it will not be described here.

[0127] Figure 6 The structural schematic diagram of an electronic device provided in an embodiment of the present application is used to exemplarily illustrate an electronic device that implements any method for determining a fully lying posture in an embodiment of the present application, and should not be understood as a specific limitation on the embodiments of the present application.

[0128] like Figure 6 As shown, the electronic device 600 may include a processor (e.g., a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage device 608 into a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the electronic device 600 are also stored. The processor 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0129] Typically, the following devices may be connected to the I / O interface 605: input devices 606 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 607 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 608 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 609. The communication device 609 may allow the electronic device 600 to communicate with other devices wirelessly or by wire to exchange data. Although an electronic device 600 having various devices is shown, it should be understood that it is not required to implement or have all the devices shown. More or fewer devices may be implemented or have alternatively.

[0130] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 609, or installed from the storage device 608, or installed from the ROM 602. When the computer program is executed by the processor 601, the functions defined in any full-lying posture determination method provided in the embodiment of the present application can be executed.

[0131] It should be noted that the computer-readable medium mentioned above in the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0132] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0133] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0134] The above-mentioned computer-readable medium carries one or more programs. When the above-mentioned one or more programs are executed by the electronic device, the electronic device: identifies the target human body in the human body image to be identified based on the human body key point detection network, and obtains multiple crotch key points and the confidence levels of the multiple crotch key points, wherein the human body image to be identified is obtained based on the image of the internal environment of the cabin taken by the camera in the vehicle; based on the confidence levels of the multiple crotch key points, determines the crotch key points among the multiple crotch key points whose confidence levels are greater than or equal to the corresponding crotch confidence threshold as at least one credible crotch key point; determines that the target human body is in a fully lying posture when at least one credible crotch key point and the crotch point distribution domain in a fully lying posture satisfy a preset relationship; wherein the preset relationship is used to indicate that there is an overlapping area between an area with at least one credible crotch key point as a vertex and the crotch point distribution domain in a fully lying posture.

[0135] In an embodiment of the present application, a computer program code for performing the operation of the present application can be written in one or more programming languages ​​or a combination thereof, and the above-mentioned programming languages ​​include but are not limited to object-oriented programming languages, such as Java, Smalltalk, C++, and also include conventional procedural programming languages, such as "C" language or similar programming languages. The program code can be executed completely on the computer, partially on the computer, as an independent software package, partially on the computer and partially on the remote computer, or completely on the remote computer or server. In the case of a remote computer, the remote computer can be connected to the computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, using an Internet service provider to connect through the Internet).

[0136] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0137] The units involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of a unit does not, in some cases, constitute a limitation on the unit itself.

[0138] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0139] In the context of the present application, a computer-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. A computer-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. A more specific example of a computer-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0140] An embodiment of the present application further provides a vehicle, comprising: the above-mentioned fully-lying posture determination device, or the above-mentioned electronic device, or the above-mentioned computer-readable storage medium.

[0141] The above description is only a preferred embodiment of the present application and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present application is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in this application (but not limited to) by each other to form a technical solution.

[0142] In addition, although each operation is described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or to be performed in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be construed as limiting the scope of the application. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0143] Although the subject matter has been described in language specific to structural features and / or methodological logical actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims.

Claims

1. A method for determining a fully lying posture, characterized in that: The method comprises: Based on the human key point detection network, a target human body in the human body image to be identified is identified to obtain a plurality of crotch key points and confidence levels of the plurality of crotch key points, wherein the human body image to be identified is obtained based on an image of the interior environment of the cabin taken by a camera in the vehicle; Based on the confidences of the multiple crotch key points, determining a crotch key point among the multiple crotch key points whose confidence is greater than or equal to a corresponding crotch confidence threshold as at least one credible crotch key point; In the case where the at least one credible crotch key point and the crotch point distribution domain in the fully lying posture satisfy a preset relationship, determining that the target human body is in the fully lying posture; The preset relationship is used to indicate that there is an overlapping area between the area with the at least one credible crotch key point as the vertex and the crotch point distribution domain in the fully lying posture.

2. The method according to claim 1, characterized in that When the at least one credible crotch key point and the crotch point distribution domain in the fully lying posture satisfy a preset relationship, determining that the target human body is in the fully lying posture includes: In the case where the point indicated by the weighted average value of the coordinates of the at least one credible crotch key point is located in the crotch point distribution domain in the fully lying posture, it is determined that the target human body is in the fully lying posture.

3. The method according to claim 1, characterized in that When the at least one credible crotch key point and the crotch point distribution domain in the fully lying posture satisfy a preset relationship, determining that the target human body is in the fully lying posture includes: In the case that there is a crotch key point located in the crotch point distribution domain in the fully lying posture among the at least one credible crotch key point, it is determined that the target human body is in the fully lying posture.

4. The method according to claim 1, characterized in that: In the case where the at least one credible crotch key point and the crotch point distribution domain in the fully lying posture satisfy a preset relationship, before determining that the target human body is in the fully lying posture, the method further includes: When the target torso length is less than or equal to the torso length threshold, determining whether the coordinates of the at least one credible crotch key point satisfy the preset relationship with the crotch point distribution domain; Among them, the target torso length is determined based on the mapping relationship between the torso key points and the torso length of the target human body. The torso key points are obtained by identifying the target human body in the human body image to be identified through a human body key point detection network, and the torso length mapping relationship is used to indicate the mapping relationship between the torso key points and the target torso length.

5. The method according to claim 1, characterized in that When the at least one credible crotch key point and the crotch point distribution domain in the fully lying posture satisfy a preset relationship, determining that the target human body is in the fully lying posture includes: In the case where the coordinates of the at least one credible crotch key point and the crotch point distribution domain satisfy the preset relationship, the lower limb length of the target human body is determined based on the mapping relationship between the lower limb key points and the lower limb length of the target human body, wherein the lower limb length mapping relationship is used to indicate the mapping relationship between the lower limb key points and the lower limb length; Determine a target torso length of the target human body based on a mapping relationship between torso key points and torso length of the target human body, wherein the torso length mapping relationship is used to indicate a mapping relationship between the torso key points and the target torso length; Determine the ratio of the lower limb length to the target trunk length as a target ratio; In a case where the target ratio is greater than or equal to a ratio threshold, determining that the target human body is in a fully lying position; Among them, the lower limb key points and the torso key points are obtained by identifying the target human body in the human body image to be identified through a human body key point detection network.

6. The method according to any one of claims 1 to 5, characterized in that After determining that the target human body is in a fully lying posture when the at least one credible crotch key point and the crotch point distribution domain in the fully lying posture satisfy a preset relationship, the method further includes: Based on the target correction coefficient, the target torso length is corrected to obtain the corrected target torso length; Wherein, the target correction coefficient is determined based on multiple torso correction coefficients, and the target correction coefficient is greater than or equal to the minimum value of the multiple torso correction coefficients, and less than or equal to the maximum value of the multiple torso correction coefficients; each torso correction coefficient is the ratio of the torso length of a human body in a sitting position to the torso length in a fully lying position, or each correction coefficient is the ratio of the torso length of a human body in a fully lying position to the torso length in a sitting position; wherein, the target torso length is determined based on the mapping relationship between the torso key points and the torso length of the target human body, the torso key points are obtained by identifying the target human body in the human body image to be identified through a human body key point detection network, and the torso length mapping relationship is used to indicate the mapping relationship between the torso key points and the target torso length.

7. A fully lying posture determination device, characterized in that: include: A recognition module, configured to recognize a target human body in a human body image to be recognized based on a human body key point detection network, and obtain a plurality of crotch key points and confidence levels of the plurality of crotch key points, wherein the human body image to be recognized is obtained based on an image of the interior environment of the cabin taken by a camera in the vehicle; a determination module, configured to determine, based on the confidences of the multiple crotch key points, a crotch key point whose confidence is greater than or equal to a corresponding crotch confidence threshold among the multiple crotch key points as at least one credible crotch key point; The determination module is further used to determine that the target human body is in a fully lying posture when the at least one credible crotch key point and the crotch point distribution domain in the fully lying posture meet a preset relationship; The preset relationship is used to indicate that there is an overlapping area between the area with the at least one credible crotch key point as the vertex and the crotch point distribution domain in the fully lying posture.

8. An electronic device, characterized in that: include: A processor, wherein the processor is used to execute a computer program stored in a memory, wherein the computer program, when executed by the processor, implements the steps of the method for determining a fully lying posture according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method for determining a fully lying posture according to any one of claims 1 to 6 are implemented.

10. A vehicle, characterized in that: include: The fully lying posture determining device as claimed in claim 7, or the electronic device as claimed in claim 8, or includes the computer-readable storage medium as claimed in claim 9.

Images