Lying posture determination method and device, electronic equipment, storage medium and vehicle

Abstract

The invention relates to a lying posture determination method and device, electronic equipment, a storage medium and a vehicle, and the method comprises the steps: carrying out the recognition of a target human body in a to-be-recognized image based on a target detection network, obtaining a human head external frame of the target human body, and enabling the to-be-recognized image to be an image of a cabin internal environment shot by a camera in the vehicle; determining the product of the height and the width of the head external frame as the absolute head area of the target human body; and when the absolute head area is smaller than or equal to the first area threshold value, determining that the target human body is in a lying posture. According to the method, whether the human body in the shot cabin environment image is in the lying posture or not can be accurately identified.

Description

Technical Field

[0001] The present application relates to the field of automobile technology, and in particular to a lying posture determination method, device, electronic device, storage medium and vehicle. Background Art

[0002] In the vehicle's cabin environment, due to the camera's installation position and camera angle, as well as the different distances between the human body and the camera, there is a big difference in the torso length (the projection distance from the top of the head to the hips on the camera's imaging screen) of the same person in the images taken when the person is sitting or lying (including fully lying and half-lying). Taking the front-row driver's seat as an example, the camera is installed above the center rearview mirror in the car, and the camera angle is from front to back and tilted downward at a small angle. When the person changes from a sitting position to a lying position, the torso length of the person in the corresponding image will become shorter.

[0003] In the cockpit environment, the torso length of the human body in the image is often used to assist in judging the human attributes, such as judging whether the human body is an adult or a child. However, since the torso length of a reclining human body is much smaller than that of a sitting human body, it is very easy to mistakenly identify a reclining adult as a child.

[0004] Existing human posture recognition algorithms include algorithms based on depth maps. Since the depth map includes the depth information of the human body (i.e., the distance information of each part of the human body from the camera), and the characteristics of the depth information of the human body are obviously different when the human body is in a sitting position or a lying position, the human posture recognition can be performed based on the algorithm of the depth map. However, the algorithm based on the depth map, on the one hand, has high requirements for the image acquisition equipment, which requires an image acquisition equipment that can obtain depth information, and has high cost. On the other hand, the algorithm based on the depth image has a large amount of data processing, a slow recognition speed, and sometimes the depth information recognition is inaccurate, and the recognition efficiency is low. Summary of the invention

[0005] The present application provides a lying posture determination method, device, electronic device, storage medium and vehicle, which can accurately identify whether a human body in a captured cabin environment image is in a lying posture.

[0006] In a first aspect, an embodiment of the present application provides a method for determining a lying posture, comprising: identifying a target human body in an image to be identified based on a target detection network to obtain an external frame of the target human body's head, wherein the image to be identified is an image of the interior environment of a cabin taken by a camera in a vehicle; multiplying the height and width of the external frame of the human head to determine the absolute head area of ​​the target human body; and determining that the target human body is in a lying posture when the absolute head area is less than or equal to a first area threshold.

[0007] In some embodiments of the present application, a target human body in an image to be identified is identified based on a target detection network to obtain a head external frame of the target human body, including: identifying the target human body in an image to be identified based on a target detection network to obtain a head external frame and a confidence level of the head external frame; determining the product of the height and width of the head external frame as the absolute head area of ​​the target human body, including: when the confidence level of the head external frame is greater than or equal to a head confidence threshold, determining the product of the height and width of the head external frame as the absolute head area of ​​the target human body.

[0008] In some embodiments of the present application, when the absolute head area is less than or equal to a first area threshold, it is determined that the target body is in a lying position, including: when the absolute head area is greater than or equal to a second area threshold, and less than or equal to the first area threshold, it is determined that the target body is in a semi-lying state; when the absolute head area is less than the second area threshold, it is determined that the target body is in a fully lying posture; wherein the second area threshold is less than the first area threshold.

[0009] In some embodiments of the present application, when the absolute head area is greater than or equal to the second area threshold and less than or equal to the first area threshold, after determining that the target human body is in a semi-reclining state, the method further includes: based on the first correction coefficient, correcting the target torso length to obtain a corrected target torso length, wherein the first correction coefficient is determined based on multiple first torso correction coefficients, the first correction coefficient is greater than or equal to the minimum value of the multiple first torso correction coefficients, and less than or equal to the maximum value of the multiple first torso correction coefficients; each first torso correction coefficient is the ratio of the torso length of a human body in a sitting position to the torso length in a semi-reclining position, or each first torso correction coefficient is the ratio of the torso length of a human body in a semi-reclining position to the torso length in a sitting position; or, when the absolute head area is less than the second area threshold After determining that the target human body is in a fully lying posture, the method further includes: based on a second correction coefficient, correcting the target torso length to obtain a corrected target torso length, wherein the second correction coefficient is determined based on multiple second torso correction coefficients, the second correction coefficient is greater than or equal to the minimum value of the multiple second torso correction coefficients, and less than or equal to the maximum value of the multiple second torso correction coefficients; each second torso correction coefficient is the ratio of the torso length of a human body in a sitting posture to the torso length in a fully lying posture, or each second torso correction coefficient is the ratio of the torso length of a human body in a fully lying posture to the torso length in a sitting posture; wherein the target torso length is determined based on the torso key points of the target human body, and the torso key points are obtained by identifying the target human body in the image to be identified through a human key point detection network.

[0010] In some embodiments of the present application, the product of the height and width of the human head external frame is determined as the absolute head area of ​​the target person, including: when the target torso length is less than or equal to the torso length threshold, the product of the height and width of the human head external frame is determined as the absolute head area of ​​the target person; wherein the target torso length is determined based on the torso key points of the target person, and the torso key points are obtained by identifying the target person in the image to be identified through a human key point detection network.

[0011] In some embodiments of the present application, the target detection network includes: a human head detection and classification network and a human key point detection network; based on the target detection network, the target human in the image to be identified is identified to obtain the target human head external frame, including: based on the human head detection and classification network, the image to be identified is identified to obtain the head external frame and the target human body external frame; the target human image to be identified corresponding to the target human body external frame is extracted from the image to be identified; based on the human key point detection network, the target human in the target human image to be identified is identified to obtain multiple head key points; based on the multiple head key points, a target head key point external frame corresponding to the target human body external frame is determined; the target intersection-union ratio of the head external frame and the target head key point external frame is calculated; when the target intersection-union ratio is greater than or equal to the intersection-union ratio threshold, it is determined that the head external frame matches the target human body external frame to determine that the head external frame belongs to the target human.

[0012] In a second aspect, an embodiment of the present application provides a lying posture determination device, including: an identification module, used to identify a target human body in an image to be identified based on a target detection network, and obtain an external frame of the target human body's head, wherein the image to be identified is an image of the interior environment of a cabin taken by a camera in a vehicle; a determination module, used to determine the product of the height and width of the external frame of the human head as the absolute head area of ​​the target human body; the determination module is also used to determine that the target human body is in a lying posture when the absolute head area is less than or equal to a first area threshold.

[0013] In some embodiments of the present application, the recognition module is specifically used to identify the target human body in the image to be recognized based on the target detection network, and obtain the external frame of the head and the confidence of the external frame of the head; the determination module is specifically used to determine the product of the height and width of the external frame of the head as the absolute head area of ​​the target human body when the confidence of the external frame of the head is greater than or equal to the confidence threshold of the head.

[0014] In some embodiments of the present application, the determination module is specifically used to determine that the target body is in a semi-lying state when the absolute head area is greater than or equal to the second area threshold and less than or equal to the first area threshold; and to determine that the target body is in a fully lying posture when the absolute head area is less than the second area threshold; wherein the second area threshold is less than the first area threshold.

[0015] In some embodiments of the present application, the device also includes: a correction module, which is used to, when the absolute head area is greater than or equal to the second area threshold and less than or equal to the first area threshold, after determining that the target human body is in a semi-reclining state, correct the target torso length based on the first correction coefficient to obtain a corrected target torso length, wherein the first correction coefficient is determined based on multiple first torso correction coefficients, the first correction coefficient is greater than or equal to the minimum value of the multiple first torso correction coefficients, and is less than or equal to the maximum value of the multiple first torso correction coefficients; each first torso correction coefficient is the ratio of the torso length of a human body in a sitting position to the torso length in a semi-reclining position, or each first torso correction coefficient is the ratio of the torso length of a human body in a semi-reclining position to the torso length in a sitting position; or, the correction module is used to, when the absolute head area is greater than or equal to the second area threshold and less than or equal to the first area threshold, correct the target torso length based on the first correction coefficient to obtain a corrected target torso length When the area of ​​the target human body is less than the second area threshold, after determining that the target human body is in a fully lying posture, the target torso length is corrected based on the second correction coefficient to obtain a corrected target torso length, wherein the second correction coefficient is determined based on multiple second torso correction coefficients, the second correction coefficient is greater than or equal to the minimum value of the multiple second torso correction coefficients, and less than or equal to the maximum value of the multiple second torso correction coefficients; each second torso correction coefficient is the ratio of the torso length of a human body in a sitting posture to the torso length in a fully lying posture, or each second torso correction coefficient is the ratio of the torso length of a human body in a fully lying posture to the torso length in a sitting posture; wherein the target torso length is determined based on the torso key points of the target human body, and the torso key points are obtained by identifying the target human body in the image to be identified through a human key point detection network.

[0016] In some embodiments of the present application, a determination module is specifically used to determine the absolute head area of ​​the target person by the product of the height and width of the human head circumference frame when the target torso length is less than or equal to a torso length threshold; wherein the target torso length is determined based on the torso key points of the target person, and the torso key points are obtained by identifying the target person in the image to be identified through a human key point detection network.

[0017] In some embodiments of the present application, the target detection network includes: a human head detection and classification network and a human key point detection network; a recognition module, which is specifically used to identify the image to be identified based on the human head detection and classification network to obtain a head external frame and a target human external frame; extract the target human image to be identified corresponding to the target human external frame from the image to be identified; identify the target human body in the target human image to be identified based on the human key point detection network to obtain multiple head key points; based on the multiple head key points, determine the target head key point external frame corresponding to the target human external frame; calculate the target intersection-and-union ratio of the head external frame and the target head key point external frame; when the target intersection-and-union ratio is greater than or equal to the intersection-and-union ratio threshold, determine that the head external frame matches the target human external frame to determine that the head external frame belongs to the target human body.

[0018] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a processor, wherein the processor is used to execute a computer program stored in a memory, wherein when the computer program is executed by the processor, the steps of any one of the lying posture determination methods provided in the first aspect are implemented.

[0019] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of any one of the lying posture determination methods provided in the first aspect are implemented.

[0020] In a fifth aspect, an embodiment of the present application provides a vehicle, comprising: the lying posture determination device as described in the second aspect, or the electronic device as described in the third aspect, or the computer-readable storage medium as described in the fourth aspect.

[0021] In a sixth aspect, an embodiment of the present application provides a computer program product, wherein the computer program product includes a computer program or instructions. When the computer program product runs on a processor, the processor executes the computer program or instructions to implement the steps of the lying posture determination method as described in the first aspect.

[0022] In the seventh aspect, an embodiment of the present application provides a chip, which includes a processor, a memory and a communication interface, the communication interface is coupled to the processor, the memory is used to store programs or instructions that can be run on the processor, and the processor is used to execute the program or instructions to implement the steps of the lying posture determination method as described in the first aspect.

[0023] The technical solution provided by the embodiment of the present application has the following advantages over the prior art: in the embodiment of the present application, the target human body in the image to be identified (the image to be identified is an image of the interior environment of the cabin taken by a camera in the vehicle) is identified based on a target detection network to obtain an external bounding box of the head of the target human body; the product of the height and the width of the external bounding box of the head is determined as the absolute head area of ​​the target human body; when the absolute head area is less than or equal to the first area threshold, it is determined that the target human body is in a lying position. Since in the image of the interior environment of the cabin taken by the camera, the head area of ​​the same human body in a lying position is smaller than that in a sitting position, in this scheme, the product of the height and the width of the head external frame is taken as the absolute head area, and then the absolute head area is compared with the first area threshold. When the absolute head area is less than or equal to the first area threshold, it is determined that the target human body is in a lying position. Therefore, this scheme does not need to judge the human body posture in combination with the depth information of the human body, and there is no need to install an image acquisition device that can collect depth information in the cabin. The requirements for the image acquisition device are low and the cost is low. By using an ordinary image acquisition device to collect a two-dimensional image of the interior environment of the cabin, it is possible to accurately identify whether the human body in the cabin is in a lying position. Compared with the depth image including depth information, the processing data is small, the processing speed is fast, the recognition accuracy is high, and the recognition efficiency is high. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0025] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0026] Figure 1 A schematic diagram of key points of the human body provided for this application;

[0027] Figure 2 A schematic diagram of a process for determining a lying posture provided in this application;

[0028] Figure 3 A schematic diagram of the projection of the torso length of a human body in a lying position and a sitting position in a camera imaging screen provided in this application;

[0029] Figure 4 A schematic flow chart of another lying posture determination method provided in the present application;

[0030] Figure 5 A schematic flow chart of another lying posture determination method provided in this application;

[0031] Figure 6 A schematic structural diagram of a lying posture determination device provided in the present application;

[0032] Figure 7 A schematic diagram of the hardware structure of an electronic device provided in this application. DETAILED DESCRIPTION

[0033] In order to more clearly understand the above-mentioned purposes, features and advantages of the present application, the scheme of the present application will be further described below. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0034] In the following description, many specific details are set forth to facilitate a full understanding of the present application, but the present application may also be implemented in other ways different from those described herein; obviously, the embodiments in the specification are only part of the embodiments of the present application, rather than all of the embodiments.

[0035] The terms "first", "second", etc. in the specification and claims of this application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first", "second", etc. are generally of the same type, and the number of objects is not limited. For example, the first object can be one or more.

[0036] First, some nouns or terms involved in the claims and description of the present invention are explained below.

[0037] Cabin: The interior space of a car, where the driver and passengers sit.

[0038] Lying posture: In the cabin, some seats have a large adjustable backrest range and can be laid flat. At this time, the posture of a person lying on the seat is defined as lying posture, or simply lying posture. Generally, it refers to the front row seats. The embodiments of this application are described using the front row seats as an example. The lying posture is further divided into a semi-lying posture and a full-lying posture.

[0039] Semi-reclining posture: The person lies on the seat, but the lower body is in the same normal sitting position, except that the upper body lies flat and the calves are not placed on the seat.

[0040] Fully reclining posture: The person lies on the seat, with the whole body completely flat, the body roughly in a straight line, and the calves placed on the seat.

[0041] Human head detection and classification network: A deep learning network built for detection and classification scenarios. It is usually composed of a backbone network, a neck network, and a head network. It is used to detect whether a human body or a human head exists and return the human body external frame and the head external frame.

[0042] Human key point detection network: A deep learning network built for detection scenarios. Usually composed of a backbone network, a shoulder network, and a head network, it is used to detect whether there are human key points and return the coordinates and confidence of the human key points.

[0043] Absolute head area: The pixel area of ​​the head bounding box, which is obtained by multiplying the height and width of the head bounding box.

[0044] Head confidence threshold: The standard for judging whether the detected head bounding box is credible.

[0045] Absolute head area threshold: the standard for judging whether a person is in a lying position, which is subdivided into the first area threshold and the second area threshold, and is used to further judge whether the person's posture is a semi-lying posture or a fully lying posture. The first area threshold is greater than the second area threshold.

[0046] Torso length: The straight-line distance from the top of the head to the hips, often used to assist in judging human attributes, is calculated from the coordinates of the key points (top of the head key point, left crotch key point, right crotch key point and lower crotch key point) obtained by the human key point detection network.

[0047] Torso correction coefficient: Due to the installation position of the camera, the shooting angle, and the different distances between the human body and the camera, the torso length of the same person is very different when sitting and lying down. It needs to be corrected by a coefficient, which is called the torso correction coefficient.

[0048] Two-stage judgment: The lying posture can be divided into semi-lying posture and full-lying posture, which correspond to different torso length repair coefficients respectively. Therefore, it needs to be divided into two stages for judgment.

[0049] like Figure 1As shown in the figure, it is a schematic diagram of the key points of the human body, the mark "1" indicates the key point of the top of the head, the mark "2" indicates the key point of the right ear, the mark "3" indicates the key point of the left ear, the mark "4" indicates the key point of the jaw, the mark "5" indicates the key point of the neck, the mark "6" indicates the key point of the right shoulder, the mark "7" indicates the key point of the right elbow, the mark "8" indicates the key point of the right wrist, the mark "9" indicates the key point of the right finger, the mark "10" indicates the key point of the left shoulder, the mark "11" indicates the key point of the left elbow, and the mark "12" indicates the key point of the left wrist. Mark “13” indicates the left finger key point, mark “14” indicates the chest key point, mark “15” indicates the waist key point, mark “16” indicates the right crotch key point, mark “17” indicates the lower crotch key point, mark “18” indicates the left crotch key point, mark “19” indicates the right knee key point, mark “20” indicates the right ankle key point, mark “21” indicates the right toe key point, mark “22” indicates the left knee key point, mark “23” indicates the left ankle key point, and mark “24” indicates the left toe key point.

[0050] An embodiment of the present application proposes a lying posture determination method, which identifies a target human body in an image to be identified (the image to be identified is an image of the interior environment of a cabin taken by a camera in a vehicle) based on a target detection network to obtain a head external frame of the target human body; the product of the height and the width of the head external frame is determined as the absolute head area of ​​the target human body; when the absolute head area is less than or equal to a first area threshold, it is determined that the target human body is in a lying posture. Since in the image of the interior environment of the cabin taken by the camera, the head area of ​​the same human body in a lying position is smaller than that in a sitting position, in this scheme, the product of the height and the width of the head external frame is taken as the absolute head area, and then the absolute head area is compared with the first area threshold. When the absolute head area is less than or equal to the first area threshold, it is determined that the target human body is in a lying position. Therefore, this scheme does not need to judge the human body posture in combination with the depth information of the human body, and there is no need to install an image acquisition device that can collect depth information in the cabin. The requirements for the image acquisition device are low and the cost is low. By using an ordinary image acquisition device to collect a two-dimensional image of the interior environment of the cabin, it is possible to accurately identify whether the human body in the cabin is in a lying position. Compared with the depth image including depth information, the processing data is small, the processing speed is fast, the recognition accuracy is high, and the recognition efficiency is high.

[0051] The present application is applied to the scenario of human lying posture recognition in a cockpit environment. The electronic device in the embodiment of the present application can be a vehicle-mounted terminal, or a mobile phone, notebook, computer, etc. that is connected to the vehicle for communication. The specific details can be determined based on actual conditions and are not limited here.

[0052] The technical solution of the present application is explained in detail below through several specific embodiments.

[0053] Figure 2 A schematic diagram of a lying posture determination method provided in this application, such as Figure 2 As shown, the lying posture determination method may include the following steps 101 to 103.

[0054] 101. Based on the target detection network, the target person in the image to be identified is identified to obtain the external frame of the target person's head. The image to be identified is an image of the interior environment of the cabin taken by a camera in the vehicle.

[0055] When performing head recognition, it is necessary to match the human body and the head. Therefore, under normal circumstances, the human head external frame will not be detected alone. The human body external frame and the head external frame will be detected, and then the human body and the head will be matched during the detection process.

[0056] In some embodiments of the present application, the target detection network includes a human body detection network and a head detection network. The human body detection network recognizes the image to be recognized to obtain a human body external bounding box; based on the human body external bounding box and the image to be recognized, a human body image to be recognized corresponding to the human body external bounding box is obtained; the head detection network recognizes the human body image to be recognized to obtain a head external bounding box, so that it can be ensured that the human body external bounding box and the head external bounding box must be completely matched.

[0057] The training process of the above-mentioned human body detection network and head detection network is as follows: collect a large amount of diverse image data in the cockpit environment, annotate the coordinate information of the human body external bounding box for each image, and then obtain the first sample training data based on each image and the coordinate information of the human body external bounding box in the image; based on the first training sample data, train the basic network composed of the backbone network, the shoulder network, and the head network to train a human body detection network. Separate at least one human body image from each image based on each external bounding box, annotate the coordinate information of the head external bounding box in each human body image, and then obtain the second sample training data based on each human body image and the coordinate information of the head external bounding box in each human body image; based on the second training sample data, train the basic network composed of the backbone network, the shoulder network, and the head network to train a head external bounding box detection network.

[0058] In some embodiments of the present application, when identifying the external frame of a head, the human body recognition is first performed on the image to be recognized (the image of the interior environment of the cabin taken by the camera in the vehicle) through the human body detection network to obtain the external frame of the human body, and then the human body image to be recognized corresponding to the external frame of the human body is extracted from the image to be recognized, and the human head detection is performed on the human body image to be recognized through the head detection network to obtain the external frame of the head. Among them, before the human body recognition is performed on the image to be recognized through the human body detection network, the image to be recognized needs to be preprocessed, and after the human body recognition is performed on the image to be recognized through the human body detection network, the human body recognition result needs to be post-processed; similarly, before the human head detection is performed on the human body image to be recognized through the head detection network, the human body image to be recognized needs to be preprocessed, and after the human head recognition is performed on the human body image to be recognized through the head detection network, the human head recognition result needs to be post-processed; the specific preprocessing and post-processing processes can refer to the relevant technology, which will not be repeated here.

[0059] 102. The product of the height and width of the head external frame is determined as the absolute head area of ​​the target person.

[0060] 103. When the absolute head area is less than or equal to the first area threshold, determine that the target human body is in a lying position.

[0061] The first area threshold may be determined according to actual conditions and is not limited here.

[0062] When the absolute head area is greater than the first area threshold, it is determined that the target human body is not in a lying position.

[0063] In the embodiment of the present application, the product of the height and the width of the external frame of the human head is taken as the absolute head area, and then the absolute head area is compared with a first area threshold. When the absolute head area is less than or equal to the first area threshold, it is determined that the target human body is in a lying position. Therefore, this scheme does not need to judge the human body posture in combination with the depth information of the human body, and there is no need to install an image acquisition device that can collect depth information in the cockpit. The requirements for the image acquisition device are low and the cost is low. By using an ordinary image acquisition device to collect a two-dimensional image of the internal environment of the cockpit, it can be accurately identified whether the human body in the cockpit is in a lying position. Compared with the depth image including depth information, the processing data is small, the processing speed is fast, the recognition accuracy is high, and the recognition efficiency is high.

[0064] In the vehicle cabin environment, due to the installation position and camera angle of the camera, as well as the different distances between the human body and the camera, the torso length of the same person in the images taken when sitting and lying down is quite different. Figure 3As shown, the black double solid line on the left is the camera imaging screen, the double arrow solid line is the projection of the torso length in the sitting state, and the double arrow dotted line is the projection of the torso length in the lying state. It can be clearly seen that there is a large gap between the two, and this gap will lead to errors in subsequent human attribute judgment tasks. For example, it is very easy to misidentify an adult in a lying position in the cabin as a child. Therefore, the lying position determination method provided in the embodiment of the present application can accurately identify whether the human body is in a lying position, and then the human body attributes can be judged according to the different postures of the human body, thereby improving the accuracy of human body attribute judgment.

[0065] In some embodiments of the present application, the above step 101 can be specifically implemented through the following step 101a, and the above step 102 can be specifically implemented through the following step 102a.

[0066] 101a. Based on the target detection network, the target human body in the image to be identified is identified to obtain the head bounding box and the confidence level of the head bounding box.

[0067] 102a. When the confidence of the head circumscribed frame is greater than or equal to the head confidence threshold, the product of the height and the width of the head circumscribed frame is determined as the absolute head area of ​​the target person.

[0068] The head confidence threshold can be determined according to actual conditions and is not limited here.

[0069] When the confidence of the head bounding box is less than the head confidence threshold, it is determined that the credibility of the head bounding box is low and cannot be used to determine whether the target human body is in a lying position.

[0070] In the embodiment of the present application, by judging the confidence of the head external frame and the size of the head confidence threshold, the credibility of the head external frame is judged, which can ensure the reliability of the absolute head area, and further ensure the accuracy of the judgment of the human body lying posture based on the absolute head area.

[0071] In some embodiments of the present application, Figure 2 ,like Figure 4 As shown, the above step 103 can be specifically implemented through the following steps 103a and 103b.

[0072] 103a. When the absolute head area is greater than or equal to the second area threshold and less than or equal to the first area threshold, it is determined that the target human body is in a semi-lying state.

[0073] 103b. When the absolute head area is smaller than the second area threshold, it is determined that the target human body is in a fully lying posture.

[0074] The second area threshold is smaller than the first area threshold.

[0075] The specific values ​​of the second area threshold and the first area threshold can be determined according to actual conditions and are not limited here.

[0076] In some embodiments of the present application, since the lying posture can be subdivided into a semi-lying posture and a fully lying posture, a first area threshold and a second area threshold can be set to distinguish whether the target human body is in a fully lying posture or a semi-lying posture, thereby improving the accuracy of judging the lying posture of the human body based on the absolute head area, and further improving the accuracy of subsequent judgments of human attributes based on the lying posture of the human body.

[0077] In some embodiments of the present application, the target detection network includes: a human head detection and classification network and a human key point detection network; the above step 101 can be specifically implemented through the following steps 101b to 101g.

[0078] 101b. Based on the human head detection and classification network, the image to be identified is identified to obtain the external bounding box of the head and the external bounding box of the target human body.

[0079] 101c. Extracting a target human body image to be identified that corresponds to the target human body bounding box from the image to be identified.

[0080] 101d. Based on the human key point detection network, the target human body in the target human body image to be identified is identified to obtain multiple human head key points.

[0081] 101e. Based on the multiple head key points, determine the target head key point bounding box corresponding to the target human bounding box.

[0082] 101f. Calculate the target intersection-union ratio between the head bounding box and the target head key point bounding box.

[0083] The intersection-union ratio is the ratio of the intersection of the area of ​​the head bounding box and the area of ​​the target head key point bounding box to the union of the area of ​​the head bounding box and the area of ​​the target head key point bounding box.

[0084] 101g. When the target intersection-over-union ratio is greater than or equal to the intersection-over-union ratio threshold, determine that the head bounding box matches the target human body bounding box to determine that the head bounding box belongs to the target human body.

[0085] The intersection-over-union ratio threshold can be determined according to actual conditions and is not limited here.

[0086] It can be understood that when the target IoU is less than the IoU threshold, the head bounding box does not match the target human bounding box.

[0087] In some embodiments of the present application, when the target intersection-and-union ratio is the largest of the intersection-and-union ratios of at least one head external bounding box obtained based on the recognition of the image to be recognized and the target head key point external bounding box, and is greater than or equal to the intersection-and-union ratio threshold, it is determined that the head external bounding box matches the target human body external bounding box to determine that the head external bounding box belongs to the target human body.

[0088] It can be understood that when the target detection network includes a human head detection classification network and a human key point detection network, the image to be identified is first identified by the human head classification detection network to obtain at least one human body external bounding box and at least one head external bounding box; then one of the at least one human body external bounding box is used as the target human body external bounding box, and the target human body image to be identified corresponding to the target human body external bounding box is extracted from the image to be identified, and the target human body in the target human body image to be identified is identified based on the human key point detection network to obtain multiple head key points; based on the multiple head key points, a target head key point external bounding box corresponding to the target human body external bounding box is determined; based on at least one head external bounding box and the target head key point external bounding box, the intersection and union ratio of at least one head external bounding box and the target head key point external bounding box is calculated; when the target intersection and union ratio is the largest of the intersection and union ratios of at least one head external bounding box obtained based on the recognition of the image to be identified and the target head key point external bounding box, and is greater than or equal to the intersection and union ratio threshold, it is determined that the head external bounding box matches the target human body external bounding box to determine that the head external bounding box belongs to the target human body.

[0089] The training process of the above-mentioned human head detection classification network and human key point detection network collects a large amount of diverse image data in the cockpit environment, and for each image, the coordinate information of the human body external frame and the coordinate information of the head external frame are annotated, and then based on each image and the coordinate information of the human body external frame and the coordinate information of the head external frame in the image, the third sample training data is obtained; based on the third training sample data, the basic network composed of the backbone network, the shoulder network, and the head network is trained to train a human head detection classification network. Based on each external frame, at least one human image is separated from each image, and for each human image, the coordinate information of the human key points in each human image is annotated, and then based on each human image and the coordinate information of the human key points in each human image, the fourth sample training data is obtained; based on the fourth training sample data, the basic network composed of the backbone network, the shoulder network, and the head network is trained to train a human key point detection network.

[0090] In some embodiments of the present application, when identifying a human body external bounding box and a head external bounding box, a human head recognition is first performed on an image to be identified (an image of the interior environment of the cabin taken by a camera inside the vehicle) through a human head detection and classification network to obtain at least one human body external bounding box and at least one head external bounding box, and one of the at least one human body external bounding box is used as a target human body external bounding box. Then, a target human image to be identified corresponding to the target human body external bounding box is extracted from the image to be identified, and human key point detection is performed on the target human image to be identified through a human key point detection network to obtain multiple head key points. A target human key point external bounding box is determined based on the multiple head key points, and the intersection-and-union ratios of the target human key point external bounding box and at least one head external bounding box are calculated, and the head external bounding box corresponding to the largest intersection-and-union ratio that is greater than or equal to the intersection-and-union ratio threshold is determined as the head external bounding box that matches the target human body external bounding box. Among them, before performing human head recognition on the image to be recognized through the human head detection and classification network, the image to be recognized needs to be preprocessed, and after performing human head recognition on the image to be recognized through the human head detection and classification network, the human head recognition result needs to be post-processed; similarly, before performing human key point detection on the image to be recognized through the human key point detection network, the image to be recognized needs to be preprocessed, and after performing human key point detection on the image to be recognized through the human key point detection network, the human key point recognition result needs to be post-processed; the specific preprocessing and post-processing processes can refer to the relevant technologies, which will not be elaborated here.

[0091] In some embodiments of the present application, the multiple key points of the head include a key point on the top of the head, and the multiple key points also include: a left crotch key point, a right crotch key point and a lower crotch key point; the method for determining the fully lying posture provided in the embodiments of the present application may also include the following step 104.

[0092] 104. Determine the target torso length of the human body based on the top of the head key point, the left crotch key point, the right crotch key point and the lower crotch key point.

[0093] In some embodiments of the present application, the straight-line distance between the point corresponding to the average coordinates of the valid key points (key points with confidence greater than a certain threshold) among the left crotch key point, the right crotch key point and the lower crotch key point and the top of the head key point can be determined as the target torso length.

[0094] In an embodiment of the present application, after the above-mentioned step 101, the target torso length is calculated, which is conducive to determining whether to subsequently judge whether the human body is in a fully lying posture, and / or, is conducive to determining whether the human body is in a fully lying posture, and / or, is conducive to correcting the target torso length when it is determined that the human body is in a fully lying posture, which can improve processing efficiency.

[0095] In some embodiments of the present application, Figure 4 ,like Figure 5 As shown, after the above step 103a, the method for determining the fully lying posture provided in the embodiment of the present application may further include the following step 105.

[0096] 105. Based on the first correction coefficient, the target torso length is corrected to obtain a corrected target torso length.

[0097] Among them, the first correction coefficient is determined based on multiple first torso correction coefficients, the first correction coefficient is greater than or equal to the minimum value of the multiple first torso correction coefficients, and less than or equal to the maximum value of the multiple first torso correction coefficients; each first torso correction coefficient is the ratio of the torso length of a human body in a sitting position to the torso length in a semi-reclining position, or each first torso correction coefficient is the ratio of the torso length of a human body in a semi-reclining position to the torso length in a sitting position.

[0098] Among them, the first correction coefficient can be the average value of multiple first torso correction coefficients, the minimum value among multiple first torso correction coefficients, the maximum value among multiple first torso correction coefficients, or the expected value after normal distribution fitting of multiple first torso correction coefficients.

[0099] The target torso length is determined based on the torso key points of the target human body, which are obtained by identifying the target human body in the image to be identified through a human key point detection network. The specific description of determining the target torso length can refer to the relevant description in the above step 104, which will not be repeated here.

[0100] In an embodiment of the present application, the target torso length is corrected based on the first correction coefficient to obtain a corrected target torso length, which is helpful for judging the attributes of a human body in a semi-reclining state based on the corrected target torso length, and can improve the accuracy of judging the attributes of the human body.

[0101] In some embodiments of the present application, Figure 4 ,like Figure 5 As shown, after the above step 103b, the method for determining the fully lying posture provided in the embodiment of the present application may further include the following step 106.

[0102] 106. Based on the second correction coefficient, the target torso length is corrected to obtain a corrected target torso length.

[0103] Among them, the second correction coefficient is determined based on multiple second torso correction coefficients, the second correction coefficient is greater than or equal to the minimum value of the multiple second torso correction coefficients, and less than or equal to the maximum value of the multiple second torso correction coefficients; each second torso correction coefficient is the ratio of the torso length of a human body in a sitting position to the torso length in a fully lying position, or each second torso correction coefficient is the ratio of the torso length of a human body in a fully lying position to the torso length in a sitting position.

[0104] Among them, the second correction coefficient can be the average value of multiple second torso correction coefficients, the minimum value among multiple second torso correction coefficients, the maximum value among multiple second torso correction coefficients, or the expected value after normal distribution fitting of multiple second torso correction coefficients.

[0105] The target torso length is determined based on the torso key points of the target human body, which are obtained by identifying the target human body in the image to be identified through a human key point detection network. The specific description of determining the target torso length can refer to the relevant description in the above step 104, which will not be repeated here.

[0106] In the embodiment of the present application, the target torso length is corrected based on the second correction coefficient to obtain a corrected target torso length, which is helpful for judging the attributes of a human body in a fully lying state based on the corrected target torso length, and can improve the accuracy of judging the attributes of the human body.

[0107] In some embodiments of the present application, the above step 102 can be specifically implemented through the following step 102b.

[0108] 102b. When the target torso length is less than or equal to the torso length threshold, the product of the height and the width of the head circumscribed frame is determined as the absolute head area of ​​the target human body.

[0109] The target torso length is determined based on the torso key points of the target human body, which are obtained by identifying the target human body in the image to be identified through a human key point detection network. The specific description of determining the target torso length can refer to the relevant description in the above step 104, which will not be repeated here.

[0110] The trunk length threshold can be determined according to actual conditions and is not limited here. For example, the trunk length threshold can be a critical value of the trunk length of an adult and a child.

[0111] In the embodiment of the present application, when the target torso length is greater than the torso length threshold, it is not necessary to determine whether the target human body is in a lying position, and it can be determined that the human body is an adult; when the target torso length is less than or equal to the torso length threshold, it is impossible to determine whether the target human body is an adult or a child, so it is necessary to determine whether the target human body is in a lying position to further assist in determining whether the target human body is an adult or a child. Therefore, by determining whether the target torso length is less than or equal to the torso length threshold, it is possible to avoid determining whether the target human body is in a lying position in all cases. For example, it is not necessary to determine whether the target human body is in a lying position when the target torso length is greater than the torso length threshold, which can save calculation amount and improve judgment efficiency.

[0112] Exemplarily, for the target human body in the image to be identified, find the target head bounding box that matches the target human body, and denoted the confidence of the target head bounding box as C, the width of the target head bounding box as W, and the height as H; simply denoted the result obtained by the human key point detection network as n; denoted the human key point-torso length mapping function as F; denoted the first torso correction coefficient corresponding to the semi-lying posture as a, and the second torso correction coefficient corresponding to the fully lying posture as b, b>a; denoted the head confidence threshold as Th, the first area threshold as S1, and the second area threshold as S2, S1>S2.

[0113] Calculate the target torso length h=F(n).

[0114] Determine whether the head frame is credible:

[0115] If C < Th, the external frame of the head is unreliable and the process terminates;

[0116] If C≥Th, the head bounding box is credible and the next step is to proceed.

[0117] Calculate the absolute head area S = W*H.

[0118] Determine whether the human body is in a lying position:

[0119] If S>S1, the target person is not in a lying position, and the process terminates.

[0120] If S≤S1 and S>S2, the target human body is in a semi-reclining posture, and enters the torso length correction processing flow in the semi-reclining posture.

[0121] If S≤S2, the target human body is in a fully lying posture, and enters the torso length correction processing flow in the fully lying posture.

[0122] The target human body is in a semi-lying posture, and the torso length is corrected by the formula h1=h*a;

[0123] The target human body is in a fully lying position, and the torso length is corrected by the formula h2=h*b.

[0124] In the embodiment of the present application, it is very important to distinguish between sitting posture and lying posture in the interior environment of the cabin, which will directly affect the accuracy of human attribute judgment. However, in the past, there was a lack of such a method for distinguishing. The present invention proposes a lying posture judgment scheme based on absolute head area inside the cabin. It starts from a large amount of data and finds that when the human body is in a lying posture, the absolute head area of ​​the human head is much smaller than other postures due to the different angles and distances facing the camera. Therefore, this characteristic can be used to distinguish between the lying posture and other postures. Furthermore, in the embodiment of the present application, the credibility of the human head external frame is judged to ensure the reliability of its absolute head area, and the first area threshold and the second area threshold are introduced to subdivide the lying posture into a semi-lying posture and a fully lying posture, which improves the accuracy of subsequent human attribute judgments.

[0125] In the embodiment of the present application, by accurately identifying whether the human body in the image in the cabin environment is in a lying posture, the human body in the lying posture is separated from the image, which greatly reduces the false detection of adults in the lying posture as children, and greatly improves the accuracy of the human attribute judgment results inside the cabin.

[0126] The present application also provides a lying posture determination device, Figure 6 A schematic diagram of a lying posture determination device provided in this application is shown in FIG. Figure 6 As shown, the lying posture determination device includes: an identification module 601, which is used to identify the target human body in the image to be identified based on the target detection network to obtain the external frame of the head of the target human body, and the image to be identified is an image of the internal environment of the cabin taken by a camera in the vehicle; a determination module 602, which is used to determine the product of the height and width of the external frame of the head as the absolute head area of ​​the target human body; the determination module is also used to determine that the target human body is in a lying posture when the absolute head area is less than or equal to the first area threshold.

[0127] In some embodiments of the present application, the recognition module 601 is specifically used to identify the target human body in the image to be identified based on the target detection network, and obtain the head external frame and the confidence of the head external frame; the determination module is specifically used to determine the product of the height and width of the head external frame as the absolute head area of ​​the target human body when the confidence of the head external frame is greater than or equal to the head confidence threshold.

[0128] In some embodiments of the present application, the determination module 602 is specifically used to determine that the target body is in a semi-lying state when the absolute head area is greater than or equal to the second area threshold and less than or equal to the first area threshold; and to determine that the target body is in a fully lying posture when the absolute head area is less than the second area threshold; wherein the second area threshold is less than the first area threshold.

[0129] In some embodiments of the present application, the device also includes: a correction module, which is used to, when the absolute head area is greater than or equal to the second area threshold and less than or equal to the first area threshold, after determining that the target human body is in a semi-reclining state, correct the target torso length based on the first correction coefficient to obtain a corrected target torso length, wherein the first correction coefficient is determined based on multiple first torso correction coefficients, the first correction coefficient is greater than or equal to the minimum value of the multiple first torso correction coefficients, and is less than or equal to the maximum value of the multiple first torso correction coefficients; each first torso correction coefficient is the ratio of the torso length of a human body in a sitting position to the torso length in a semi-reclining position, or each first torso correction coefficient is the ratio of the torso length of a human body in a semi-reclining position to the torso length in a sitting position; or, the correction module is used to, when the absolute head area is greater than or equal to the second area threshold and less than or equal to the first area threshold, correct the target torso length based on the first correction coefficient to obtain a corrected target torso length When the area of ​​the target human body is less than the second area threshold, after determining that the target human body is in a fully lying posture, the target torso length is corrected based on the second correction coefficient to obtain a corrected target torso length, wherein the second correction coefficient is determined based on multiple second torso correction coefficients, the second correction coefficient is greater than or equal to the minimum value of the multiple second torso correction coefficients, and less than or equal to the maximum value of the multiple second torso correction coefficients; each second torso correction coefficient is the ratio of the torso length of a human body in a sitting posture to the torso length in a fully lying posture, or each second torso correction coefficient is the ratio of the torso length of a human body in a fully lying posture to the torso length in a sitting posture; wherein the target torso length is determined based on the torso key points of the target human body, and the torso key points are obtained by identifying the target human body in the image to be identified through a human key point detection network.

[0130] In some embodiments of the present application, the determination module 602 is specifically used to determine the absolute head area of ​​the target person by the product of the height and width of the human head external frame when the target torso length is less than or equal to the torso length threshold; wherein the target torso length is determined based on the torso key points of the target person, and the torso key points are obtained by identifying the target person in the image to be identified through a human key point detection network.

[0131] In some embodiments of the present application, the target detection network includes: a human head detection and classification network and a human key point detection network; an identification module 601, which is specifically used to identify the image to be identified based on the human head detection and classification network to obtain a head external frame and a target human external frame; extract a target human image to be identified corresponding to the target human external frame from the image to be identified; identify the target human body in the target human image to be identified based on the human key point detection network to obtain multiple head key points; based on the multiple head key points, determine the target head key point external frame corresponding to the target human external frame; calculate the target intersection-and-union ratio of the head external frame and the target head key point external frame; when the target intersection-and-union ratio is greater than or equal to the intersection-and-union ratio threshold, determine that the head external frame matches the target human external frame to determine that the head external frame belongs to the target human body.

[0132] It should be noted that the above-mentioned lying posture determination device can be the electronic device in the above-mentioned method embodiment of the present application, or it can be a functional module and / or functional entity in the electronic device that can realize the functions of the device embodiment, and the embodiment of the present application is not limited.

[0133] In the embodiments of the present application, each module can implement the lying posture determination method provided in the above method embodiments and can achieve the same technical effect. To avoid repetition, it will not be described here.

[0134] Figure 7 The structural diagram of an electronic device provided in the embodiment of the present application is used to exemplarily illustrate the electronic device that implements the method for determining an arbitrary lying posture in the embodiment of the present application, and should not be understood as a specific limitation on the embodiment of the present application.

[0135] like Figure 7 As shown, the electronic device 700 may include a processor (e.g., a central processing unit, a graphics processing unit, etc.) 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage device 708 into a random access memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the electronic device 700 are also stored. The processor 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0136] Typically, the following devices may be connected to the I / O interface 705: input devices 706 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 707 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 708 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 709. The communication device 709 may allow the electronic device 700 to communicate with other devices wirelessly or by wire to exchange data. Although the electronic device 700 having various devices is shown, it should be understood that it is not required to implement or have all the devices shown. More or fewer devices may be implemented or have alternatively.

[0137] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 709, or installed from the storage device 708, or installed from the ROM 702. When the computer program is executed by the processor 701, the functions defined in any lying posture determination method provided in the embodiment of the present application can be executed.

[0138] It should be noted that the computer-readable medium mentioned above in the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0139] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0140] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0141] The above-mentioned computer-readable medium carries one or more programs. When the above-mentioned one or more programs are executed by the electronic device, the electronic device: identifies the target human body in the image to be identified based on the target detection network, and obtains the external frame of the head of the target human body, where the image to be identified is an image of the internal environment of the cabin taken by a camera in the vehicle; determines the product of the height and width of the external frame of the head as the absolute head area of ​​the target human body; and determines that the target human body is in a lying position when the absolute head area is less than or equal to a first area threshold.

[0142] In an embodiment of the present application, a computer program code for performing the operation of the present application can be written in one or more programming languages ​​or a combination thereof, and the above-mentioned programming languages ​​include but are not limited to object-oriented programming languages, such as Java, Smalltalk, C++, and also include conventional procedural programming languages, such as "C" language or similar programming languages. The program code can be executed completely on the computer, partially on the computer, as an independent software package, partially on the computer and partially on the remote computer, or completely on the remote computer or server. In the case of a remote computer, the remote computer can be connected to the computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, using an Internet service provider to connect through the Internet).

[0143] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0144] The units involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of a unit does not, in some cases, constitute a limitation on the unit itself.

[0145] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0146] In the context of the present application, a computer-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. A computer-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. A more specific example of a computer-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0147] An embodiment of the present application further provides a vehicle, comprising: the above-mentioned lying posture determination device, or the above-mentioned electronic device, or the above-mentioned computer-readable storage medium.

[0148] The above description is only a preferred embodiment of the present application and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present application is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in this application (but not limited to) by each other to form a technical solution.

[0149] In addition, although each operation is described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or to be performed in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be construed as limiting the scope of the application. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0150] Although the subject matter has been described in language specific to structural features and / or methodological logical actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims.

Claims

1. A method for determining a lying posture, characterized in that: The method comprises: Based on the target detection network, a target human body in the image to be identified is identified to obtain a head external frame of the target human body, wherein the image to be identified is an image of the interior environment of the cabin taken by a camera in the vehicle; The product of the height and the width of the human head circumference frame is determined as the absolute head area of ​​the target human body; When the absolute head area is less than or equal to the first area threshold, it is determined that the target human body is in a lying position.

2. The method according to claim 1, characterized in that The method of identifying a target human body in the image to be identified based on the target detection network and obtaining a head external frame of the target human body includes: Based on the target detection network, the target human body in the image to be identified is identified to obtain the human head external bounding box and the confidence of the human head external bounding box; The step of determining the absolute head area of ​​the target human body by multiplying the height and the width of the human head circumference frame includes: When the confidence of the head circumference frame is greater than or equal to the head confidence threshold, the product of the height and width of the head circumference frame is determined as the absolute head area of ​​the target person.

3. The method according to claim 1, characterized in that When the absolute head area is less than or equal to a first area threshold, determining that the target human body is in a lying position includes: When the absolute head area is greater than or equal to the second area threshold and less than or equal to the first area threshold, determining that the target human body is in a semi-lying state; When the absolute head area is less than the second area threshold, determining that the target human body is in a fully lying posture; The second area threshold is smaller than the first area threshold.

4. The method according to claim 3, characterized in that: After determining that the target human body is in a semi-lying state when the absolute head area is greater than or equal to the second area threshold and less than or equal to the first area threshold, the method further includes: Based on the first correction coefficient, the target torso length is corrected to obtain the corrected target torso length, wherein the first correction coefficient is determined based on a plurality of first torso correction coefficients, the first correction coefficient is greater than or equal to a minimum value among the plurality of first torso correction coefficients, and less than or equal to a maximum value among the plurality of first torso correction coefficients; each first torso correction coefficient is a ratio of a torso length of a human body in a sitting position to a torso length in a semi-reclining position, or each first torso correction coefficient is a ratio of a torso length of a human body in a semi-reclining position to a torso length in a sitting position; or, In the case where the absolute head area is less than the second area threshold, after determining that the target human body is in a fully lying posture, the method further includes: Based on the second correction coefficient, the target torso length is corrected to obtain the corrected target torso length, wherein the second correction coefficient is determined based on a plurality of second torso correction coefficients, the second correction coefficient is greater than or equal to a minimum value among the plurality of second torso correction coefficients, and less than or equal to a maximum value among the plurality of second torso correction coefficients; each second torso correction coefficient is a ratio of a torso length of a human body in a sitting position to a torso length in a fully lying position, or each second torso correction coefficient is a ratio of a torso length of a human body in a fully lying position to a torso length in a sitting position; Among them, the target torso length is determined based on the torso key points of the target human body, and the torso key points are obtained by identifying the target human body in the image to be identified through a human key point detection network.

5. The method according to claim 1, characterized in that: The step of determining the absolute head area of ​​the target human body by multiplying the height and the width of the human head circumference frame includes: When the target torso length is less than or equal to the torso length threshold, the product of the height and the width of the head circumscribed frame is determined as the absolute head area of ​​the target human body; Among them, the target torso length is determined based on the torso key points of the target human body, and the torso key points are obtained by identifying the target human body in the image to be identified through a human key point detection network.

6. The method according to any one of claims 1 to 5, characterized in that The target detection network includes: a human head detection classification network and a human key point detection network; The method of identifying a target human body in the image to be identified based on the target detection network and obtaining a head external frame of the target human body includes: Based on the human head detection and classification network, the image to be identified is identified to obtain the human head external bounding box and the target human body external bounding box; Extracting a target human body image to be identified corresponding to the target human body external bounding box from the image to be identified; Based on the human key point detection network, a target human body in the target human body image to be identified is identified to obtain a plurality of human head key points; Based on the multiple head key points, determine a target head key point circumscribed frame corresponding to the target human body circumscribed frame; Calculate the target intersection-over-union ratio of the head bounding box and the target head key point bounding box; When the target intersection-over-union ratio is greater than or equal to the intersection-over-union ratio threshold, it is determined that the head circumscribed frame matches the target human body circumscribed frame to determine that the head circumscribed frame belongs to the target human body.

7. A lying posture determination device, characterized in that: include: A recognition module, used to recognize a target person in an image to be recognized based on a target detection network, and obtain a head external frame of the target person, wherein the image to be recognized is an image of the interior environment of the cabin taken by a camera in the vehicle; A determination module, used for determining the absolute head area of ​​the target human body by multiplying the height and width of the human head circumference frame; The determination module is further used to determine that the target human body is in a lying position when the absolute head area is less than or equal to a first area threshold.

8. An electronic device, characterized in that: include: A processor, wherein the processor is used to execute a computer program stored in a memory, wherein the computer program, when executed by the processor, implements the steps of the lying posture determination method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the lying posture determination method according to any one of claims 1 to 6 are implemented.

10. A vehicle, characterized in that: include: The lying posture determination device as claimed in claim 7, or the electronic device as claimed in claim 8, or includes the computer-readable storage medium as claimed in claim 9.

Images