Lying posture determination method and device, electronic equipment, storage medium and vehicle

Abstract

The invention relates to a lying posture determination method and device, electronic equipment, a storage medium and a vehicle, and the method comprises the steps: carrying out the recognition of a target human body in a to-be-recognized image based on a target detection network, obtaining a human head external frame of the target human body, and enabling the to-be-recognized image to be an image of a cabin internal environment shot by a camera in the vehicle; determining the product of the height and the width of the head external frame as the absolute head area of the target human body; the ratio of the absolute head area to a standard head area is determined as a relative head area, and the standard head area is used for indicating a standard value of the absolute head area of the human body in the sitting posture; and when the relative head area is smaller than or equal to the first head area threshold value, determining that the target human body is in a lying posture. According to the method, whether the human body in the shot cabin environment image is in the lying posture or not can be accurately identified.

Description

Technical Field

[0001] The present application relates to the field of automobile technology, and in particular to a lying posture determination method, device, electronic device, storage medium and vehicle. Background Art

[0002] In the vehicle's cabin environment, due to the camera's installation position and camera angle, as well as the different distances between the human body and the camera, there is a big difference in the torso length (the projection distance from the top of the head to the hips on the camera's imaging screen) of the same person in the images taken when the person is sitting or lying (including fully lying and half-lying). Taking the front-row driver's seat as an example, the camera is installed above the center rearview mirror in the car, and the camera angle is from front to back and tilted downward at a small angle. When the person changes from a sitting position to a lying position, the torso length of the person in the corresponding image will become shorter.

[0003] In the cockpit environment, the torso length of the human body in the image is often used to assist in judging the human attributes, such as judging whether the human body is an adult or a child. However, since the torso length of a reclining human body is much smaller than that of a sitting human body, it is very easy to mistakenly identify a reclining adult as a child.

[0004] Existing human posture recognition algorithms include algorithms based on depth maps. Since the depth map includes the depth information of the human body (i.e., the distance information of each part of the human body from the camera), and the characteristics of the depth information of the human body are obviously different when the human body is in a sitting position or a lying position, the human posture recognition can be performed based on the algorithm of the depth map. However, the algorithm based on the depth map, on the one hand, requires an image acquisition device that can obtain depth information, which has high requirements for the image acquisition device and high cost. On the other hand, the algorithm based on the depth image has a large amount of data processing, a slow recognition speed, and sometimes inaccurate depth recognition and low recognition efficiency. Summary of the invention

[0005] The present application provides a lying posture determination method, device, electronic device, storage medium and vehicle, which can accurately and efficiently identify whether a human body in a captured cabin environment image is in a lying posture.

[0006] In a first aspect, an embodiment of the present application provides a method for determining a lying posture, comprising: identifying a target human body in an image to be identified based on a target detection network to obtain an external frame of the target human body's head, wherein the image to be identified is an image of the interior environment of a cabin taken by a camera in a vehicle; determining the product of the height and the width of the external frame of the human head as the absolute head area of ​​the target human body; determining the ratio of the absolute head area to the standard head area as the relative head area, wherein the standard head area is used to indicate the standard value of the absolute head area of ​​a human body in a sitting posture; and determining that the target human body is in a lying posture when the relative head area is less than or equal to a first head area threshold.

[0007] In some embodiments of the present application, a target human body in an image to be identified is identified based on a target detection network to obtain a head external frame of the target human body, including: identifying the target human body in an image to be identified based on a target detection network to obtain a head external frame and a confidence level of the head external frame; determining the product of the height and width of the head external frame as the absolute head area of ​​the target human body, including: when the confidence level of the head external frame is greater than or equal to a head confidence threshold, determining the product of the height and width of the head external frame as the absolute head area of ​​the target human body.

[0008] In some embodiments of the present application, the standard head area is determined based on multiple reference absolute head areas, and the standard head area is greater than or equal to the minimum value of the multiple reference absolute head areas, and less than or equal to the maximum value of the multiple reference absolute head areas; wherein each reference absolute head area is an absolute head area within the normal head area domain; or, each reference absolute head area is an absolute head area of ​​a human body recorded in a vehicle and within the normal head area domain; or, each reference absolute head area is an absolute head area of ​​the target human body recorded in the vehicle and within the normal head area domain.

[0009] In some embodiments of the present application, the standard head area is an average value of multiple reference absolute head areas; or, the standard head area is an expected value obtained by performing normal distribution fitting on multiple reference absolute head areas.

[0010] In some embodiments of the present application, each reference absolute head area is an absolute head area of ​​a target human body recorded in a vehicle and is within a normal head area domain. The method further includes: when the relative head area is greater than a first head area threshold, determining that the target human body is in a sitting position; when the absolute head area of ​​the target human body is within the normal head area domain, determining the absolute head area of ​​the target human body as a reference absolute head area, so as to determine a standard head area based on multiple reference absolute head areas and the absolute head area of ​​the target human body.

[0011] In some embodiments of the present application, when the relative head area is less than or equal to a first head area threshold, it is determined that the target body is in a lying position, including: when the relative head area is greater than or equal to a second head area threshold, and less than or equal to the first head area threshold, it is determined that the target body is in a semi-lying state; when the relative head area is less than the second head area threshold, it is determined that the target body is in a fully lying posture; wherein the second head area threshold is less than the first head area threshold.

[0012] In some embodiments of the present application, when the relative head area is greater than or equal to the second head area threshold and less than or equal to the first head area threshold, after determining that the target human body is in a semi-reclining state, the method further includes: based on the first correction coefficient, correcting the target torso length to obtain a corrected target torso length, wherein the first correction coefficient is determined based on multiple first torso correction coefficients, each first torso correction coefficient is the ratio of the torso length of a human body in a sitting position to the torso length in a semi-reclining position, or each first torso correction coefficient is the ratio of the torso length of a human body in a semi-reclining position to the torso length in a sitting position; or, when the relative head area is less than the second head area threshold In the case where it is determined that the target human body is in a fully lying posture, the method further includes: based on a second correction coefficient, correcting the target torso length to obtain a corrected target torso length, wherein the second correction coefficient is determined based on multiple second torso correction coefficients, each second torso correction coefficient is the ratio of the torso length of a human body in a sitting posture to the torso length in a fully lying posture, or each second torso correction coefficient is the ratio of the torso length of a human body in a fully lying posture to the torso length in a sitting posture; wherein the target torso length is determined based on the torso key points of the target human body, and the torso key points are obtained by identifying the target human body in the image to be identified through a human key point detection network.

[0013] In some embodiments of the present application, when the relative head area is less than or equal to a first head area threshold, it is determined that the target human body is in a lying position, including: when the relative head area is less than or equal to the first head area threshold, based on the lower limb key points of the target human body, determining the lower limb length of the target human body; based on the torso key points of the target human body, determining the target torso length of the target human body; determining the ratio of the lower limb length to the target torso length as the target ratio; when the target ratio is greater than or equal to the ratio threshold, determining that the target human body is in a lying position; wherein the lower limb key points and the torso key points are obtained by identifying the target human body in the image to be identified through a human key point detection network.

[0014] In a second aspect, an embodiment of the present application provides a lying posture determination device, comprising: a recognition module, used to identify a target human body in an image to be identified based on a target detection network, and obtain a head external frame of the target human body, wherein the image to be identified is an image of the interior environment of a cabin taken by a camera in a vehicle; a determination module, used to determine the product of the height and the width of the head external frame as the absolute head area of ​​the target human body; determine the ratio of the absolute head area to the standard head area as the relative head area, wherein the standard head area is used to indicate the standard value of the absolute head area of ​​a human body in a sitting position; when the relative head area is less than or equal to a first head area threshold, it is determined that the target human body is in a lying position.

[0015] In some embodiments of the present application, the recognition module is specifically used to identify the target human body in the image to be recognized based on the target detection network, and obtain the external frame of the head and the confidence of the external frame of the head; the determination module is specifically used to determine the product of the height and width of the external frame of the head as the absolute head area of ​​the target human body when the confidence of the external frame of the head is greater than or equal to the confidence threshold of the head.

[0016] In some embodiments of the present application, the standard head area is determined based on multiple reference absolute head areas, and the standard head area is greater than or equal to the minimum value of the multiple reference absolute head areas, and less than or equal to the maximum value of the multiple reference absolute head areas; wherein each reference absolute head area is an absolute head area within the normal head area domain; or, each reference absolute head area is an absolute head area of ​​a human body recorded in a vehicle and within the normal head area domain; or, each reference absolute head area is an absolute head area of ​​the target human body recorded in the vehicle and within the normal head area domain.

[0017] In some embodiments of the present application, the standard head area is an average value of multiple reference absolute head areas; or, the standard head area is an expected value obtained by performing normal distribution fitting on multiple reference absolute head areas.

[0018] In some embodiments of the present application, each reference absolute head area is an absolute head area recorded for a target human body in a vehicle and is within a normal head area domain. The determination module is also used to determine that the target human body is in a sitting position when the relative head area is greater than a first head area threshold; when the absolute head area of ​​the target human body is within the normal head area domain, the absolute head area of ​​the target human body is determined as a reference absolute head area, so as to determine a standard head area based on multiple reference absolute head areas and the absolute head area of ​​the target human body.

[0019] In some embodiments of the present application, the determination module is specifically used to determine that the target human body is in a semi-lying state when the relative head area is greater than or equal to a second head area threshold and less than or equal to a first head area threshold; and to determine that the target human body is in a fully lying posture when the relative head area is less than the second head area threshold; wherein the second head area threshold is less than the first head area threshold.

[0020] In some embodiments of the present application, the device also includes: a correction module, which is used to, when the relative head area is greater than or equal to the second head area threshold value and less than or equal to the first head area threshold value, after determining that the target human body is in a semi-reclining state, correct the target torso length based on the first correction coefficient to obtain a corrected target torso length, wherein the first correction coefficient is determined based on multiple first torso correction coefficients, each first torso correction coefficient is the ratio of the torso length of a human body in a sitting position to the torso length in a semi-reclining position, or each first torso correction coefficient is the ratio of the torso length of a human body in a semi-reclining position to the torso length in a sitting position; or, the correction module is used to When the product is less than the second head area threshold, after determining that the target human body is in a fully lying posture, the target torso length is corrected based on the second correction coefficient to obtain a corrected target torso length, wherein the second correction coefficient is determined based on multiple second torso correction coefficients, each second torso correction coefficient is the ratio of the torso length of a human body in a sitting posture to the torso length in a fully lying posture, or each second torso correction coefficient is the ratio of the torso length of a human body in a fully lying posture to the torso length in a sitting posture; wherein the target torso length is determined based on the torso key points of the target human body, and the torso key points are obtained by identifying the target human body in the image to be identified through a human key point detection network.

[0021] In some embodiments of the present application, the multiple key points also include: a left knee key point and a right knee key point; a determination module, specifically used to determine the length of the target body's lower limbs based on the target body's lower limb key points when the relative head area is less than or equal to a first head area threshold; determine the target torso length of the target body based on the target body's torso key points; determine the ratio of the lower limb length to the target torso length as the target ratio; when the target ratio is greater than or equal to the ratio threshold, determine that the target body is in a lying position; wherein the lower limb key points and the torso key points are obtained by identifying the target body in the image to be identified through a human key point detection network.

[0022] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a processor, wherein the processor is used to execute a computer program stored in a memory, wherein when the computer program is executed by the processor, the steps of any one of the lying posture determination methods provided in the first aspect are implemented.

[0023] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of any one of the lying posture determination methods provided in the first aspect are implemented.

[0024] In a fifth aspect, an embodiment of the present application provides a vehicle, comprising: the lying posture determination device as described in the second aspect, or the electronic device as described in the third aspect, or the computer-readable storage medium as described in the fourth aspect.

[0025] In a sixth aspect, an embodiment of the present application provides a computer program product, wherein the computer program product includes a computer program or instructions. When the computer program product runs on a processor, the processor executes the computer program or instructions to implement the steps of the lying posture determination method as described in the first aspect.

[0026] In the seventh aspect, an embodiment of the present application provides a chip, which includes a processor, a memory and a communication interface, the communication interface is coupled to the processor, the memory is used to store programs or instructions that can be run on the processor, and the processor is used to execute the program or instructions to implement the steps of the lying posture determination method as described in the first aspect.

[0027] The technical solution provided by the embodiment of the present application has the following advantages over the prior art: in the embodiment of the present application, the target human body in the image to be identified (the image of the interior environment of the cabin taken by the camera in the vehicle) is identified based on the target detection network to obtain the external frame of the head of the target human body; the product of the height and the width of the external frame of the head is determined as the absolute head area of ​​the target human body; the ratio of the absolute head area to the standard head area is determined as the relative head area; when the relative head area is less than or equal to the first head area threshold, it is determined that the target human body is in a lying position. In this way, this scheme identifies the external frame of the head in the captured image, takes the product of the height and width of the external frame of the head as the absolute head area, and then determines the ratio of the absolute head area to the standard head area (the standard head area is used to indicate the standard value of the absolute head area of ​​a person in a sitting position) as the relative head area, and then compares the relative head area with the first head area threshold. When the relative head area is less than or equal to the first head area threshold, it is determined that the target person is in a lying position. Therefore, this scheme does not need to judge the posture of the person in combination with the depth information of the person, and there is no need to install an image acquisition device that can collect depth information in the cockpit. The requirements for the image acquisition device are low and the cost is low. By using an ordinary image acquisition device to collect a two-dimensional image of the internal environment of the cockpit, it can be accurately identified whether the person in the cockpit is in a lying position. Compared with the depth image including depth information, the processing data is small, the processing speed is fast, the recognition accuracy is high, and the recognition efficiency is high. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0029] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0030] Figure 1 A schematic diagram of key points of the human body provided for this application;

[0031] Figure 2 A schematic diagram of a process for determining a lying posture provided in this application;

[0032] Figure 3 A schematic flow chart of another lying posture determination method provided in the present application;

[0033] Figure 4 A schematic diagram of the recognition process of a human head external frame provided in this application;

[0034] Figure 5 A schematic diagram of another process of identifying a human head external frame provided by this application;

[0035] Figure 6 A schematic structural diagram of a lying posture determination device provided in the present application;

[0036] Figure 7 A schematic diagram of the hardware structure of an electronic device provided in this application. DETAILED DESCRIPTION

[0037] In order to more clearly understand the above-mentioned purposes, features and advantages of the present application, the scheme of the present application will be further described below. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0038] In the following description, many specific details are set forth to facilitate a full understanding of the present application, but the present application may also be implemented in other ways different from those described herein; obviously, the embodiments in the specification are only part of the embodiments of the present application, rather than all of the embodiments.

[0039] The terms "first", "second", etc. in the specification and claims of this application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first", "second", etc. are generally of the same type, and the number of objects is not limited. For example, the first object can be one or more.

[0040] First, some nouns or terms involved in the claims and description of the present invention are explained below.

[0041] Cabin: The interior space of a car, where the driver and passengers sit.

[0042] Lying posture: In the cabin, some seats have a large adjustable backrest range and can be laid flat. At this time, the posture of a person lying on the seat is defined as lying posture, or simply lying posture. Generally, it refers to the front row seats. The embodiments of this application are described using the front row seats as an example. The lying posture is further divided into a semi-lying posture and a full-lying posture.

[0043] Semi-reclining posture: The person lies on the seat, but the lower body is in the same normal sitting position, except that the upper body lies flat and the calves are not placed on the seat.

[0044] Fully reclining posture: The person lies on the seat, with the whole body completely flat, the body roughly in a straight line, and the calves placed on the seat.

[0045] Human head detection and classification network: A deep learning network built for detection and classification scenarios. It is usually composed of a backbone network, a neck network, and a head network. It is used to detect whether a human body or a human head exists and return the human body external frame and the head external frame.

[0046] Human key point detection network: A deep learning network built for detection scenarios. Usually composed of a backbone network, a shoulder network, and a head network, it is used to detect whether there are human key points and return the coordinates and confidence of the human key points.

[0047] Absolute head area: The pixel area of ​​the head bounding box, which is obtained by multiplying the height and width of the head bounding box.

[0048] Head confidence threshold: The standard for judging whether the detected head bounding box is credible.

[0049] Normal head area range: After a large amount of data statistics, the absolute head area is basically within a range when sitting in a relatively standard posture. This range is called the normal head area range.

[0050] Standard head area: The standard value of the absolute head area of ​​a human head that falls within the normal head area range.

[0051] Relative head area: The ratio of absolute head area to standard head area, which shows the degree of scaling of head size.

[0052] Absolute head area threshold: The standard for judging whether a person is in a lying position. It is subdivided into the first head area threshold and the second head area threshold, which are used to further judge whether the person's posture is a semi-lying posture or a fully lying posture. The first head area threshold is greater than the second head area threshold.

[0053] Torso length: The straight-line distance from the top of the head to the hips, often used to assist in judging human attributes, is calculated from the coordinates of the key points (top of the head key point, left crotch key point, right crotch key point and lower crotch key point) obtained by the human key point detection network.

[0054] Torso correction coefficient: Due to the installation position of the camera, the shooting angle, and the different distances between the human body and the camera, the torso length of the same person is very different when sitting and lying down. It needs to be corrected by a coefficient, which is called the torso correction coefficient.

[0055] Two-stage judgment: The lying posture can be divided into semi-lying posture and full-lying posture, which correspond to different torso length repair coefficients respectively. Therefore, it needs to be divided into two stages for judgment.

[0056] like Figure 1 As shown in the figure, it is a schematic diagram of the key points of the human body, the mark "1" indicates the key point of the top of the head, the mark "2" indicates the key point of the right ear, the mark "3" indicates the key point of the left ear, the mark "4" indicates the key point of the jaw, the mark "5" indicates the key point of the neck, the mark "6" indicates the key point of the right shoulder, the mark "7" indicates the key point of the right elbow, the mark "8" indicates the key point of the right wrist, the mark "9" indicates the key point of the right finger, the mark "10" indicates the key point of the left shoulder, the mark "11" indicates the key point of the left elbow, and the mark "12" indicates the key point of the left wrist. Mark “13” indicates the left finger key point, mark “14” indicates the chest key point, mark “15” indicates the waist key point, mark “16” indicates the right crotch key point, mark “17” indicates the lower crotch key point, mark “18” indicates the left crotch key point, mark “19” indicates the right knee key point, mark “20” indicates the right ankle key point, mark “21” indicates the right toe key point, mark “22” indicates the left knee key point, mark “23” indicates the left ankle key point, and mark “24” indicates the left toe key point.

[0057] An embodiment of the present application proposes a lying posture determination method, which identifies a target human body in an image to be identified (an image of the interior environment of a cabin taken by a camera in a vehicle) based on a target detection network to obtain a head external frame of the target human body; the product of the height and the width of the head external frame is determined as the absolute head area of ​​the target human body; the ratio of the absolute head area to the standard head area is determined as the relative head area; and when the relative head area is less than or equal to a first head area threshold, it is determined that the target human body is in a lying posture. In this way, in this scheme, the external frame of the head in the captured image is identified, and the product of the height and width of the external frame of the head is taken as the absolute head area. Then, the ratio of the absolute head area to the standard head area (the standard head area is used to indicate the standard value of the absolute head area of ​​a person in a sitting position) is determined as the relative head area. The relative head area is then compared with the first head area threshold. When the relative head area is less than or equal to the first head area threshold, it is determined that the target person is in a lying position. Therefore, this scheme does not need to judge the posture of the person in combination with the depth information of the person, and there is no need to install an image acquisition device that can collect depth information in the cabin. The requirements for the image acquisition device are low and the cost is low. By using an ordinary image acquisition device to collect a two-dimensional image of the internal environment of the cabin, it is possible to accurately identify whether the person in the cabin is in a lying position. Compared with the depth image including depth information, the processing data is small, the processing speed is fast, the recognition accuracy is high, and the recognition efficiency is high.

[0058] The present application is applied to the scenario of human lying posture recognition in a cockpit environment. The electronic device in the embodiment of the present application can be a vehicle-mounted terminal, or a mobile phone, notebook, computer, etc. that is connected to the vehicle for communication. The specific details can be determined based on actual conditions and are not limited here.

[0059] The technical solution of the present application is explained in detail below through several specific embodiments.

[0060] Figure 2 A schematic diagram of a lying posture determination method provided in this application, such as Figure 2 As shown, the lying posture determination method may include the following steps 101 to 104.

[0061] 101. Based on the target detection network, the target person in the image to be identified is identified to obtain the external frame of the target person's head.

[0062] The image to be recognized is an image of the interior environment of the cabin taken by a camera inside the vehicle.

[0063] When performing head recognition, it is necessary to match the human body and the head. Therefore, under normal circumstances, the human head external frame will not be detected alone. The human body external frame and the head external frame will be detected, and then the human body and the head will be matched during the detection process.

[0064] In some embodiments of the present application, the target detection network includes a human body detection network and a head detection network. The human body detection network recognizes the image to be recognized to obtain a human body external bounding box; based on the human body external bounding box and the image to be recognized, a human body image to be recognized corresponding to the human body external bounding box is obtained; the head detection network recognizes the human body image to be recognized to obtain a head external bounding box, so that it can be ensured that the human body external bounding box and the head external bounding box must be completely matched.

[0065] The training process of the above-mentioned human body detection network and head detection network is as follows: collect a large amount of diverse image data in the cockpit environment, annotate the coordinate information of the human body external bounding box for each image, and then obtain the first sample training data based on each image and the coordinate information of the human body external bounding box in the image; based on the first training sample data, train the basic network composed of the backbone network, the shoulder network, and the head network to train a human body detection network. Separate at least one human body image from each image based on each external bounding box, annotate the coordinate information of the head external bounding box in each human body image, and then obtain the second sample training data based on each human body image and the coordinate information of the head external bounding box in each human body image; based on the second training sample data, train the basic network composed of the backbone network, the shoulder network, and the head network to train a head external bounding box detection network.

[0066] In some embodiments of the present application, Figure 3 As shown, when identifying the external frame of a head, the human body recognition is first performed on the image to be recognized (the image of the interior environment of the cabin taken by the camera in the vehicle) through the human body detection network to obtain the external frame of the human body, and then the human body image to be recognized corresponding to the external frame of the human body is extracted from the image to be recognized, and the human head detection is performed on the human body image to be recognized through the head detection network to obtain the external frame of the head. Among them, before the human body recognition is performed on the image to be recognized through the human body detection network, the image to be recognized needs to be preprocessed, and after the human body recognition is performed on the image to be recognized through the human body detection network, the human body recognition result needs to be post-processed; similarly, before the head detection is performed on the image to be recognized through the head detection network, the human body image to be recognized needs to be preprocessed, and after the head recognition is performed on the image to be recognized through the head detection network, the head recognition result needs to be post-processed; the specific preprocessing and post-processing processes can refer to the relevant technology, which will not be repeated here.

[0067] 102. The product of the height and width of the head external frame is determined as the absolute head area of ​​the target person.

[0068] 103. The ratio of the absolute head area to the standard head area is determined as the relative head area.

[0069] Among them, the standard head area is used to indicate the standard value of the absolute head area of ​​a person in a sitting position.

[0070] In some embodiments of the present application, the standard head area is determined based on multiple reference absolute head areas, and the standard head area is greater than or equal to the minimum value of the multiple reference absolute head areas, and less than or equal to the maximum value of the multiple reference absolute head areas.

[0071] In some embodiments of the present application, the standard head area can be the maximum value among multiple reference absolute head areas, or the minimum value among multiple reference absolute head areas, or the weighted average of multiple reference absolute head areas, or the expected value obtained by normal distribution fitting of multiple reference absolute head areas, or other values, which are not limited here.

[0072] In some embodiments of the present application, the standard head area is the average of multiple reference absolute head areas; or the standard head area is the expected value obtained by normal distribution fitting of multiple reference absolute head areas. In this way, the determined standard head area is closer to the absolute head area of ​​the target body, and the determined relative head area is more accurate, and the posture of the target body can be better determined.

[0073] In some embodiments of the present application, each reference absolute head area is an absolute head area within the normal head area domain; or, each reference absolute head area is an absolute head area of ​​a human body recorded in a vehicle and within the normal head area domain; or, each reference absolute head area is an absolute head area of ​​the target human body recorded in a vehicle and within the normal head area domain. The specifics can be determined according to actual conditions and are not limited here.

[0074] In the embodiments of the present application, a variety of methods for obtaining the standard head area are provided, which can be determined based on actual conditions and are not limited here.

[0075] In some embodiments of the present application, when the number of absolute head areas of human bodies recorded in a vehicle that are located within the normal head area domain is less than or equal to a first quantity threshold, each reference absolute head area is an absolute head area within the normal head area domain; when the number of absolute head areas of human bodies recorded in a vehicle that are located within the normal head area domain is greater than the first quantity threshold, and the number of absolute head areas of target human bodies recorded in the vehicle that are located within the normal head area domain is less than or equal to a second quantity threshold, each reference absolute head area is an absolute head area of ​​the human body recorded in the vehicle that is within the normal head area domain; when the number of absolute head areas of target human bodies recorded in the vehicle that are located within the normal head area domain is greater than the second quantity threshold, each reference absolute head area is an absolute head area of ​​the target human body recorded in the vehicle that is within the normal head area domain.

[0076] The first quantity threshold and the second quantity threshold can be determined according to actual conditions and are not limited here. The recorded absolute head area of ​​the target human body in the vehicle that is located within the normal head area domain belongs to the recorded absolute head area of ​​the human body in the vehicle that is located within the normal head area domain.

[0077] It can be understood that the reference absolute head area can be determined in stages based on the number of absolute head areas of the human bodies in the vehicle recorded within the normal head area range, the number of absolute head areas of the target human bodies in the vehicle recorded within the normal head area range, and the number thresholds corresponding to the two (the first number threshold and the second number threshold, respectively). Then, different standard head areas can be determined based on different multiple reference absolute head areas at different stages, and then the relative head area can be determined based on the ratio of the absolute head area of ​​the target human body to the standard head area. In this way, as time goes by, the reference head area is, in sequence, an absolute head area within the normal head area domain (a reference value of the absolute head area of ​​a wide range of human bodies), an absolute head area of ​​a human body recorded in a vehicle and within the normal head area domain (a reference value of the absolute head area of ​​a human body in a vehicle, which may be a reference value of the absolute head area of ​​a target human body in a vehicle), and an absolute head area of ​​a target human body recorded in a vehicle and within the normal head area domain (a reference value of the absolute head area of ​​a target human body in a vehicle). In this way, the determined standard head area becomes closer and closer to the reference value of the absolute head area of ​​the target human body. As the number of reference head areas increases, the standard head area becomes more and more stable. Therefore, the determined relative head area value becomes more accurate.

[0078] In some embodiments of the present application, each reference absolute head area is an absolute head area within the normal head area domain, that is, the standard head area is determined based on multiple absolute head areas within the normal head area domain.

[0079] In some embodiments of the present application, each reference absolute head area is an absolute head area of ​​a human body recorded in a vehicle and within the normal head area domain, that is, the standard head area is determined based on multiple reference absolute head areas of a human body recorded in a vehicle and within the normal head area domain. At this time, the standard head area changes dynamically, and as time goes by, the number of reference absolute head areas increases, and the standard head area tends to be stable and accurate. For example, the standard head area can be the average of multiple reference absolute head areas of a human body recorded in a vehicle and within the normal head area domain.

[0080] In some embodiments of the present application, each reference absolute head area is an absolute head area of ​​the target human body recorded in the vehicle and within the normal head area domain, that is, the standard head area is determined based on multiple reference absolute head areas of the target human body recorded in the vehicle and within the normal head area domain. At this time, the standard head area changes dynamically, and as time goes by, the number of reference absolute head areas increases, and the standard head area tends to be stable and accurate. For example, the standard head area can be the average of multiple reference absolute head areas of the target human body recorded in the vehicle and within the normal head area domain.

[0081] 104. When the relative head area is less than or equal to the first head area threshold, determine that the target human body is in a lying position.

[0082] When the relative head area is greater than the first head area threshold, it is determined that the target human body is not in a lying position.

[0083] In the embodiment of the present application, the external frame of the head in the captured image is identified, and the product of the height and width of the external frame of the head is taken as the absolute head area. Then, the ratio of the absolute head area to the standard head area (the standard head area is used to indicate the standard value of the absolute head area of ​​a person in a sitting position) is determined as the relative head area. The relative head area is then compared with a first head area threshold. When the relative head area is less than or equal to the first head area threshold, it is determined that the target person is in a lying position. Therefore, this solution does not need to judge the posture of the person in combination with the depth information of the person, and there is no need to install an image acquisition device that can collect depth information in the cabin. The requirements for the image acquisition device are low and the cost is low. By using an ordinary image acquisition device to collect a two-dimensional image of the internal environment of the cabin, it is possible to accurately identify whether the person in the cabin is in a lying position. Compared with the depth image including depth information, the processing data is small, the processing speed is fast, the recognition accuracy is high, and the recognition efficiency is high.

[0084] In some embodiments of the present application, each reference absolute head area is an absolute head area of ​​a target human body recorded in a vehicle and is within a normal head area domain. The method further includes: when the relative head area is greater than a first head area threshold, determining that the target human body is in a sitting position; when the absolute head area of ​​the target human body is within the normal head area domain, determining the absolute head area of ​​the target human body as a reference absolute head area, so as to determine a standard head area based on multiple reference absolute head areas and the absolute head area of ​​the target human body.

[0085] It can be understood that when it is determined that the target human body is in a sitting position and the absolute head area of ​​the target human body is within the normal head area range, the absolute head area of ​​the target human body can be determined as a reference absolute head area to determine the standard head area based on multiple reference absolute head areas and the absolute head area of ​​the target human body. In this way, the number of reference head areas used to determine the standard head area can be increased, making the determined standard head area more accurate, and then making the determined relative head area more accurate, which can improve the accuracy of determining the lying position of the human body.

[0086] In some embodiments of the present application, the above step 101 can be specifically implemented through the following step 101a, and the above step 102 can be specifically implemented through the following step 102a.

[0087] 101a. Based on the target detection network, the target human body in the image to be identified is identified to obtain the head bounding box and the confidence level of the head bounding box.

[0088] 102a. When the confidence of the head circumscribed frame is greater than or equal to the head confidence threshold, the product of the height and the width of the head circumscribed frame is determined as the absolute head area of ​​the target person.

[0089] The head confidence threshold can be determined according to actual conditions and is not limited here.

[0090] When the confidence of the head bounding box is less than the head confidence threshold, it is determined that the credibility of the head bounding box is low and cannot be used to determine whether the target human body is in a lying position.

[0091] In the embodiment of the present application, by judging the confidence of the head external frame and the size of the head confidence threshold, the credibility of the head external frame is judged, which can ensure the reliability of the absolute head area, and further ensure the accuracy of the judgment of the human body lying posture based on the absolute head area.

[0092] In some embodiments of the present application, Figure 2 ,like Figure 4 As shown, the above step 104 can be specifically implemented through the following steps 104a and 104b.

[0093] 104a. When the relative head area is greater than or equal to the second head area threshold and less than or equal to the first head area threshold, it is determined that the target human body is in a semi-lying state.

[0094] 104b. When the relative head area is smaller than the second head area threshold, it is determined that the target human body is in a fully lying posture.

[0095] The second head area threshold is smaller than the first head area threshold. The second head area threshold and the first head area threshold can be determined according to actual conditions and are not limited here.

[0096] In some embodiments of the present application, since the lying posture can be subdivided into a semi-lying posture and a fully lying posture, a first head area threshold and a second head area threshold can be set to distinguish whether the target human body is in a fully lying posture or a semi-lying posture, thereby improving the accuracy of judging the lying posture of the human body based on the relative head area, and further improving the accuracy of subsequent judgments of human attributes based on the lying posture of the human body.

[0097] In some embodiments of the present application, in order to determine the lying position of the target human body, it is necessary to match the human body and the human head in the image to be identified. The specific matching scheme may include the following two schemes. The matching scheme of the human body and the human head in the embodiments of the present application may also be other. For details, please refer to the relevant technology, which will not be elaborated here.

[0098] Matching scheme 1, the target detection network includes: a human body detection network and a head detection network. The human body detection network is used to identify the image to be identified, and the human body external bounding box of the target human body is obtained; based on the human body external bounding box and the image to be identified, the human body image to be identified corresponding to the human body external bounding box is obtained; the human body image to be identified is identified through the head detection network, and the head external bounding box is obtained. In this way, it can be ensured that the human body external bounding box and the head external bounding box are completely matched.

[0099] Matching scheme 2, the target detection network includes: a human head detection and classification network and a human key point detection network; the above step 101 can be specifically implemented through the following steps 101b to 101g.

[0100] 101b. Based on the human head detection and classification network, the image to be identified is identified to obtain the external bounding box of the head and the external bounding box of the target human body.

[0101] 101c. Extracting a target human body image to be identified that corresponds to the target human body bounding box from the image to be identified.

[0102] 101d. Based on the human key point detection network, the target human body in the target human body image to be identified is identified to obtain multiple key points.

[0103] Among them, the multiple key points include: multiple head key points.

[0104] 101e. Based on the multiple head key points, determine the target head key point bounding box corresponding to the target human bounding box.

[0105] 101f. Calculate the target intersection-union ratio between the head bounding box and the target head key point bounding box.

[0106] The intersection-union ratio is the ratio of the intersection of the area of ​​the head bounding box and the area of ​​the target head key point bounding box to the union of the area of ​​the head bounding box and the area of ​​the target head key point bounding box.

[0107] 101g. When the target intersection-over-union ratio is greater than or equal to the intersection-over-union ratio threshold, determine that the head bounding box matches the target human body bounding box to determine that the head bounding box belongs to the target human body.

[0108] The intersection-over-union ratio threshold can be determined according to actual conditions and is not limited here.

[0109] It can be understood that when the target IoU is less than the IoU threshold, the head bounding box does not match the target human bounding box.

[0110] In some embodiments of the present application, when the target intersection-and-union ratio is the largest of the intersection-and-union ratios of at least one head external bounding box obtained based on the recognition of the image to be recognized and the target head key point external bounding box, and is greater than or equal to the intersection-and-union ratio threshold, it is determined that the head external bounding box matches the target human body external bounding box to determine that the head external bounding box belongs to the target human body.

[0111] It can be understood that when the target detection network includes a human head detection classification network and a human key point detection network, the image to be identified is first identified by the human head classification detection network to obtain at least one human body external bounding box and at least one head external bounding box; then one of the at least one human body external bounding box is used as the target human body external bounding box, and the target human body image to be identified corresponding to the target human body external bounding box is extracted from the image to be identified, and the target human body in the target human body image to be identified is identified based on the human key point detection network to obtain multiple head key points; based on the multiple head key points, a target head key point external bounding box corresponding to the target human body external bounding box is determined; based on at least one head external bounding box and the target head key point external bounding box, the intersection and union ratio of at least one head external bounding box and the target head key point external bounding box is calculated; when the target intersection and union ratio is the largest of the intersection and union ratios of at least one head external bounding box obtained based on the recognition of the image to be identified and the target head key point external bounding box, and is greater than or equal to the intersection and union ratio threshold, it is determined that the head external bounding box matches the target human body external bounding box to determine that the head external bounding box belongs to the target human body.

[0112] The training process of the above-mentioned human head detection and classification network and human key point detection network specifically includes: collecting a large amount of diverse image data in the cockpit environment, for each image, marking the coordinate information of the human body external frame and the coordinate information of the head external frame, and then based on each image and the coordinate information of the human body external frame and the coordinate information of the head external frame in the image, obtaining the third sample training data, based on the third training sample data, training the basic network composed of the backbone network, the shoulder network, and the head network to train a human head detection and classification network; based on each external frame, separating at least one human image from each image, for each human image, marking the coordinate information of the human body key points in each human image, and then based on each human image and the coordinate information of the human body key points in each human image, obtaining the fourth sample training data, based on the fourth training sample data, training the basic network composed of the backbone network, the shoulder network, and the head network to train a human key point detection network.

[0113] In some embodiments of the present application, Figure 5 As shown, when identifying the human body external bounding box and the head external bounding box, the human head recognition is first performed on the image to be identified (the image of the interior environment of the cabin taken by the camera in the vehicle) through the human head detection and classification network to obtain at least one human body external bounding box and at least one head external bounding box, and one human body external bounding box in the at least one human body external bounding box is used as the target human body external bounding box, and then the target human image to be identified corresponding to the target human body external bounding box is extracted from the image to be identified, and the human body key point detection network is used to perform human body key point detection on the target human image to be identified to obtain multiple head key points, and the target human body key point external bounding box is determined based on the multiple head key points, and the intersection and union ratio of the target human body key point external bounding box and at least one head external bounding box is calculated, and the head external bounding box corresponding to the largest intersection and union ratio that is greater than or equal to the intersection and union ratio threshold is determined as the head external bounding box that matches the target human body external bounding box. Among them, before performing human head recognition on the image to be recognized through the human head detection and classification network, the image to be recognized needs to be preprocessed, and after performing human head recognition on the image to be recognized through the human head detection and classification network, the human head recognition result needs to be post-processed; similarly, before performing human key point detection on the image to be recognized through the human key point detection network, the image to be recognized needs to be preprocessed, and after performing human key point detection on the image to be recognized through the human key point detection network, the human key point recognition result needs to be post-processed; the specific preprocessing and post-processing processes can refer to the relevant technologies, which will not be elaborated here.

[0114] In some embodiments of the present application, Figure 4 As shown, after the above step 104a, the lying posture determination method provided in the embodiment of the present application may further include the following step 105.

[0115] 105. Based on the first correction coefficient, the target torso length is corrected to obtain a corrected target torso length.

[0116] Among them, the first correction coefficient is determined based on multiple first torso correction coefficients, each first torso correction coefficient is the ratio of the torso length of a human body in a sitting position to the torso length in a semi-reclining position, or each first torso correction coefficient is the ratio of the torso length of a human body in a semi-reclining position to the torso length in a sitting position.

[0117] Among them, the first correction coefficient can be the average value of multiple first torso correction coefficients, the minimum value among multiple first torso correction coefficients, the maximum value among multiple first torso correction coefficients, or the expected value after normal distribution fitting of multiple first torso correction coefficients; wherein, the target torso length is determined based on the torso key points of the target human body, and the torso key points are obtained by identifying the target human body in the image to be identified through a human body key point detection network.

[0118] In an embodiment of the present application, the target torso length is corrected based on the first correction coefficient to obtain a corrected target torso length, which is helpful for judging the attributes of a human body in a semi-reclining state based on the corrected target torso length, and can improve the accuracy of judging the attributes of the human body.

[0119] In some embodiments of the present application, Figure 4 As shown, after the above step 104b, the lying posture determination method provided in the embodiment of the present application may further include the following step 106.

[0120] 106. Based on the second correction coefficient, the target torso length is corrected to obtain a corrected target torso length.

[0121] Among them, the second correction coefficient is determined based on multiple second torso correction coefficients, each second torso correction coefficient is the ratio of the torso length of a human body in a sitting position to the torso length in a fully lying position, or each second torso correction coefficient is the ratio of the torso length of a human body in a fully lying position to the torso length in a sitting position.

[0122] Among them, the second correction coefficient can be the average value of multiple second torso correction coefficients, the minimum value among multiple second torso correction coefficients, the maximum value among multiple second torso correction coefficients, or the expected value after normal distribution fitting of multiple second torso correction coefficients; wherein, the target torso length is determined based on the torso key points of the target human body, and the torso key points are obtained by identifying the target human body in the image to be identified through a human body key point detection network.

[0123] In the embodiment of the present application, the target torso length is corrected based on the second correction coefficient to obtain a corrected target torso length, which is helpful for judging the attributes of the human body in a fully lying state based on the corrected target torso length, and can improve the accuracy of the judgment of the human body attributes.

[0124] In some embodiments of the present application, the torso key points may include a top of the head key point, a left crotch key point, a right crotch key point and a lower crotch key point; the lying posture determination method provided in the embodiments of the present application may also include the following step 107.

[0125] 107. Determine the target torso length of the human body based on the top of the head key point, the left crotch key point, the right crotch key point and the lower crotch key point.

[0126] In some embodiments of the present application, the straight-line distance between the point corresponding to the average coordinates of the valid key points (key points with confidence greater than a certain threshold) among the left crotch key point, the right crotch key point and the lower crotch key point and the top of the head key point can be determined as the target torso length.

[0127] In an embodiment of the present application, after the above-mentioned step 101, the target torso length is calculated, which is conducive to determining whether to subsequently judge whether the human body is in a fully lying posture, and / or, is conducive to determining whether the human body is in a fully lying posture, and / or, is conducive to correcting the target torso length when it is determined that the human body is in a fully lying posture, which can improve processing efficiency.

[0128] In some embodiments of the present application, the above step 102 can be specifically implemented through the following step 102b.

[0129] 102b. When the target torso length is less than or equal to the torso length threshold, the product of the height and the width of the head circumscribed frame is determined as the absolute head area of ​​the target human body.

[0130] The target torso length is determined based on the torso key points of the target human body, which are obtained by identifying the target human body in the image to be identified through a human key point detection network. The specific description of determining the target torso length can refer to the relevant description in the above step 107, which will not be repeated here.

[0131] The trunk length threshold can be determined according to actual conditions and is not limited here. For example, the trunk length threshold can be a critical value of the trunk length of an adult and a child.

[0132] In the embodiment of the present application, when the target torso length is greater than the torso length threshold, it is not necessary to determine whether the target human body is in a lying position, and it can be determined that the human body is an adult; when the target torso length is less than or equal to the torso length threshold, it is impossible to determine whether the target human body is an adult or a child, so it is necessary to determine whether the target human body is in a lying position to further assist in determining whether the target human body is an adult or a child. Therefore, by determining whether the target torso length is less than or equal to the torso length threshold, it is possible to avoid determining whether the target human body is in a lying position in all cases. For example, it is not necessary to determine whether the target human body is in a lying position when the target torso length is greater than the torso length threshold, which can save calculation amount and improve judgment efficiency.

[0133] In some embodiments of the present application, the above step 104 can be specifically implemented through the following steps 104c to 104f.

[0134] 104c. When the relative head area is less than or equal to the first head area threshold, determine the length of the lower limbs of the target person based on the key points of the lower limbs of the target person.

[0135] 104d. Based on the torso key points of the target human body, determine the target torso length of the target human body.

[0136] Among them, the lower limb key points and torso key points are obtained by identifying the target human body in the image to be identified through the human key point detection network.

[0137] In some embodiments of the present application, the lower limb key points include: a left crotch key point, a right crotch key point, a lower crotch key point, a left knee key point, and a right knee key point. The length of the lower limb of the target human body can be determined based on the left crotch key point, the right crotch key point, the lower crotch key point, the left knee key point, and the right knee key point.

[0138] In some embodiments of the present application, the distance between the position corresponding to the average coordinates of the coordinates of the valid key points (key points with confidence greater than a certain threshold) among the left crotch key point, the right crotch key point and the lower crotch key point, and the position corresponding to the average coordinates of the coordinates of the valid key points among the left knee key point and the right knee key point can be determined as the length of the lower limb.

[0139] For the specific description of determining the target torso length, reference may be made to the relevant description in the above step 107, which will not be repeated here.

[0140] 104e. The ratio of the lower limb length to the target trunk length is determined as the target ratio.

[0141] 104f. When the target ratio is greater than or equal to the ratio threshold, it is determined that the target human body is in a lying position.

[0142] The ratio threshold can be determined according to actual conditions and is not limited here.

[0143] When the target ratio is less than the ratio threshold, it is determined that the target human body is not in a lying position.

[0144] It can be understood that when a human body is in a sitting position (the lower limbs cannot be stretched out), the ratio of the lower limb length to the target torso length is smaller than the ratio of the lower limb length to the target torso length when the human body is in a fully lying position (the lower limbs can be stretched out). Therefore, when the target ratio is greater than or equal to the ratio threshold, it is determined that the target human body is in a fully lying position. In this way, it can be more accurately determined that the current human body is in a fully lying position.

[0145] Exemplarily, the normal domain of the head area is denoted by M.

[0146] Calculate the standard head area Ss: for the target human body, within a time sequence, record the absolute head areas of all heads of the target human body that fall into M as S = [S1, S2, ..., Sn], take the average value of S to obtain the standard head area of ​​the target human body Ss = Sum(S) / n (Sum is the summation function, n is the number of absolute head areas that meet the requirements).

[0147] For the target human body in the image to be identified, identify the target human head external frame of the target human body, record the confidence of the target human head external frame as C, the width of the target human head external frame as W, and the height as H. The result obtained by the human key point detection network is simply recorded as n; the human key point-torso length mapping function is recorded as F; the first torso correction coefficient corresponding to the semi-lying posture is recorded as a, and the second torso correction coefficient corresponding to the full-lying posture is recorded as b, b>a; the head confidence threshold is recorded as Th, the first head area threshold is recorded as R1, and the second head area threshold is recorded as R2, R1>R2.

[0148] Determine whether the target head frame is credible:

[0149] If C≤Th, the target head bounding box is unreliable and the process terminates.

[0150] If C>Th, the target head external frame is credible and proceed to the next step of judgment.

[0151] Calculate the absolute head area Sa=W*H, call the standard head area Ss at this time, and calculate the relative head area R=Sa / Ss.

[0152] Determine whether the target person is in a lying position:

[0153] If R>R1, the human body is not in a lying position and the process terminates.

[0154] If R≤R1 and R≥R2, the target human body is in a semi-reclining posture and enters the torso length correction processing flow in the semi-reclining posture.

[0155] If R < R2, the target human body is in a full reclining posture, and enters the trunk length correction processing flow in the full reclining posture.

[0156] Calculate the target trunk length h = F(n).

[0157] The target human body is in a semi-reclining posture, and the trunk length is corrected by the formula h1 = h * a;

[0158] The target human body is in a full reclining posture, and the trunk length is corrected by the formula h2 = h * b.

[0159] The human body is in a full reclining posture, and enters the trunk length correction processing flow in the full reclining posture.

[0160] In the embodiments of the present application, in the internal environment of the cockpit, it is very important to distinguish between the sitting posture and the reclining posture, which will directly affect the accuracy rate of human body attribute judgment. However, there was a lack of such a method for distinction in the past. The present application proposes a reclining posture determination scheme based on head region information (relative head area) inside the cockpit. Starting from a large amount of data, it is found that when the human body is in a reclining posture, due to different angles and distances facing the camera, the absolute head area of its human head is much smaller than other postures. Therefore, this characteristic can be used to distinguish the reclining posture from other postures. Further, in the embodiments of the present application, the credibility of the circumscribed frame of the human head is judged to ensure the reliability of its absolute head area, and the concept of relative head area is introduced. By calculating the ratio of the absolute head area to the standard head area, the scaling ratio of the human head is judged, so as to determine whether the current posture is a reclining posture. This is more applicable and accurate than directly using the absolute head area for judgment. Moreover, the first head area threshold and the second head area threshold are introduced to subdivide the reclining posture into a semi-reclining posture and a full reclining posture, which improves the correctness of subsequent human body attribute judgment.

[0161] It should be noted that when the standard head area is not determined or not stable, only the absolute head area can still be used for judgment.

[0162] In the embodiments of the present application, by accurately identifying whether the human body in the cockpit environment is in a reclining posture, the human body in the reclining posture is separated from the image, greatly reducing the misdetection of identifying an adult as a child in the reclining posture, and significantly improving the accuracy rate of the human body attribute judgment result inside the cockpit.

[0163] The present application also provides a reclining posture determination device, Figure 6 is a structural schematic diagram of a reclining posture determination device provided by the present application, as Figure 6As shown, the lying posture determination device includes: an identification module 601, which is used to identify the target human body in the image to be identified based on the target detection network, and obtain the target human body's head external frame, and the image to be identified is an image of the interior environment of the cabin taken by a camera in the vehicle; a determination module 602, which is used to determine the product of the height and width of the human head external frame as the absolute head area of ​​the target human body; determine the ratio of the absolute head area to the standard head area as the relative head area, and the standard head area is used to indicate the standard value of the absolute head area of ​​a human body in a sitting position; when the relative head area is less than or equal to the first head area threshold, it is determined that the target human body is in a lying position.

[0164] In some embodiments of the present application, the recognition module 601 is specifically used to identify the target human body in the image to be identified based on the target detection network, and obtain the head external bounding box and the confidence of the head external bounding box; the determination module 602 is specifically used to determine the product of the height and width of the head external bounding box as the absolute head area of ​​the target human body when the confidence of the head external bounding box is greater than or equal to the head confidence threshold.

[0165] In some embodiments of the present application, the standard head area is determined based on multiple reference absolute head areas, and the standard head area is greater than or equal to the minimum value of the multiple reference absolute head areas, and less than or equal to the maximum value of the multiple reference absolute head areas; wherein each reference absolute head area is an absolute head area within the normal head area domain; or, each reference absolute head area is an absolute head area of ​​a human body recorded in a vehicle and within the normal head area domain; or, each reference absolute head area is an absolute head area of ​​the target human body recorded in the vehicle and within the normal head area domain.

[0166] In some embodiments of the present application, the standard head area is an average value of multiple reference absolute head areas; or, the standard head area is an expected value obtained by performing normal distribution fitting on multiple reference absolute head areas.

[0167] In some embodiments of the present application, each reference absolute head area is an absolute head area of ​​a target human body recorded in a vehicle and is within a normal head area domain. The determination module 602 is also used to determine that the target human body is in a sitting position when the relative head area is greater than a first head area threshold; when the absolute head area of ​​the target human body is within the normal head area domain, the absolute head area of ​​the target human body is determined as a reference absolute head area, so as to determine a standard head area based on multiple reference absolute head areas and the absolute head area of ​​the target human body.

[0168] In some embodiments of the present application, the determination module 602 is specifically used to determine that the target human body is in a semi-lying state when the relative head area is greater than or equal to a second head area threshold and less than or equal to a first head area threshold; and to determine that the target human body is in a fully lying posture when the relative head area is less than the second head area threshold; wherein the second head area threshold is less than the first head area threshold.

[0169] In some embodiments of the present application, the device also includes: a correction module, which is used to, when the relative head area is greater than or equal to the second head area threshold value and less than or equal to the first head area threshold value, after determining that the target human body is in a semi-reclining state, correct the target torso length based on the first correction coefficient to obtain a corrected target torso length, wherein the first correction coefficient is determined based on multiple first torso correction coefficients, each first torso correction coefficient is the ratio of the torso length of a human body in a sitting position to the torso length in a semi-reclining position, or each first torso correction coefficient is the ratio of the torso length of a human body in a semi-reclining position to the torso length in a sitting position; or, the correction module is used to When the product is less than the second head area threshold, after determining that the target human body is in a fully lying posture, the target torso length is corrected based on the second correction coefficient to obtain a corrected target torso length, wherein the second correction coefficient is determined based on multiple second torso correction coefficients, each second torso correction coefficient is the ratio of the torso length of a human body in a sitting posture to the torso length in a fully lying posture, or each second torso correction coefficient is the ratio of the torso length of a human body in a fully lying posture to the torso length in a sitting posture; wherein the target torso length is determined based on the torso key points of the target human body, and the torso key points are obtained by identifying the target human body in the image to be identified through a human key point detection network.

[0170] In some embodiments of the present application, the multiple key points also include: a left knee key point and a right knee key point; a determination module 602 is specifically used to determine the length of the target body's lower limbs based on the target body's lower limb key points when the relative head area is less than or equal to a first head area threshold; determine the target torso length of the target body based on the target body's torso key points; determine the ratio of the lower limb length to the target torso length as the target ratio; when the target ratio is greater than or equal to the ratio threshold, determine that the target body is in a lying position; wherein the lower limb key points and the torso key points are obtained by identifying the target body in the image to be identified through a human key point detection network.

[0171] It should be noted that the above-mentioned lying posture determination device can be the electronic device in the above-mentioned method embodiment of the present application, or it can be a functional module and / or functional entity in the electronic device that can realize the functions of the device embodiment, and the embodiment of the present application is not limited.

[0172] In the embodiments of the present application, each module can implement the lying posture determination method provided in the above method embodiments and can achieve the same technical effect. To avoid repetition, it will not be described here.

[0173] Figure 7 The structural diagram of an electronic device provided in the embodiment of the present application is used to exemplarily illustrate the electronic device that implements the method for determining an arbitrary lying posture in the embodiment of the present application, and should not be understood as a specific limitation on the embodiment of the present application.

[0174] like Figure 7 As shown, the electronic device 700 may include a processor (e.g., a central processing unit, a graphics processing unit, etc.) 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage device 708 into a random access memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the electronic device 700 are also stored. The processor 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0175] Typically, the following devices may be connected to the I / O interface 705: input devices 706 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 707 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 708 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 709. The communication device 709 may allow the electronic device 700 to communicate with other devices wirelessly or by wire to exchange data. Although the electronic device 700 having various devices is shown, it should be understood that it is not required to implement or have all the devices shown. More or fewer devices may be implemented or have alternatively.

[0176] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 709, or installed from the storage device 708, or installed from the ROM 702. When the computer program is executed by the processor 701, the functions defined in any lying posture determination method provided in the embodiment of the present application can be executed.

[0177] It should be noted that the computer-readable medium mentioned above in the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0178] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0179] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0180] The above-mentioned computer-readable medium carries one or more programs. When the above-mentioned one or more programs are executed by the electronic device, the electronic device: identifies the target human body in the image to be identified based on the target detection network, and obtains the external frame of the head of the target human body, where the image to be identified is an image of the interior environment of the cabin taken by a camera in the vehicle; determines the product of the height and width of the external frame of the head as the absolute head area of ​​the target human body; determines the ratio of the absolute head area to the standard head area as the relative head area, where the standard head area is used to indicate the standard value of the absolute head area of ​​a human body in a sitting position; and determines that the target human body is in a lying position when the relative head area is less than or equal to the first head area threshold.

[0181] In an embodiment of the present application, a computer program code for performing the operation of the present application can be written in one or more programming languages ​​or a combination thereof, and the above-mentioned programming languages ​​include but are not limited to object-oriented programming languages, such as Java, Smalltalk, C++, and also include conventional procedural programming languages, such as "C" language or similar programming languages. The program code can be executed completely on the computer, partially on the computer, as an independent software package, partially on the computer and partially on the remote computer, or completely on the remote computer or server. In the case of a remote computer, the remote computer can be connected to the computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, using an Internet service provider to connect through the Internet).

[0182] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0183] The units involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of a unit does not, in some cases, constitute a limitation on the unit itself.

[0184] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0185] In the context of the present application, a computer-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. A computer-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. A more specific example of a computer-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0186] An embodiment of the present application further provides a vehicle, comprising: the above-mentioned lying posture determination device, or the above-mentioned electronic device, or the above-mentioned computer-readable storage medium.

[0187] The above description is only a preferred embodiment of the present application and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present application is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in this application (but not limited to) by each other to form a technical solution.

[0188] In addition, although each operation is described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or to be performed in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be construed as limiting the scope of the application. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0189] Although the subject matter has been described in language specific to structural features and / or methodological logical actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims.

Claims

1. A method for determining a lying posture, characterized in that: The method comprises: Based on the target detection network, a target human body in the image to be identified is identified to obtain a head external frame of the target human body, wherein the image to be identified is an image of the interior environment of the cabin taken by a camera in the vehicle; The product of the height and the width of the human head circumference frame is determined as the absolute head area of ​​the target human body; Determine the ratio of the absolute head area to the standard head area as the relative head area, wherein the standard head area is used to indicate the standard value of the absolute head area of ​​a person in a sitting position; When the relative head area is less than or equal to the first head area threshold, it is determined that the target human body is in a lying position.

2. The method according to claim 1, characterized in that The method of identifying a target human body in the image to be identified based on the target detection network and obtaining a head external frame of the target human body includes: Based on the target detection network, the target human body in the image to be identified is identified to obtain the human head external bounding box and the confidence of the human head external bounding box; The step of determining the absolute head area of ​​the target human body by multiplying the height and the width of the human head circumference frame includes: When the confidence of the head circumference frame is greater than or equal to the head confidence threshold, the product of the height and width of the head circumference frame is determined as the absolute head area of ​​the target person.

3. The method according to claim 1, characterized in that The standard head area is determined based on a plurality of reference absolute head areas, the standard head area being greater than or equal to a minimum value among the plurality of reference absolute head areas, and less than or equal to a maximum value among the plurality of reference absolute head areas; Wherein, each reference absolute head area is an absolute head area within the normal head area domain; or, Each reference absolute head area is an absolute head area of ​​a human body in the vehicle that is recorded and is within the normal head area range; or, Each reference absolute head area is an absolute head area recorded for the target human body in the vehicle and is within a normal head area range.

4. The method according to claim 3, characterized in that The standard head area is an average value of the multiple reference absolute head areas; or, The standard head area is an expected value obtained by performing normal distribution fitting on the multiple reference absolute head areas.

5. The method according to claim 3, characterized in that: Each reference absolute head area is an absolute head area of ​​the target human body recorded in the vehicle and within a normal head area domain, and the method further includes: When the relative head area is greater than the first head area threshold, determining that the target human body is in a sitting position; The absolute head area of ​​the target human body is determined as a reference absolute head area, so as to determine the standard head area based on the multiple reference absolute head areas and the absolute head area of ​​the target human body.

6. The method according to any one of claims 1 to 5, characterized in that When the relative head area is less than or equal to a first head area threshold, determining that the target human body is in a lying position includes: When the relative head area is greater than or equal to a second head area threshold and less than or equal to the first head area threshold, determining that the target human body is in a semi-lying state; When the relative head area is smaller than the second head area threshold, determining that the target human body is in a fully lying posture; Among them, the second head area threshold is smaller than the first head area threshold.

7. The method according to claim 6, characterized in that After determining that the target human body is in a semi-lying state when the relative head area is greater than or equal to the second head area threshold and less than or equal to the first head area threshold, the method further includes: Based on the first correction coefficient, the target torso length is corrected to obtain the corrected target torso length, wherein the first correction coefficient is determined based on a plurality of first torso correction coefficients, each of which is a ratio of a torso length of a human body in a sitting position to a torso length in a semi-reclining position, or each of which is a ratio of a torso length of a human body in a semi-reclining position to a torso length in a sitting position; or, In the case where the relative head area is less than or equal to the second head area threshold, after determining that the target human body is in a fully lying posture, the method further includes: Based on the second correction coefficient, the target torso length is corrected to obtain the corrected target torso length, wherein the second correction coefficient is determined based on a plurality of second torso correction coefficients, each of which is a ratio of a torso length of a human body in a sitting position to a torso length in a fully lying position, or each of which is a ratio of a torso length of a human body in a fully lying position to a torso length in a sitting position; Among them, the target torso length is determined based on the torso key points of the target human body, and the torso key points are obtained by identifying the target human body in the image to be identified through a human key point detection network.

8. The method according to any one of claims 1 to 5, characterized in that When the relative head area is less than or equal to a first head area threshold, determining that the target human body is in a lying position includes: When the relative head area is less than or equal to a first head area threshold, determining the length of the lower limbs of the target human body based on the key points of the lower limbs of the target human body; Determining a target torso length of the target human body based on key points of the torso of the target human body; Determine the ratio of the lower limb length to the target trunk length as a target ratio; When the target ratio is greater than or equal to a ratio threshold, determining that the target human body is in a lying position; Among them, the lower limb key points and the torso key points are obtained by identifying the target human body in the image to be identified through a human key point detection network.

9. A lying posture determination device, characterized in that: include: A recognition module, used to recognize a target person in an image to be recognized based on a target detection network, and obtain a head external frame of the target person, wherein the image to be recognized is an image of the interior environment of the cabin taken by a camera in the vehicle; A determination module is used to determine the product of the height and the width of the human head external frame as the absolute head area of ​​the target human body; determine the ratio of the absolute head area to the standard head area as the relative head area, and the standard head area is used to indicate the standard value of the absolute head area of ​​a human body in a sitting position; when the relative head area is less than or equal to a first human head area threshold, determine that the target human body is in a lying position.

10. An electronic device, characterized in that: include: A processor, wherein the processor is used to execute a computer program stored in a memory, wherein the computer program, when executed by the processor, implements the steps of the lying posture determination method according to any one of claims 1 to 8.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the lying posture determination method according to any one of claims 1 to 8 are implemented.

12. A vehicle, characterized in that: include: The lying posture determination device as claimed in claim 9, or the electronic device as claimed in claim 10, or includes the computer-readable storage medium as claimed in claim 11.

Images