Indoor positioning method under occlusion condition

Abstract

The application discloses an indoor positioning method under a shielding condition, and comprises the following steps: step one: obtaining the circumscribed rectangular coordinates of a human body through a human body detection model from the image collected by a high-definition monitoring camera, and obtaining a human body image frame by cutting the human body image in the camera image according to the circumscribed rectangular coordinates of the human body; the detection model of the human body detection model is a YOLO detection model; the situation that the feet and legs of a pedestrian are shielded is common in an actual camera scene, and the application effectively solves the problem that the existing visual method using a head-to-foot point pair cannot be positioned correctly when the foot point is shielded; compared with the visual method of fixing and approximating the height (such as 165 cm), when a person is in a non-erect posture such as a bent-over posture, a sitting posture or a kneeling posture, the error of the fixed and approximate height method will be larger, and the application can maintain the positioning accuracy of the human body in the diversified postures by effectively estimating the actual height of the lower edge of the human body frame.

Description

Technical Field

[0001] This invention relates to the field of indoor positioning technology, specifically to an indoor positioning method under obstruction conditions. Background Technology

[0002] In the digitalization of offline commercial spaces, a crucial and fundamental issue is how to accurately locate the real-time position of customers (pedestrians) within the indoor environment. Existing indoor positioning methods mainly include the following:

[0003] Based on mobile phone Wi-Fi positioning, the distance between a pedestrian holding a mobile device and each Wi-Fi access device is estimated by the signal strength between the mobile phone and multiple Wi-Fi access devices. Since the installation location of each Wi-Fi access device has been measured, the pedestrian's location can be determined by methods such as triangulation.

[0004] Ultra-wideband (UWB) positioning and Wi-Fi positioning are greatly affected by indoor furnishings. UWB positioning technology reduces the impact of environmental furnishings on ranging accuracy by sending extremely narrow pulses, but requires the person being located to hold a UWB tag device.

[0005] Binocular vision positioning uses images from two cameras with known optical center distances to match visual features. Based on the parallax information between the images and the pre-calibrated camera parameters, the position information of the target person at the distance from the camera is calculated.

[0006] Monocular vision positioning requires a monocular camera to obtain depth information of an object. Given that the camera's internal parameters and external parameters such as its installation position and orientation are known, other information, such as paired head-to-toe points, is needed to obtain an estimate of the object's depth in order to calculate the position of the target person relative to the camera.

[0007] In existing technologies, the accuracy of WIFI positioning is generally between 5 and 10 meters, and it is quite sensitive to the influence of the environment and furnishings. The positioning accuracy fluctuates greatly. In fact, it is difficult to directly determine the positional relationship between indoor pedestrians (customers) and commercial facilities through WIFI positioning.

[0008] While UWB positioning technology can achieve positioning accuracy within 1 meter (and in the decimeter range under ideal unobstructed conditions), it is difficult to apply to pedestrian (mainly customer) positioning in offline store scenarios because it requires the person being located to hold a UWB tag.

[0009] Binocular vision positioning is less affected by the environment, but the parallax generated by binoculars decreases as the target object is further away from the camera system. When the distance between the person to be positioned and the camera system is more than 5 meters, the optical center distance of the binocular camera needs to be increased accordingly in order to ensure effective parallax, resulting in a large or difficult-to-calibrate camera that cannot be installed in commercial scenarios. Due to the lack of depth information for objects, monocular vision localization commonly employs two methods to calculate the distance between a person and the camera: The first method utilizes a pair of "head vertex" and "foot points" for the person. By aligning the vertical projection of the head vertex onto the ground with the foot points and solving similar triangles, the distance between the person and the camera can be obtained. However, in real-world scenarios, the foot points of pedestrians are frequently obscured indoors, rendering this method ineffective. The second method leverages the fact that the head vertex is less likely to be obscured compared to the foot points. It uses an approximate height value to address the lack of depth information in monocular cameras and calculates the person's distance. Since it does not require a foot point paired with the head vertex, this method still works even when the feet are obscured. Its main drawback is that the approximate height estimate has an error compared to the pedestrian's actual height. A larger problem arises when the actual height of the pedestrian differs significantly from the approximate height, such as when the pedestrian is not standing, sitting, or kneeling, leading to a larger error in position estimation. Summary of the Invention

[0010] The purpose of this invention is to provide an indoor positioning method under obstruction conditions, which has the advantages of accurate positioning and high precision, and solves the problems raised in the background art.

[0011] To achieve the above objectives, the present invention provides the following technical solution: an indoor positioning method under obstruction conditions, comprising the following steps:

[0012] Step 1: The image captured by the high-definition surveillance camera is processed by the human detection model to obtain the bounding rectangle coordinates of the human body. Based on the bounding rectangle coordinates of the human body, the human body image is extracted from the camera image to obtain the human image frame. The detection model of the human detection model is the YOLO detection model.

[0013] Step 2: Input the human figure image frame into the human skeleton point model to obtain the coordinates of the head, shoulders, hips, knees and feet of the human figure.

[0014] Step 3: Feed the human figure image box into the human figure lower edge estimation model. Based on the image content of the human figure image box, the model estimates which human body part is the lower edge of the human figure box, that is, estimates the probability that the lower edge of the human figure box is the shoulder, hip, knee, or foot, and outputs the two human body parts P1 and P2 with the highest probability of the lower edge of the human figure box.

[0015] Step 4: Take the vertical coordinate value of the estimated bone point corresponding to part P1 as Y1, and the vertical coordinate value of the estimated bone point corresponding to part P2 as Y2. When the difference between Y1 and Y2 is less than 15 pixels, take the lower part of P1 and P2 as the lower edge of the human shape box. Otherwise, take the part with the highest probability of the lower edge of the human shape as the lower edge of the human shape box, and denote it as Pmin.

[0016] Step 5: Assign different height values ​​to the bottom edge of the human figure frame according to different parts of Pmin. Specifically, when Pmin = foot, the bottom edge height of the human figure frame is 10cm; when Pmin = knee, the bottom edge height of the human figure frame is 50cm; when Pmin = hip, the bottom edge height of the human figure frame is 90cm; and when Pmin = shoulder, the bottom edge height of the human figure frame is 140cm.

[0017] Step Six: Calculate the distance between the pedestrian and the camera based on the height of the lower edge of the human figure image frame. First, based on the known camera intrinsic and extrinsic parameters (i.e., height is 0), we can calculate the intersection point K of the ray passing through the midpoint E of the lower edge of the human figure image frame and the camera's optical center C with the ground. Since the camera's installation height AC is known, and the height BE of the lower edge of the human figure image frame was obtained in Step Five, we have AK / AC = BK / BE. Therefore, BK = AK * BE / AC. Thus, the distance AB between the human figure and the camera is AK – BK, thereby completing the spatial positioning of the occluded human figure frame.

[0018] In this invention, the training process of the human figure lower edge estimation model is as follows:

[0019] Step 1: Collect a large number of human image frames with different poses and occlusion conditions. The different occlusion conditions mainly include: visible from head to toe, feet are occluded, lower legs are occluded, below the knees are occluded, below the thighs are occluded, below the hips are occluded, and below the shoulders are occluded. In Step 1, different poses include, but are not limited to, standing, sitting, bending over, and squatting.

[0020] Step 2: Manually label the lower edge of each human figure image frame, dividing it into four categories: shoulder, hip, knee, and foot. That is, when the lower edge of the human figure image frame is the shoulder, the manual labeling category is "shoulder"; when the lower edge of the human figure image frame is the hip, the manual labeling category is "hip", and so on.

[0021] Step 3: Using the above supervised training data, train a classification neural network. The classification neural network uses ResNet-50. For images labeled with the bottom edge of the human-shaped image box as the "hip" part, supervised training is performed by using the output value of the hip category label = 1 and the output value of the other three categories = 0.

[0022] Step 4: During the model's inference run, for the input human-shaped image box, the model will provide a probability estimate of whether its lower edge is a "shoulder", "hip", "knee" or "foot".

[0023] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0024] 1. In real-world camera scenarios, it is common for pedestrians' feet and legs to be obscured. This invention effectively solves the problem that existing visual methods using head-to-foot point pairs cannot accurately locate the feet when the foot point is obscured.

[0025] 2. Compared to visual methods that use a fixed approximation of height (e.g., 165cm), the error of fixed approximation height methods will be larger when a person is in a non-upright posture such as bending over, sitting, or kneeling. However, this invention can maintain the accuracy of human positioning under the above-mentioned diverse postures by effectively estimating the actual height of the lower edge of the human frame. 3. This invention does not require the person being located to wear a positioning tag or other electronic devices, and can complete the positioning without the person's awareness. The positioning only uses the coordinates of the human detection frame and does not involve any privacy data. Attached Figure Description

[0026] Figure 1 This is a schematic diagram illustrating the humanoid position calculation of the present invention;

[0027] In the diagram: point A is the vertical projection of the camera's optical center onto the ground, point C is the location of the camera's optical center, point D is the head position at the top edge of the human figure frame, point E is the bottom edge of the human figure frame, and EB is the occluded and invisible part of the human figure. Detailed Implementation

[0028] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0029] Please see Figure 1 -4. This invention provides a technical solution: an indoor positioning method under obstruction conditions, comprising the following steps:

[0030] Step 1: The image captured by the high-definition surveillance camera is processed by the human detection model to obtain the bounding rectangle coordinates of the human body. Based on the bounding rectangle coordinates of the human body, the human body image is extracted from the camera image to obtain the human image frame. The detection model of the human detection model is the YOLO detection model.

[0031] Step 2: Input the human figure image frame into the human skeleton point model to obtain the coordinates of the head, shoulders, hips, knees and feet of the human figure.

[0032] Step 3: Feed the human figure image box into the human figure lower edge estimation model. Based on the image content of the human figure image box, the model estimates which human body part is the lower edge of the human figure box, that is, estimates the probability that the lower edge of the human figure box is the shoulder, hip, knee, or foot, and outputs the two human body parts P1 and P2 with the highest probability of the lower edge of the human figure box.

[0033] Step 4: Take the vertical coordinate value of the estimated bone point corresponding to part P1 as Y1, and the vertical coordinate value of the estimated bone point corresponding to part P2 as Y2. When the difference between Y1 and Y2 is less than 15 pixels, take the lower part of P1 and P2 as the lower edge of the human shape box. Otherwise, take the part with the highest probability of the lower edge of the human shape as the lower edge of the human shape box, and denote it as Pmin.

[0034] Step 5: Assign different height values ​​to the bottom edge of the human figure frame according to different parts of Pmin. Specifically, when Pmin = foot, the bottom edge height of the human figure frame is 10cm; when Pmin = knee, the bottom edge height of the human figure frame is 50cm; when Pmin = hip, the bottom edge height of the human figure frame is 90cm; and when Pmin = shoulder, the bottom edge height of the human figure frame is 140cm.

[0035] Step Six: Calculate the distance between the pedestrian and the camera based on the height of the lower edge of the human figure image frame. First, based on the known camera intrinsic and extrinsic parameters (i.e., height is 0), we can calculate the intersection point K of the ray passing through the midpoint E of the lower edge of the human figure image frame and the camera's optical center C with the ground. Since the camera's installation height AC is known, and the height BE of the lower edge of the human figure image frame was obtained in Step Five, we have AK / AC = BK / BE. Therefore, BK = AK * BE / AC. Thus, the distance AB between the human figure and the camera is AK – BK, thereby completing the spatial positioning of the occluded human figure frame.

[0036] In this invention, the training process of the human figure lower edge estimation model is as follows:

[0037] Step 1: Collect a large number of human image frames with different poses and occlusion conditions. The different occlusion conditions mainly include: visible from head to toe, feet are occluded, lower legs are occluded, below the knees are occluded, below the thighs are occluded, below the hips are occluded, and below the shoulders are occluded. In Step 1, different poses include, but are not limited to, standing, sitting, bending over, and squatting.

[0038] Step 2: Manually label the lower edge of each human figure image frame, dividing it into four categories: shoulder, hip, knee, and foot. That is, when the lower edge of the human figure image frame is the shoulder, the manual labeling category is "shoulder"; when the lower edge of the human figure image frame is the hip, the manual labeling category is "hip", and so on.

[0039] Step 3: Using the above supervised training data, train a classification neural network. The classification neural network uses ResNet-50. For images labeled with the bottom edge of the human-shaped image box as the "hip" part, supervised training is performed by using the output value of the hip category label = 1 and the output value of the other three categories = 0.

[0040] Step 4: During the model's inference run, for the input human-shaped image box, the model will provide a probability estimate of whether its lower edge is a "shoulder", "hip", "knee" or "foot".

[0041] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0042] All electrical components mentioned in this article are electrically connected to an external main controller and 220V AC mains power, and the main controller can be a conventional known device such as a computer for control. Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. An indoor positioning method under obstruction conditions, characterized in that: Includes the following steps: Step 1: The image captured by the high-definition surveillance camera is processed by the human detection model to obtain the bounding rectangle coordinates of the human body. Based on the bounding rectangle coordinates of the human body, the human body image is extracted from the camera image to obtain the human image frame. Step 2: Input the human figure image frame into the human skeleton point model to obtain the coordinates of the head, shoulders, hips, knees, and feet of the human figure; Step 3: Feed the human figure image box into the human figure lower edge estimation model. Based on the image content of the human figure image box, the model estimates which human body part is the lower edge of the human figure box, that is, estimates the probability that the lower edge of the human figure box is the shoulder, hip, knee, or foot, and outputs the two human body parts P1 and P2 with the highest probability of the lower edge of the human figure box. Step 4: Take the vertical coordinate value of the estimated bone point corresponding to part P1 as Y1, and the vertical coordinate value of the estimated bone point corresponding to part P2 as Y2. When the difference between Y1 and Y2 is less than 15 pixels, take the lower part of P1 and P2 as the lower edge of the human shape box. Otherwise, take the part with the highest probability of the lower edge of the human shape as the lower edge of the human shape box, and denote it as Pmin. Step 5: Assign different height values ​​to the bottom edge of the human figure frame according to different parts of Pmin. Specifically, when Pmin = foot, the bottom edge height of the human figure frame is 10cm; when Pmin = knee, the bottom edge height of the human figure frame is 50cm; when Pmin = hip, the bottom edge height of the human figure frame is 90cm; and when Pmin = shoulder, the bottom edge height of the human figure frame is 140cm. Step Six: Calculate the distance between the pedestrian and the camera based on the height of the lower edge of the human figure image frame. First, based on the known camera intrinsic and extrinsic parameters (i.e., height is 0), we can calculate the intersection point K of the ray passing through the midpoint E of the lower edge of the human figure image frame and the camera's optical center C with the ground. Since the camera's installation height AC is known, and the height BE of the lower edge of the human figure image frame was obtained in Step Five, we have AK / AC = BK / BE. Therefore, BK = AK * BE / AC. Thus, the distance AB between the human figure and the camera is AK – BK, thereby completing the spatial positioning of the occluded human figure frame.

2. The indoor positioning method under obstruction conditions according to claim 1, characterized in that: The training process of the human figure lower edge estimation model is as follows: Step 1: Collect a large number of human image frames with different poses and different occlusion situations. The different occlusion situations mainly include: visible from head to toe, feet are occluded, lower legs are occluded, below the knees are occluded, below the thighs are occluded, below the hips are occluded, and below the shoulders are occluded. Step 2: Manually label the lower edge of each human figure image frame, dividing it into four categories: shoulder, hip, knee, and foot. That is, when the lower edge of the human figure image frame is the shoulder, the manual labeling category is "shoulder"; when the lower edge of the human figure image frame is the hip, the manual labeling category is "hip", and so on. Step 3: Using the supervised training data described above, train a classification neural network; Step 4: During the model's inference run, for the input human-shaped image box, the model will provide probability estimates of whether its lower edge is a "shoulder", "hip", "knee", or "foot".

3. The indoor positioning method under obstruction conditions according to claim 2, characterized in that: In step one: different postures include, but are not limited to, standing, sitting, bending over and squatting.

4. The indoor positioning method under obstruction conditions according to claim 2, characterized in that: In step three: the classification neural network uses ResNet-50. For images labeled with the lower edge of the human-shaped image box as the "hip" part, during training, the output value of the hip category label is 1, and the output value of the other three categories is 0 for supervised training.

5. The indoor positioning method under obstruction conditions according to claim 1, characterized in that: In step one: the detection model for the human figure detection model is the YOLO detection model.

Images