A method, device, electronic terminal, and storage medium for locating people indoors.

Abstract

This invention discloses a method, device, electronic terminal, and storage medium for locating people indoors. The method includes: real-time acquisition of images of at least two perspectives of a target area; extraction of head and shoulder feature points from the images; matching the head and shoulder feature points in the simultaneously acquired images of at least two perspectives; determining the location tracking information of the successfully matched head and shoulder feature points; responding to a target device within the target area receiving a preset operation; determining the personnel identity information, operation time, and device identifier corresponding to the preset operation; determining the device location based on the device identifier; determining a first target feature point from the successfully matched head and shoulder feature points based on the device location, operation time, and location tracking information of each successfully matched head and shoulder feature point; and associating the location tracking information of the first target feature point with the personnel identity information to obtain the location tracking information of each person. This method enables compliant, terminal-free indoor personnel location tracking.

Description

Technical Field

[0001] The embodiments of the present invention relate to computer vision technology, and more particularly to a method, device, electronic terminal and storage medium for locating people indoors. Background Technology

[0002] In existing technologies, indoor personnel positioning methods can be divided into two categories: those based on terminal devices and those without terminal devices.

[0003] Positioning methods based on terminal devices utilize positioning terminals such as mobile phones and employ positioning technologies including cellular base stations, Bluetooth, Ultra Wide Band (UWB), and Wi-Fi. They leverage positioning principles based on power measurement (triangulation, fingerprinting, etc.), signal arrival time (time of arrival, time difference of arrival, etc.), and signal angle (angle of arrival, angle of departure) to achieve real-time location tracking of the positioning terminal associated with the person. This type of solution requires personnel to carry the positioning terminal associated with the tracking, and the platform must associate the person with the positioning terminal, cooperating with the positioning and tracking base station to achieve location tracking; this is a type of active positioning technology.

[0004] Positioning methods based on no-terminal devices are used to locate people through methods such as camera video recognition. People do not need to carry positioning terminals, which belongs to passive positioning technology.

[0005] In some scenarios (such as public places with high pedestrian traffic), location methods based on terminal devices are not suitable because it is difficult to ensure that everyone carries a location device. Even if devices are distributed on-site, it is still necessary to associate and bind the location devices with individuals, which is very inconvenient for management. However, if a location method without terminal devices is used in these scenarios, since most current methods rely on facial recognition to compare against facial databases to pinpoint individuals, this involves sensitive personal information and raises compliance issues. Summary of the Invention

[0006] In view of this, embodiments of the present invention provide a method, device, electronic terminal, and storage medium for locating indoor personnel, which can achieve compliant, terminal-free indoor personnel location.

[0007] In a first aspect, embodiments of the present invention provide a method for locating people indoors, comprising:

[0008] Real-time acquisition of images of the target area from at least two perspectives;

[0009] Extract head and shoulder feature points from the image, match the head and shoulder feature points in images acquired simultaneously from at least two viewpoints, and determine the localization and tracking information of the successfully matched head and shoulder feature points;

[0010] In response to a target device within the target area receiving a preset operation, determine the personnel identity information, operation time, and device identifier corresponding to the preset operation;

[0011] The device location is determined based on the device identifier. Based on the device location, operation time, and positioning and tracking information of each successfully matched head and shoulder feature point, the first target feature point is determined from each successfully matched head and shoulder feature point.

[0012] The location tracking information of the first target feature point is associated with the personnel identity information to obtain the location tracking information of each person.

[0013] Secondly, embodiments of the present invention also provide an indoor personnel positioning device, comprising:

[0014] The image acquisition module is used to acquire images of the target area from at least two perspectives in real time.

[0015] The location recognition module is used to extract head and shoulder feature points in the image, match the head and shoulder feature points in images acquired simultaneously from at least two viewpoints, and determine the location and tracking information of the successfully matched head and shoulder feature points.

[0016] The operation response module is used to respond to the target device within the target area receiving a preset operation and to determine the personnel identity information, operation time and device identifier corresponding to the preset operation.

[0017] The feature point recognition module is used to determine the device location based on the device identifier, and to determine the first target feature point from each successfully matched head and shoulder feature point based on the device location, operation time, and positioning and tracking information of each successfully matched head and shoulder feature point.

[0018] The personnel tracking module is used to associate the location tracking information of the first target feature point with the personnel identity information to obtain the location tracking information of each person.

[0019] Thirdly, embodiments of the present invention also provide an electronic terminal, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the indoor personnel positioning method provided in any embodiment of the present application.

[0020] Fourthly, embodiments of the present invention also provide a computer-readable storage medium having a computer program stored thereon, characterized in that, when executed by a processor, the program implements the indoor personnel positioning method provided in any embodiment of the present application.

[0021] This invention provides a method for locating people indoors. The method involves real-time acquisition of images from at least two perspectives of a target area; extraction of head and shoulder feature points from the images; matching the head and shoulder feature points from the simultaneously acquired images from at least two perspectives; determining the location tracking information of the successfully matched head and shoulder feature points; responding to a target device within the target area receiving a preset operation; determining the personnel identity information, operation time, and device identifier corresponding to the preset operation; determining the device location based on the device identifier; and determining a first target feature point from the successfully matched head and shoulder feature points based on the device location, operation time, and location tracking information of each matched head and shoulder feature point; and associating the location tracking information of the first target feature point with the personnel identity information to obtain the location tracking information of each person. This method enables compliant indoor personnel location tracking without terminal devices. Attached Figure Description

[0022] Figure 1 This is a flowchart illustrating an indoor personnel positioning method provided in Embodiment 1 of the present invention;

[0023] Figure 2 This is a flowchart illustrating an indoor personnel positioning method provided in Embodiment 2 of the present invention;

[0024] Figure 3 This is a flowchart of an optional example of an indoor personnel positioning method provided in Embodiment 2 of the present invention;

[0025] Figure 4 This is a flowchart of another optional example of an indoor personnel positioning method provided in Embodiment 2 of the present invention;

[0026] Figure 5 This is a flowchart of another optional example of an indoor personnel positioning method provided in Embodiment 2 of the present invention;

[0027] Figure 6 This is a schematic diagram of the structure of an indoor personnel positioning device provided in Embodiment 3 of the present invention;

[0028] Figure 7 This is a schematic diagram of the structure of an electronic terminal provided in Embodiment 4 of the present invention. Detailed Implementation

[0029] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, the accompanying drawings show only the parts relevant to the invention and not all structures. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without inventive effort are within the scope of protection of the present invention. In the following embodiments, each embodiment provides optional features and examples. The various features described in the embodiments can be combined to form multiple optional solutions, and each numbered embodiment should not be considered as only one technical solution.

[0030] Example 1

[0031] Figure 1 This is a flowchart illustrating an indoor personnel location method according to Embodiment 1 of the present invention. This embodiment is applicable to indoor personnel location in the absence of terminal devices. The method can be executed by the indoor personnel location device provided in this embodiment, which can be implemented in software and / or hardware and can be configured in an electronic terminal, such as a computer.

[0032] See Figure 1 The indoor personnel positioning method provided in this embodiment may include:

[0033] S110: Real-time acquisition of images of at least two perspectives of the target area.

[0034] The target area can be understood as the area where personnel need to be located, such as an indoor area where personnel need to be located; the number of target areas can be one or more, without specific limitation.

[0035] For example, the target area can be equipped with at least one binocular camera, which can be mounted on the indoor ceiling. Each binocular camera can capture images of the target area from at least two perspectives in real time and record the time of the photo capture. The binocular camera consists of two camera devices, allowing one camera to capture images from two different perspectives simultaneously. The binocular cameras can be pre-calibrated with the target device in the target area to maintain clock synchronization.

[0036] S120. Extract head and shoulder feature points from the image, match the head and shoulder feature points in at least two images acquired simultaneously, and determine the localization and tracking information of the successfully matched head and shoulder feature points.

[0037] This involves identifying whether the overall head and shoulder features of an object in an image match those of a human. For example, it can be done by checking if the head and shoulder proportions of the object in the image conform to those of a human. After identifying these features, pixels at the head or shoulder positions in the image can be labeled as head and shoulder feature points. Locating these feature points allows for the location of the person. Location tracking information can be understood as information that enables the tracking and positioning of a person. This could include the person's position coordinates at the time of image acquisition in the world coordinate system, the person's position coordinates at the time of image acquisition in the device coordinate system corresponding to at least two viewpoints, or the person's position coordinates at the time of image acquisition in a custom coordinate system established within the target area. No specific limitations are imposed here. Location tracking information can be relatively continuous information determined based on at least two viewpoints of the target area acquired in real-time. It can exist in the form of a library, such as a (P, t) library, where P is the person's position coordinates and t is the acquisition time of at least two viewpoints of the target area. Successfully matched head and shoulder feature points can be understood as head and shoulder feature points corresponding to the same person.

[0038] For example, head and shoulder feature points corresponding to all people in the image can be extracted. Since a certain head and shoulder feature point in an image corresponding to one viewpoint may be the same person as certain head and shoulder feature points in images corresponding to other views, meaning that there may be a correspondence between head and shoulder feature points in images acquired simultaneously from at least two viewpoints, the head and shoulder feature points in images acquired simultaneously from at least two viewpoints can be matched to determine the correspondence between people in the images from at least two viewpoints, and to determine the location and tracking information of the people corresponding to the successfully matched head and shoulder feature points.

[0039] S130, responding to the target device within the target area receiving a preset operation, and determining the personnel identity information, operation time, and device identifier corresponding to the preset operation.

[0040] Personnel identification information can be information that identifies a person, such as name, ID number, or corresponding ID. Operation time can be understood as the time point at which the target device receives the preset operation. Device identification can be understood as an identifier that distinguishes different target devices, such as the target device's name or number.

[0041] It should be noted that there may be target devices that can be operated by personnel within the target area, such as ticket dispensers or automated teller machines (ATMs). Personnel can operate through the target devices, and at least one of the operations that personnel can perform through the target devices can be preset as a default operation, such as the operation of personnel taking a ticket from a ticket dispenser.

[0042] For example, when a person in the target area takes a number from the ticket dispenser by swiping their ID card, the system can determine the person's identity information, the time of the action, and the ticket dispenser's identifier based on the ID card information swiped into the dispenser.

[0043] In practical applications, embodiments of the present invention can also respond to preset operations by personnel on target devices within the target area, and provide personnel on the target device with a selection interface for whether to authorize the provision of personnel identity information, operation time and device identifier for location purposes. If the personnel agree to the authorization, the personnel identity information, operation time and device identifier corresponding to the preset operation are determined.

[0044] S140. Determine the device location based on the device identifier. Based on the device location, operation time, and positioning and tracking information of each successfully matched head and shoulder feature point, determine the first target feature point from each successfully matched head and shoulder feature point.

[0045] Here, "device location" can be understood as the current spatial location of the device, such as its coordinates in a world coordinate system. "First target feature point" can be understood as the head and shoulder feature points corresponding to the person performing the preset operation during the operation time.

[0046] It's important to note that the target device may be stationary or mobile, such as a service terminal in the hands of a staff member or a mobile robot that can be operated by a user. Therefore, different methods can be used to determine the device's location based on its identifier, depending on whether the target device is mobile. For example, if the target device is stationary, a mapping relationship can be established between the device identifier and a fixed device location, allowing the device's location to be determined directly based on the identifier. If the target device is mobile, its location during the operation time can be obtained based on the device identifier and the operation time.

[0047] For example, considering that the person performing the preset operation on the target device is located within a certain range around the target device during operation, a preset area of ​​fixed size can be pre-defined, such as a 50cm*50cm rectangular area in front of the target device. This area is used to determine the location tracking information of each successfully matched head and shoulder feature point within the certain area where the target device is located when it is necessary to determine the first target feature point, thereby reducing the amount of computation and time consumption. In this embodiment of the invention, the device position can be determined according to the device identifier, and the location tracking information of each successfully matched head and shoulder feature point within the preset area where the device position is located can be determined during operation. Based on the location tracking information of each successfully matched head and shoulder feature point and the device position, the Euclidean distance between each successfully matched head and shoulder feature point within the preset area within a certain range around the target device and the target device can be calculated during operation. The successfully matched head and shoulder feature point closest to the target device can be determined and used as the first target feature point.

[0048] S150. Associate the location tracking information of the first target feature point with the personnel identity information to obtain the location tracking information of each person.

[0049] For example, the person corresponding to the first target feature point can be regarded as the person who performs the preset operation on the target device during the operation time. Therefore, the location tracking information of the first target feature point can be associated with the personnel identity information of the person who performs the preset operation on the target device during the operation time to obtain the location tracking information of the person with known identity information.

[0050] In practical applications, with the customer's authorization, location tracking information can be linked to the customer's preset operations on the target device. This allows for the determination of the customer's input data for the target device, such as business contract data. Linking this input data with the customer's location tracking information helps staff provide in-store services and marketing to customers.

[0051] This invention provides a method for locating people indoors. The method involves real-time acquisition of images from at least two perspectives of a target area; extraction of head and shoulder feature points from the images; matching the head and shoulder feature points from the simultaneously acquired images from at least two perspectives; determining the location tracking information of the successfully matched head and shoulder feature points; responding to a target device within the target area receiving a preset operation; determining the personnel identity information, operation time, and device identifier corresponding to the preset operation; determining the device location based on the device identifier; and determining a first target feature point from the successfully matched head and shoulder feature points based on the device location, operation time, and location tracking information of each matched head and shoulder feature point; and associating the location tracking information of the first target feature point with the personnel identity information to obtain the location tracking information of each person. This method enables compliant indoor personnel location tracking without terminal devices.

[0052] An optional technical solution for extracting head and shoulder feature points from an image includes: processing the image based on at least one histogram determination algorithm to obtain a histogram of at least one format; determining a joint histogram based on the histogram of at least one format; inputting the joint histogram into a support vector machine model; and outputting the head and shoulder feature points in the image through the support vector machine model.

[0053] A histogram is a statistical report graph that represents the distribution of data using a series of vertical bars or line segments of varying heights. A histogram determination algorithm can be understood as an algorithm that processes an image into a histogram. In this embodiment of the invention, the types and number of histogram determination algorithms are not specifically limited. A joint histogram is a graph obtained by statistically analyzing the occurrence frequency of gray-level pairs at corresponding positions in at least one histogram format. A Support Vector Machine (SVM) model is a type of generalized linear classifier that performs binary classification of data using supervised learning. Its decision boundary is the hyperplane with the maximum margin calculated from the training samples.

[0054] It is important to note that in order to extract head and shoulder feature points from an image, the support vector machine model needs to be trained and deployed in advance. The trained and deployed support vector machine model can output the head and shoulder feature points from the image after inputting the joint histogram.

[0055] For example, in this embodiment of the invention, the image can be processed using the Histogram of Oriented Gradient (HOG) algorithm to obtain the HOG histogram calculated by the HOG algorithm; the image can also be processed using the Local Binary Patterns (LBP) algorithm to obtain the LBP histogram calculated by the LBP algorithm; the HOG histogram and the LBP histogram are then concatenated in series to determine the joint histogram; the joint histogram is then input into a pre-trained support vector machine model, where a kernel function is used to transform the linearly inseparable low-dimensional space into a linearly separable high-dimensional space, thereby realizing the head and shoulder feature points in the output image.

[0056] In this embodiment of the invention, by determining a joint histogram from the histograms obtained by different histogram determination algorithms, and inputting the joint histogram into the head and shoulder feature points in the output image of the support vector machine model, a relatively accurate determination of the head and shoulder feature points in the image can be achieved.

[0057] Another optional technical solution, while extracting head and shoulder feature points from the image, the indoor personnel positioning method of this embodiment of the invention further includes: identifying whether a preset marker is contained in the area corresponding to the head and shoulder feature points in the image; if a preset marker is contained, the corresponding head and shoulder feature points are marked as staff feature points; accordingly, based on the device location, operation time, and positioning tracking information of each successfully matched head and shoulder feature point, a first target feature point is determined from each successfully matched head and shoulder feature point, including: determining the first target feature point from each successfully matched head and shoulder feature point based on the device location, operation time, positioning tracking information of each successfully matched head and shoulder feature point, and positioning tracking information of staff feature points.

[0058] The head and shoulder feature point corresponding region can be understood as the area within a certain range where the head and shoulder feature points are located. The range and size of the head and shoulder feature point corresponding region can be preset. For example, the head and shoulder feature point corresponding region can be a rectangular area of ​​50cm*50cm centered on the head and shoulder feature points. The staff feature point can be understood as the feature point that can identify the person corresponding to the head and shoulder feature points as a staff member.

[0059] It is understandable that in practical applications, the personnel in the target area may be of various types. For example, in an office setting, the personnel in the target area may be divided into customer personnel and staff. If it is necessary to distinguish between different types of personnel, some types of personnel can wear preset identification items. These preset identification items can be understood as pre-designed identifiers used to distinguish personnel types, such as employee badges or brooches.

[0060] For example, a support vector machine (SVM) model can be pre-trained to determine head and shoulder feature points based on images from at least two viewpoints. Then, worker feature points can be identified from these head and shoulder feature points by detecting whether a preset marker is present in the region corresponding to the head and shoulder feature points. Specifically, when a preset marker is detected, the corresponding head and shoulder feature point is identified as a worker feature point.

[0061] It should be noted that since there may be not only customer personnel performing preset operations on the target equipment in the target area, but also staff assisting the customer personnel in performing preset operations, the staff feature points can be excluded when determining the first target feature point after the staff are identified.

[0062] For example, the device location is determined based on the device identifier, and the location tracking information of each successfully matched head and shoulder feature point in the preset area where the device location is located is determined during the operation time. Based on the location tracking information of each successfully matched head and shoulder feature point and the device location, the successfully matched head and shoulder feature point of the non-worker feature point closest to the target device during the operation time is determined and used as the first target feature point.

[0063] In this embodiment of the invention, the first target feature point is determined based on the device location, operation time, the location tracking information of each successfully matched head and shoulder feature point, and the location tracking information of the staff feature point. This can reduce the error rate of associating the location tracking information of the first target feature point with the staff identity information.

[0064] Another optional technical solution involves matching head and shoulder feature points in images acquired simultaneously from at least two viewpoints. This includes: correcting the images acquired simultaneously from at least two viewpoints based on the calibration information of the image acquisition device, so that the corrected images from at least two viewpoints are aligned in the horizontal direction; determining the epipolar lines in the horizontal direction between the corrected images from at least two viewpoints using an epipolar constraint algorithm; wherein each epipolar line contains head and shoulder feature points from the corrected images from at least two viewpoints; and matching the head and shoulder feature points located on the same epipolar line using a stereo matching algorithm.

[0065] The calibration information can include the intrinsic and extrinsic parameters of the acquisition device and the homography matrix, etc. The epipolar constraint algorithm can be understood as a matching constraint algorithm that, when an image acquisition device captures head and shoulder feature points in physical space from at least two viewpoints, at least two head and shoulder feature points are formed on the images from at least two viewpoints respectively. For one head and shoulder feature point in one viewpoint image, the algorithm finds the line on which the corresponding head and shoulder feature point is located in the images from other viewpoints. An epipolar line can be understood as the line on which the corresponding head and shoulder feature point is located in the images from at least two viewpoints, for one head and shoulder feature point in one viewpoint image. The stereo matching algorithm estimates the disparity value of a pixel by establishing an energy cost function and minimizing this energy cost function. In this embodiment of the invention, the stereo matching algorithm can find the matching head and shoulder feature points in the images from other viewpoints for one head and shoulder feature point in one viewpoint image from at least two viewpoints. The stereo matching algorithm can be a region-based stereo matching algorithm, a feature-based stereo matching algorithm, or a phase-based stereo matching algorithm, etc. An image acquisition device can be understood as a device that acquires images of a target area from at least two perspectives, such as a binocular camera.

[0066] Specifically, based on the calibration information of the image acquisition device, horizontal correction can be performed on images acquired simultaneously from at least two perspectives to align them horizontally. This ensures that the subsequently determined epipolar lines are on the same horizontal line. Using an epipolar constraint algorithm, horizontal epipolar lines are determined between the corrected images from at least two perspectives. Each epipolar line contains head and shoulder feature points from the corrected images from at least two perspectives. For each head and shoulder feature point in one of the at least two perspectives, a matching head and shoulder feature point can be found on the corresponding horizontal epipolar line in the other perspectives. Finally, a stereo matching algorithm is used to match the head and shoulder feature points located on the same horizontally corresponding epipolar line.

[0067] In this embodiment of the invention, the epipolar constraint algorithm and the stereo matching algorithm are used to improve the accuracy of matching head and shoulder feature points in images acquired simultaneously from at least two viewpoints.

[0068] Another optional technical solution is to determine the localization and tracking information of the successfully matched head and shoulder feature points, including: determining the depth map of the successfully matched head and shoulder feature points based on the principle of triangulation; and determining the position coordinates of the successfully matched head and shoulder feature points at the time of image acquisition based on the calibration information of the image acquisition device and the depth map, so as to obtain the localization and tracking information of the successfully matched head and shoulder feature points.

[0069] The triangulation principle can be understood as a ranging principle that determines the depth of a head and shoulder feature point at different times by projecting its position onto images from at least two viewpoints. A depth map can be understood as an image showing the distance between the head and shoulder feature point and the acquisition device, obtained from images from at least two viewpoints. The acquisition time can be understood as the moment when images from at least two viewpoints were acquired. The position coordinates can be understood as the coordinates of the head and shoulder feature point in the world coordinate system.

[0070] For example, the depth map of successfully matched head and shoulder feature points can be determined based on the principle of triangulation. The calibration information of the image acquisition device can include the transformation relationship from the coordinate system corresponding to the image acquisition device to the world coordinate system. Based on the depth map, the device coordinates of the successfully matched head and shoulder feature points at the time of image acquisition in the coordinate system corresponding to the image acquisition device can be determined. Based on the transformation relationship from the coordinate system corresponding to the image acquisition device to the world coordinate system and the device coordinates in the coordinate system corresponding to the image acquisition device, the position coordinates of the successfully matched head and shoulder feature points at the time of image acquisition can be determined, so as to obtain the positioning and tracking information of the successfully matched head and shoulder feature points.

[0071] After determining the depth map based on the principle of triangulation, the localization and tracking information of the successfully matched head and shoulder feature points can be obtained, which can effectively reduce latency and improve real-time performance.

[0072] Example 2

[0073] The indoor personnel positioning method provided in this embodiment can be combined with various optional schemes in the indoor personnel positioning methods provided in the above embodiments. In this embodiment, optionally, the target area includes at least two, and each target area overlaps with at least one other target area besides the target area itself; the indoor personnel positioning method further includes: in response to a successfully matched head and shoulder feature point in the target area entering the corresponding overlapping area, the successfully matched head and shoulder feature point entering the corresponding overlapping area is taken as a second target feature point; at least one other target area related to the corresponding overlapping area is determined; based on the positioning tracking information of the second target feature point in the target area, a head and shoulder feature point identical to the second target feature point is identified from each successfully matched head and shoulder feature point in the at least one other target area; the positioning tracking information of the identical head and shoulder feature point in the at least one other target area is associated with the positioning tracking information of the second target feature point in the target area. The explanations of terms that are the same as or corresponding to those in the above embodiments are not repeated here.

[0074] Figure 2 This is a flowchart illustrating an indoor personnel positioning method according to Embodiment 2 of the present invention. (See also...) Figure 2 The indoor personnel positioning method provided in this embodiment may include:

[0075] S210. Real-time acquisition of images of at least two perspectives of the target area; wherein the target area includes at least two, and each target area overlaps with at least one other target area besides its own target area.

[0076] Each target region is its own region relative to itself. Overlapping regions can be understood as the areas where this target region and other target regions overlap.

[0077] S220. Extract head and shoulder feature points from the image, match the head and shoulder feature points in at least two images acquired simultaneously, and determine the localization and tracking information of the successfully matched head and shoulder feature points.

[0078] S230, responding to the target device within the target area receiving a preset operation, and determining the personnel identity information, operation time, and device identifier corresponding to the preset operation.

[0079] S240. Determine the device location based on the device identifier. Based on the device location, operation time, and positioning and tracking information of each successfully matched head and shoulder feature point, determine the first target feature point from each successfully matched head and shoulder feature point.

[0080] S250. Associate the location tracking information of the first target feature point with the personnel identity information to obtain the location tracking information of each person.

[0081] S260. In response to the successful matching of head and shoulder feature points within the target area entering the corresponding overlapping area, the successful matching of head and shoulder feature points entering the corresponding overlapping area are used as the second target feature points.

[0082] The second target feature point can be understood as the head and shoulder feature point that has successfully matched the corresponding overlapping area within the target area.

[0083] It should be noted that since there may be more than one successfully matched head and shoulder feature point entering the corresponding overlapping area from the successfully matched head and shoulder feature point in this target area, there may be multiple second target feature points.

[0084] S270. Identify at least one other target region that is related to the corresponding overlapping region.

[0085] Among them, at least one other target region related to the corresponding overlapping region can be understood as other regions that overlap with this target region and correspond to the corresponding overlapping region.

[0086] It should be noted that there may be more than one other target area adjacent to each target area, and therefore the number of other target areas related to the corresponding overlapping area may also be more than one.

[0087] S280. Based on the positioning and tracking information of the second target feature point in this target area, identify the same head and shoulder feature point as the second target feature point from each successfully matched head and shoulder feature point in at least one other target area.

[0088] Among them, the positioning and tracking information of the second target feature point in this target area can be the position coordinates of the second target feature point in the world coordinate system.

[0089] S290. Associate the location tracking information of the same head and shoulder feature points in at least one other target area with the location tracking information of the second target feature points in this target area.

[0090] For example, the starting position of the location tracking information of the same head and shoulder feature point in at least one other target area can be associated with the ending position of the location tracking information of the second target feature point in this target area. This can form a complete and coherent personnel movement trajectory corresponding to the location tracking information of the same head and shoulder feature point when it moves in at least two target areas, which is convenient for assisting indoor services and marketing.

[0091] It should be noted that in practical applications, S260 to S290 can be executed after S220 or after S250, depending on the specific circumstances of the indoor personnel's location, and no specific limitations are made here.

[0092] The indoor personnel positioning method provided in this invention includes at least two target areas, and each target area overlaps with at least one other target area besides its own target area. The method responds to a successfully matched head and shoulder feature point within the target area entering the corresponding overlapping area, designating the successfully matched head and shoulder feature point entering the overlapping area as a second target feature point; it then determines at least one other target area related to the overlapping area; based on the positioning tracking information of the second target feature point within the target area, it identifies head and shoulder feature points identical to the second target feature point from among the successfully matched head and shoulder feature points within the at least one other target area; and it associates the positioning tracking information of the identical head and shoulder feature point within the at least one other target area with the positioning tracking information of the second target feature point within the target area. This enables compliant, terminal-device-free positioning of indoor personnel moving across multiple areas.

[0093] Based on the above solution, an optional technical solution identifies a head and shoulder feature point identical to a second target feature point from among the successfully matched head and shoulder feature points in at least one other target region, comprising: selecting any time when the second target feature point is located in the corresponding overlapping region as the target time; determining the first position coordinates of the second target feature point in this target region at the target time, and the second position coordinates of each of the successfully matched head and shoulder feature points in at least one other target region; determining the distance between the first position coordinates and each second position coordinate; and identifying the head and shoulder feature point corresponding to the second position coordinate with the smallest distance as the same head and shoulder feature point as the second target feature point.

[0094] Here, the target time can be understood as the time when an arbitrarily selected second target feature point is located within its corresponding overlapping region. The first position coordinate can be understood as the position coordinate of the second target feature point in the world coordinate system within this target region. The second position coordinate can be understood as the position coordinate of a successfully matched head and shoulder feature point in at least one other target region in the world coordinate system within at least one other target region.

[0095] In practical applications, the second position coordinates can optionally be the position coordinates of each successfully matched head and shoulder feature point in the overlapping area. This can reduce the amount of computation in determining the distance between the first position coordinates and each second position coordinate, and save time in identifying head and shoulder feature points that are the same as the second target feature point.

[0096] The advantage of identifying the head and shoulder feature point corresponding to the second position coordinate with the smallest distance as the same head and shoulder feature point as the second target feature point is that it can accurately identify the same head and shoulder feature point as the second target feature point.

[0097] Figure 3 This is a flowchart of an optional example of an indoor personnel positioning method provided in Embodiment 2 of the present invention. To better understand the technical solutions of the above embodiments of the present invention, another optional example is provided here. For example, see... Figure 3After personnel enter the room, the indoor binocular cameras take real-time photos and record the photo capture time; extract head and shoulder feature points from images from at least two perspectives; identify staff feature points from the head and shoulder feature points based on preset markers, and simultaneously match head and shoulder feature points from images from at least two perspectives, and determine the depth map of the successfully matched head and shoulder feature points; determine the location tracking information of the personnel corresponding to the successfully matched head and shoulder feature points; when personnel perform preset operations on the target device, determine the personnel identity information, operation time, and device identifier corresponding to the preset operation; determine the personnel performing the preset operation on the target device based on the device location, operation time, and location tracking information of each successfully matched head and shoulder feature point; based on the previously identified staff feature points, determine whether the determined personnel performing the preset operation are staff; if they are staff, re-determine the personnel performing the preset operation on the target device; if they are not staff, continue with subsequent steps; associate the location tracking information of the personnel performing the preset operation on the target device with the personnel identity information.

[0098] Figure 4 This is a flowchart of another optional example of an indoor personnel positioning method provided in Embodiment 2 of the present invention. To better understand the technical solutions of the above embodiments of the present invention, another optional example is provided here. For example, see... Figure 4 If a person enters the corresponding overlapping area c from target area A, the system can respond by having the person's successfully matched head and shoulder feature points within target area A enter the corresponding overlapping area c. The successfully matched head and shoulder feature points of the person entering the corresponding overlapping area c are then designated as the second target feature points. A target area B related to the corresponding overlapping area c is determined, meaning target area A and target area B overlap, and this overlapping area is the corresponding overlapping area c. A target time t is selected where the second target feature point is located within the corresponding overlapping area c. The first position coordinate Pa of the second target feature point in target area A at time t is determined, as well as the second position coordinates P1 to Pn of each successfully matched head and shoulder feature point in target area B. The Euclidean distance between Pa and P1 to Pn is calculated. The head and shoulder feature point corresponding to the second position coordinate Pi with the smallest Euclidean distance is identified as the same head and shoulder feature point as the second target feature point. The positioning and tracking information of the same head and shoulder feature point in target area B is associated with the positioning and tracking information of the second target feature point in target area A, thereby achieving trajectory association of the person in different target areas.

[0099] Figure 5 This is a flowchart of another optional example of an indoor personnel positioning method provided in Embodiment 2 of the present invention. To better understand the technical solutions of the above embodiments of the present invention, another optional example is provided here. For example, see... Figure 5It can acquire images in real time using binocular cameras and perform personnel positioning and trajectory tracking; after determining personnel positioning and tracking information, if a person performs a preset operation on the target device, it can determine the positioning and tracking information of the person who performed the preset operation on the target device, and associate the personnel positioning and tracking information with the personnel's preset operation on the target device to perform personnel positioning and trajectory tracking.

[0100] Furthermore, the indoor personnel positioning method provided in this embodiment belongs to the same technical concept as the indoor personnel positioning method provided in the above embodiments. Technical details not described in detail in this embodiment can be found in the above embodiments, and the same technical features have the same beneficial effects in this embodiment and the above embodiments.

[0101] Example 3

[0102] Figure 6 This is a schematic diagram of an indoor personnel positioning device provided in Embodiment 3 of the present invention. This embodiment is applicable to situations where indoor personnel positioning is required without terminal equipment.

[0103] See Figure 6 The indoor personnel positioning device provided by the present invention may include:

[0104] Image acquisition module 310 is used to acquire images of the target area from at least two perspectives in real time;

[0105] The position recognition module 320 is used to extract head and shoulder feature points in the image, match the head and shoulder feature points in the images acquired simultaneously from at least two viewpoints, and determine the positioning and tracking information of the successfully matched head and shoulder feature points.

[0106] The operation response module 330 is used to respond to the target device in the target area receiving a preset operation and to determine the personnel identity information, operation time and device identifier corresponding to the preset operation.

[0107] The feature point recognition module 340 is used to determine the device location based on the device identifier, and to determine the first target feature point from each successfully matched head and shoulder feature point based on the device location, operation time, and positioning and tracking information of each successfully matched head and shoulder feature point.

[0108] The personnel tracking module 350 is used to associate the location tracking information of the first target feature point with the personnel identity information to obtain the location tracking information of each person.

[0109] Optionally, the location recognition module 320 may include:

[0110] Histogram obtaining unit, used to process an image based on at least one histogram determination algorithm to obtain a histogram in at least one format;

[0111] A joint histogram determination unit is used to determine a joint histogram based on a histogram of at least one format.

[0112] The head and shoulder feature point output unit is used to input the joint histogram into the support vector machine model and output the head and shoulder feature points in the image through the support vector machine model.

[0113] Optionally, the indoor occupant positioning device may also include:

[0114] The preset marker judgment module is used to identify whether preset markers are contained in the area corresponding to the head and shoulder feature points in the image while extracting head and shoulder feature points from the image.

[0115] The staff feature point identification module is used to identify the corresponding head and shoulder feature points as staff feature points if preset markers are included.

[0116] Correspondingly, the feature point recognition module 340 may include:

[0117] The first target feature point determination unit is used to determine the first target feature point from the successfully matched head and shoulder feature points based on the equipment location, operation time, positioning and tracking information of each successfully matched head and shoulder feature point, and positioning and tracking information of the worker feature points.

[0118] Optionally, the location recognition module 320 may include:

[0119] The image correction unit is used to correct images acquired simultaneously from at least two perspectives based on the calibration information of the image acquisition device, so that the corrected images from at least two perspectives are aligned in the horizontal direction.

[0120] The epipolar determination unit is used to determine the horizontal epipolar lines between at least two corrected viewpoints using an epipolar constraint algorithm; wherein each epipolar line contains head and shoulder feature points of at least two corrected viewpoints.

[0121] The head and shoulder feature point matching unit uses a stereo matching algorithm to match head and shoulder feature points located on the same epipolar line.

[0122] Optionally, the location recognition module 320 may include:

[0123] The depth map determination unit is used to determine the depth map of successfully matched head and shoulder feature points based on the principle of triangulation.

[0124] The positioning and tracking information acquisition unit is used to determine the position coordinates of the successfully matched head and shoulder feature points at the time of image acquisition based on the calibration information of the image acquisition device and the depth map, so as to obtain the positioning and tracking information of the successfully matched head and shoulder feature points.

[0125] Optionally, based on the above scheme, the target area may include at least two, and each target area overlaps with at least one other target area besides itself.

[0126] The indoor occupant positioning device may also include:

[0127] The second target feature point is used as a module to respond to the head and shoulder feature points that are successfully matched in the target area entering the corresponding overlapping area, and to use the successfully matched head and shoulder feature points that enter the corresponding overlapping area as the second target feature points.

[0128] The other target region determination module is used to determine at least one other target region related to the corresponding overlapping region;

[0129] The head and shoulder feature point recognition module is used to identify head and shoulder feature points that are the same as the second target feature point from each successfully matched head and shoulder feature point in at least one other target area, based on the positioning and tracking information of the second target feature point in this target area.

[0130] The positioning and tracking information association module is used to associate the positioning and tracking information of the same head and shoulder feature points in at least one other target area with the positioning and tracking information of the second target feature point in this target area.

[0131] Based on the above scheme, the optional head and shoulder feature point recognition module may include:

[0132] The target time determination unit is used to select any time when the second target feature point is located in the corresponding overlapping area as the target time;

[0133] The first and second position coordinate determination unit is used to determine the first position coordinate of the second target feature point in the target area at the target time, and the second position coordinate of each successfully matched head and shoulder feature point in at least one other target area.

[0134] The distance determination unit is used to determine the distance between the first position coordinates and each of the second position coordinates;

[0135] The head and shoulder feature point recognition unit is used to identify the head and shoulder feature point corresponding to the second position coordinate with the smallest distance as the same head and shoulder feature point as the second target feature point.

[0136] This invention provides an indoor personnel positioning device. It acquires images of a target area from at least two perspectives in real time using an image acquisition module; extracts head and shoulder feature points from the images using a location recognition module; matches the head and shoulder feature points from the simultaneously acquired images from at least two perspectives; and determines the positioning and tracking information of the successfully matched head and shoulder feature points. An operation response module responds to a target device within the target area receiving a preset operation, determining the personnel identity information, operation time, and device identifier corresponding to the preset operation. A feature point recognition module determines the device location based on the device identifier, and based on the device location, operation time, and positioning and tracking information of each successfully matched head and shoulder feature point, determines a first target feature point. A personnel tracking module associates the positioning and tracking information of the first target feature point with the personnel identity information to obtain the positioning and tracking information of each person. This device enables compliant indoor personnel positioning without terminal devices.

[0137] The indoor occupant positioning device provided in this embodiment of the invention can execute the indoor occupant positioning method provided in this embodiment of the invention, and has the corresponding functional modules and beneficial effects of the method. For technical details not described in detail, please refer to the indoor occupant positioning method provided in this embodiment of the invention.

[0138] Example 4

[0139] Figure 7 This is a schematic diagram of the structure of an electronic terminal provided in Embodiment 4 of the present invention. Figure 7 As shown, the terminal may include a processor 410, a memory 420, an input device 430, and an output device 440; the number of processors 410 in the terminal may be one or more. Figure 4 Taking a processor 410 as an example; the processor 410, memory 420, input device 430, and output device 440 in the terminal / can be connected via a bus or other means. Figure 7 Taking the example of a connection between China and Israel via a bus.

[0140] The memory 420, as a computer-readable storage medium, can be used to store software programs, computer-executable programs, and modules, such as the program instructions / modules corresponding to the indoor personnel positioning method in this embodiment of the invention (e.g., the image acquisition module, location recognition module, operation response module, feature point recognition module, and personnel tracking module in the indoor personnel positioning device). The processor 410 executes various functional applications and data processing of the device / terminal / server by running the software programs, instructions, and modules stored in the memory 420, thereby realizing the aforementioned indoor personnel positioning method.

[0141] The memory 420 may primarily include a program storage area and a data storage area. The program storage area may store the operating system and at least one application program required for a given function; the data storage area may store data created based on the use of the terminal. Furthermore, the memory 420 may include high-speed random access memory and non-volatile memory, such as at least one disk storage device, flash memory, or other non-volatile solid-state storage device. In some instances, the memory 420 may further include memory remotely located relative to the processor 410, which can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0142] Input device 430 can be used to receive input digital or character information, and to generate key signal inputs related to user settings and function control of the terminal. Output device 440 may include display devices such as a display screen.

[0143] Example 5

[0144] Embodiment 5 of the present invention also provides a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform an indoor personnel positioning method, the method comprising:

[0145] Real-time acquisition of images of the target area from at least two perspectives;

[0146] Extract head and shoulder feature points from the image, match the head and shoulder feature points in images acquired simultaneously from at least two viewpoints, and determine the localization and tracking information of the successfully matched head and shoulder feature points;

[0147] In response to a target device within the target area receiving a preset operation, determine the personnel identity information, operation time, and device identifier corresponding to the preset operation;

[0148] The device location is determined based on the device identifier. Based on the device location, operation time, and positioning and tracking information of each successfully matched head and shoulder feature point, the first target feature point is determined from each successfully matched head and shoulder feature point.

[0149] The location tracking information of the first target feature point is associated with the personnel identity information to obtain the location tracking information of each person.

[0150] Of course, the computer-executable instructions provided in the embodiments of the present invention are not limited to the method operations described above, but can also perform related operations in the indoor personnel positioning method provided in any embodiment of the present invention.

[0151] Based on the above description of the implementation methods, those skilled in the art can clearly understand that the present invention can be implemented using software and necessary general-purpose hardware, and of course, it can also be implemented using hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk, or optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0152] It is worth noting that in the embodiments of the search device described above, the various units and modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of each functional unit are only for easy differentiation and are not used to limit the scope of protection of the present invention.

[0153] Note that the above description is merely a preferred embodiment of the present invention and the technical principles employed. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of the present invention, the scope of which is determined by the scope of the appended claims.

Claims

1. A method for locating people indoors, characterized in that, include: Real-time acquisition of images of the target area from at least two perspectives; Extract head and shoulder feature points from the image, match head and shoulder feature points from at least two simultaneously acquired images, and determine the localization and tracking information of the successfully matched head and shoulder feature points. In response to a target device within the target area receiving a preset operation, the personnel identity information, operation time, and device identifier corresponding to the preset operation are determined; wherein, the target area includes at least two, and each target area overlaps with at least one other target area besides its own target area; The device location is determined based on the device identifier. Based on the device location, the operation time, and the positioning and tracking information of each successfully matched head and shoulder feature point, a first target feature point is determined from each successfully matched head and shoulder feature point. The location tracking information of the first target feature point is associated with the personnel identity information to obtain the location tracking information of each person; The method further includes: In response to a head and shoulder feature point that is successfully matched within the target area entering the corresponding overlapping area, the head and shoulder feature point that is successfully matched within the corresponding overlapping area is taken as the second target feature point. Identify at least one other target region that is related to the corresponding overlapping region; Based on the location tracking information of the second target feature point in the target area, identify the same head and shoulder feature point as the second target feature point from each successfully matched head and shoulder feature point in the at least one other target area; The location tracking information of the same head and shoulder feature points in at least one other target area is associated with the location tracking information of the second target feature point in this target area; The step of identifying head and shoulder feature points identical to the second target feature point from among the successfully matched head and shoulder feature points in at least one other target region includes: Select any moment when the second target feature point is located within the corresponding overlapping region as the target moment; Determine the first position coordinates of the second target feature point in the target region at the target time, and the second position coordinates of each successfully matched head and shoulder feature point in the at least one other target region; Determine the distance between the first position coordinates and each of the second position coordinates; The head and shoulder feature point corresponding to the second position coordinate with the smallest distance is identified as the same head and shoulder feature point as the second target feature point.

2. The method according to claim 1, characterized in that, The extraction of head and shoulder feature points from the image includes: The image is processed based on at least one histogram determination algorithm to obtain a histogram in at least one format; Determine the joint histogram based on the histogram of at least one format; The joint histogram is input into a support vector machine model, and the head and shoulder feature points in the image are output by the support vector machine model.

3. The method according to claim 1, characterized in that, In addition to extracting head and shoulder feature points from the image, the process also includes: Identify whether the region in the image corresponding to the head and shoulder feature points contains a preset marker; If the preset marker is included, the corresponding head and shoulder feature points will be marked as staff feature points; Accordingly, determining the first target feature point from the successfully matched head and shoulder feature points based on the device location, the operation time, and the positioning and tracking information of each successfully matched head and shoulder feature point includes: Based on the device location, the operation time, the location tracking information of each successfully matched head and shoulder feature point, and the location tracking information of the worker's feature point, a first target feature point is determined from each successfully matched head and shoulder feature point.

4. The method according to claim 1, characterized in that, The matching of head and shoulder feature points in images acquired simultaneously from at least two viewpoints includes: Based on the calibration information of the image acquisition device, the images of at least two simultaneously acquired perspectives are corrected so that the corrected images of at least two perspectives are aligned in the horizontal direction. Using the epipolar constraint algorithm, the horizontal epipolar lines between at least two corrected viewpoints are determined; each epipolar line contains head and shoulder feature points of at least two corrected viewpoints. Using a stereo matching algorithm, head and shoulder feature points located on the same epipolar line are matched.

5. The method according to claim 1, characterized in that, The location and tracking information of the successfully matched head and shoulder feature points includes: Based on the principle of triangulation, a depth map is determined for successfully matched head and shoulder feature points. Based on the calibration information of the image acquisition device and the depth map, the position coordinates of the successfully matched head and shoulder feature points at the acquisition time of the corresponding image are determined, so as to obtain the positioning and tracking information of the successfully matched head and shoulder feature points.

6. A positioning device for indoor personnel, characterized in that, include: The image acquisition module is used to acquire images of the target area from at least two perspectives in real time. The location recognition module is used to extract head and shoulder feature points in the image, match head and shoulder feature points in images acquired simultaneously from at least two viewpoints, and determine the location and tracking information of the successfully matched head and shoulder feature points. An operation response module is used to respond to a target device within the target area receiving a preset operation, and to determine the personnel identity information, operation time, and device identifier corresponding to the preset operation; wherein, the target area includes at least two, and each target area overlaps with at least one other target area besides its own target area; The feature point recognition module is used to determine the device location based on the device identifier, and to determine the first target feature point from the matched head and shoulder feature points based on the device location, the operation time, and the positioning and tracking information of each successfully matched head and shoulder feature point. The personnel tracking module is used to associate the location tracking information of the first target feature point with the personnel identity information to obtain the location tracking information of each person. The second target feature point is used as a module to respond to the head and shoulder feature points that are successfully matched in the target area entering the corresponding overlapping area, and to use the successfully matched head and shoulder feature points that enter the corresponding overlapping area as the second target feature points. The other target region determination module is used to determine at least one other target region related to the corresponding overlapping region; The head and shoulder feature point recognition module is used to identify head and shoulder feature points that are the same as the second target feature point from each successfully matched head and shoulder feature point in at least one other target area, based on the positioning and tracking information of the second target feature point in this target area. The positioning and tracking information association module is used to associate the positioning and tracking information of the same head and shoulder feature points in at least one other target area with the positioning and tracking information of the second target feature points in this target area. The head and shoulder feature point recognition module includes: The target time determination unit is used to select any time when the second target feature point is located in the corresponding overlapping area as the target time; The first and second position coordinate determination unit is used to determine the first position coordinate of the second target feature point in the target area at the target time, and the second position coordinate of each successfully matched head and shoulder feature point in at least one other target area. The distance determination unit is used to determine the distance between the first position coordinates and each of the second position coordinates; The head and shoulder feature point recognition unit is used to identify the head and shoulder feature point corresponding to the second position coordinate with the smallest distance as the same head and shoulder feature point as the second target feature point.

7. An electronic terminal, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the indoor personnel positioning method as described in any one of claims 1-5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by the processor, the program implements the indoor personnel location method as described in any one of claims 1-5.

Images