A method and system for global positioning of pedestrian flow integrating wireless signals and video

By combining video and wireless signal data, the problems of pedestrian positioning and identity ID changes caused by obstacle occlusion are solved, and high-accuracy pedestrian global positioning and tracking are achieved in indoor and outdoor scenarios.

CN114169423BActive Publication Date: 2024-11-26SHENZHEN UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202111461849.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-12-02
Publication Date
2024-11-26
Estimated Expiration
2041-12-02

AI Technical Summary

Technical Problem

In the prior art, when pedestrian positioning in indoor and outdoor scenarios, the positioning is inaccurate due to obstacles, and the identity ID changes after the target object is blocked, making it difficult to achieve global positioning tracking.

Method used

Using the method of fusing wireless signals and video, the target object is positioned by obtaining video data and wireless signal data, combining video positioning information and wireless signal positioning information, and using wireless signal data to determine the position of the target object under occlusion, and maintain the identity. Unchanged to achieve global tracking.

Benefits of technology

Improve the accuracy of pedestrian positioning, avoid identity ID changes caused by occlusion, and achieve continuous tracking of target objects.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114169423B_ABST
    Figure CN114169423B_ABST
Patent Text Reader

Abstract

The present invention discloses a method and system for global positioning of pedestrian flows that integrates wireless signals and videos, wherein the method comprises: obtaining video data, and obtaining video positioning information of a target object based on the video data; obtaining wireless signal data, and obtaining wireless signal positioning information of a target object based on the wireless signal data; and positioning the target object based on the video positioning information and the wireless signal positioning information. Compared with the prior art, in the present invention, the target object is positioned by combining video positioning information and wireless signal positioning information. When the target object is obscured in the video, the obscured target object can also be positioned by wireless signal data to improve the accuracy of positioning. At the same time, the identity of the target object identified based on the wireless signal data remains globally unchanged, so changes in the target object identity ID due to obstruction can be avoided, which is conducive to global positioning and tracking of the target object.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of target motion perception and positioning tracking, and in particular to a pedestrian flow global positioning method and system integrating wireless signals and videos. Background Art

[0002] With the development of science and technology, pedestrian positioning technology has also gradually developed. In recent years, the demand for intelligent security, public management, traffic information services and other fields in large public scenes has also grown rapidly, and the problem of pedestrian positioning in indoor and outdoor spaces has received more and more attention.

[0003] In the prior art, a camera is usually used to collect video, and pedestrian detection and recognition is performed on the collected video to achieve the positioning of the target object (pedestrian). The problem with the prior art is that in indoor and outdoor scenes, obstacles may block the target object, which is not conducive to improving the accuracy of positioning, and may cause the target object tracked by the camera to be lost. When the target object appears again, the identity ID will change, which is not conducive to the global positioning and tracking of the target object.

[0004] Therefore, the existing technology still needs to be improved and developed. Summary of the invention

[0005] The main purpose of the present invention is to provide a method and system for global positioning of pedestrian flows that integrates wireless signals and videos, aiming to solve the problem in the prior art that pedestrian detection and identification are directly performed on videos captured by cameras to achieve positioning, which is not conducive to improving the positioning accuracy and is not conducive to global positioning and tracking of target objects in the presence of obstacles.

[0006] In order to achieve the above-mentioned object, the first aspect of the present invention provides a pedestrian flow global positioning method integrating wireless signals and videos, wherein the pedestrian flow global positioning method integrating wireless signals and videos comprises:

[0007] Acquire video data, and acquire video positioning information of a target object based on the video data;

[0008] Acquire wireless signal data, and acquire wireless signal positioning information of the target object based on the wireless signal data;

[0009] The target object is located based on the video positioning information and the wireless signal positioning information.

[0010] Optionally, the acquiring of video data and acquiring video positioning information of the target object based on the video data includes:

[0011] Acquire video data of the target area based on a preset camera device;

[0012] Detect and track the target object in the video data based on a preset target tracking algorithm to obtain the pixel coordinates corresponding to the target object;

[0013] The pixel coordinates are transformed based on a preset coordinate transformation model to obtain video positioning information of the target object, wherein the video positioning information includes the coordinates of the target object in a world coordinate system.

[0014] Optionally, the pixel coordinates are converted based on a preset coordinate conversion model to obtain the video positioning information of the target object, including:

[0015] Based on the coordinate transformation model, a transformation is performed between the pixel coordinate system and the image plane coordinate system to obtain the image plane coordinates corresponding to the pixel coordinates;

[0016] Based on the coordinate transformation model, the image plane coordinate system and the camera coordinate system are transformed to obtain the camera visual three-dimensional coordinates corresponding to the image plane coordinates, wherein the camera visual three-dimensional coordinates are coordinates in the camera coordinate system;

[0017] Based on the above coordinate transformation model, the camera coordinate system and the world coordinate system are transformed to obtain the world coordinates of the target object corresponding to the above camera visual three-dimensional coordinates.

[0018] Optionally, the acquiring of wireless signal data and acquiring wireless signal positioning information of the target object based on the wireless signal data includes:

[0019] Obtaining a preset wireless signal adaptive propagation model, wireless signals in a target area, and the number of the wireless signals, wherein the wireless signal adaptive propagation model is used to obtain the distance between the target object and the wireless signal transmitting end according to the wireless signal strength corresponding to the target object;

[0020] When the number of the wireless signals is less than 3, based on the wireless signal adaptive propagation model, the wireless identity information and distance information of the target object are obtained according to the wireless signals, and used as the wireless signal positioning information, wherein the distance information is the distance between the target object and the wireless signal transmitting end;

[0021] When the number of the wireless signals is not less than 3, based on the wireless signal adaptive propagation model, the wireless identity information and wireless positioning coordinates of the target object are obtained according to the wireless signals and used as the wireless signal positioning information.

[0022] Optionally, the step of obtaining the wireless identity information and wireless positioning coordinates of the target object according to the wireless signal based on the wireless signal adaptive propagation model includes:

[0023] Acquire the wireless signal connection ID of the target object based on the wireless signal and use it as the wireless identity information of the target object;

[0024] Based on the wireless signal adaptive propagation model, three target wireless signal transmitting ends are obtained according to the signal strength of each of the wireless signals at the target object, wherein the target wireless signal transmitting ends are the three wireless signal transmitting ends closest to the target object;

[0025] Based on the distance between each of the above-mentioned target wireless signal transmitting terminals and the above-mentioned target object, the wireless positioning coordinates of the above-mentioned target object are obtained.

[0026] Optionally, the wireless signal adaptive propagation model is iteratively trained in advance based on the following steps:

[0027] Perform synchronous visual acquisition and wireless signal acquisition on the training objects moving in the target area in sequence to obtain multiple sets of visual distances and wireless signal strengths;

[0028] Based on each set of the visual distance and the wireless signal strength, the wireless signal adaptive propagation model is iteratively trained to update the target parameters of the wireless signal adaptive propagation model, wherein the target parameters include an environmental attenuation factor.

[0029] Optionally, the positioning of the target object based on the video positioning information and the wireless signal positioning information includes:

[0030] Matching the video positioning information and the wireless signal positioning information based on an identity matching algorithm to obtain an identity correspondence relationship;

[0031] Based on the above identity correspondence, the target object with the same identity in the above video positioning information and the above wireless signal positioning information is positioned to obtain a positioning result.

[0032] A second aspect of the present invention provides a pedestrian flow global positioning system integrating wireless signals and videos, wherein the pedestrian flow global positioning system integrating wireless signals and videos comprises:

[0033] A video positioning information acquisition module is used to acquire video data and acquire video positioning information of a target object based on the video data;

[0034] A wireless signal positioning information acquisition module, used to acquire wireless signal data, and acquire wireless signal positioning information of a target object based on the wireless signal data;

[0035] A positioning module is used to locate the target object based on the video positioning information and the wireless signal positioning information.

[0036] A third aspect of the present invention provides an intelligent terminal, which includes a memory, a processor, and a global pedestrian positioning program that integrates wireless signals and videos, which is stored in the memory and can be run on the processor. When the global pedestrian positioning program that integrates wireless signals and videos is executed by the processor, it implements any step of the global pedestrian positioning method that integrates wireless signals and videos.

[0037] A fourth aspect of the present invention provides a computer-readable storage medium, on which is stored a program for global positioning of pedestrian flows that integrates wireless signals and videos. When the program for global positioning of pedestrian flows that integrates wireless signals and videos is executed by a processor, the program implements any step of the method for global positioning of pedestrian flows that integrates wireless signals and videos.

[0038] As can be seen from the above, in the scheme of the present invention, video data is obtained, and video positioning information of the target object is obtained based on the above video data; wireless signal data is obtained, and wireless signal positioning information of the target object is obtained based on the above wireless signal data; the target object is positioned based on the above video positioning information and the above wireless signal positioning information. Compared with the method of positioning based only on the video captured by the camera in the prior art, in the present invention, the target object is positioned in combination with the video positioning information and the wireless signal positioning information. When the target object is obscured in the video, the obscured target object can also be positioned by the wireless signal data to improve the accuracy of positioning. At the same time, the identity of the target object identified based on the wireless signal data remains globally unchanged, so the change of the target object identity ID due to occlusion can be avoided, which is conducive to the global positioning and tracking of the target object. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0040] Figure 1 It is a flow chart of a method for global positioning of pedestrian flow by integrating wireless signals and videos provided by an embodiment of the present invention;

[0041] Figure 2 Embodiment of the present invention Figure 1 Specific flow diagram of step S100;

[0042] Figure 3 Embodiment of the present invention Figure 2 Specific flow diagram of step S103;

[0043] Figure 4 Embodiment of the present invention Figure 1 Specific flow diagram of step S200;

[0044] Figure 5 is a schematic diagram of neighbor three-side positioning provided by an embodiment of the present invention;

[0045] Figure 6 Embodiment of the present invention Figure 1 Specific flow diagram of step S300;

[0046] Figure 7 This is a flowchart of a complex scene multi-target global trajectory positioning that integrates wireless signals and videos provided by an embodiment of the present invention;

[0047] Figure 8 It is a structural schematic diagram of a pedestrian flow global positioning system integrating wireless signals and videos provided by an embodiment of the present invention;

[0048] Fig. 9 It is a block diagram of the internal structure principle of an intelligent terminal provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0049] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present invention. However, it should be clear to those skilled in the art that the present invention may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present invention.

[0050] It should be understood that when used in this specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0051] It should also be understood that the terms used in the present specification are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the present specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0052] It should be further understood that the term "and / or" used in the present description and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0053] As used in this specification and the appended claims, the term "if" may be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrases "if it is determined" or "if [described condition or event] is detected" may be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0054] The following is a clear and complete description of the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0055] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0056] With the development of science and technology, pedestrian positioning technology has also gradually developed. In recent years, the demand for intelligent security, public management, traffic information services and other fields in large public scenes has also grown rapidly, and the problem of pedestrian positioning in indoor and outdoor spaces has received more and more attention.

[0057] In the prior art, a camera is usually used to collect video, and pedestrian detection and recognition is performed on the collected video to achieve the positioning of the target object (pedestrian). The problem with the prior art is that in indoor and outdoor scenes, obstacles may block the target object, which is not conducive to improving the accuracy of positioning, and may cause the target object tracked by the camera to be lost. When the target object appears again, the identity ID will change, which is not conducive to the global positioning and tracking of the target object.

[0058] At present, the rapidly developing satellite positioning technology is mainly applicable to outdoor scenes. It is difficult to apply to pedestrian positioning in large public indoor scenes or even indoor and outdoor composite scenes (such as airports, transportation hubs, large office buildings, commercial complexes, etc.), and cannot meet the increasingly urgent positioning needs. Existing indoor pedestrian positioning technologies (such as Bluetooth, ultrasound, radio frequency identification, ultra-wideband, etc.) are all heavily dependent on the laying of a large number of infrastructure or instruments, and it is difficult to promote them on a large scale. Therefore, the current public scene security monitoring mainly uses pedestrian monitoring technology based on fixed cameras, which has the advantages of low cost and easy information backtracking. In recent years, pedestrian detection and recognition technology based on camera video has developed rapidly, and can quickly identify pedestrians in the video. However, due to the large space and complex building structure of public scenes, a large number of cameras (multi-camera networks) are usually required to complete the coverage of the main areas of the scene, and most of them have many monitoring blind spots. At the same time, a large number of pedestrian flows usually walk densely and have strong visual similarities. Therefore, the existing visual detection technology has serious problems such as pedestrian misdetection, missed detection, only screen pixel coordinates can be obtained, and the global unique identity ID cannot be determined, which makes it difficult to apply to refined security prevention and control applications. Therefore, current public scene monitoring based on camera networks still mainly relies on manual interpretation, which has low efficiency and intelligence, and it is difficult to achieve precise three-dimensional global spatial positioning of large-scale pedestrians.

[0059] In this application, pedestrian positioning is performed in combination with a wireless signal network (such as Wi-Fi), and the pedestrian's network access device (such as a smart phone) can be used as the pedestrian's globally unique ID. In one application scenario, fingerprint positioning can be performed based on wireless signals, but additional fingerprint map data needs to be collected, a large amount of fingerprint map data needs to be collected and updated regularly, and the processing process is complicated. Therefore, in this application, wireless signal positioning is performed based on the wireless signal propagation model to achieve multi-pedestrian (or single pedestrian) perception and global positioning in various scenarios.

[0060] In order to solve at least one of the above-mentioned problems, in the solution of the present invention, video data is obtained, and video positioning information of the target object is obtained based on the above-mentioned video data; wireless signal data is obtained, and wireless signal positioning information of the target object is obtained based on the above-mentioned wireless signal data; the target object is positioned based on the above-mentioned video positioning information and the above-mentioned wireless signal positioning information. Compared with the method of positioning based only on the video captured by the camera in the prior art, in the present invention, the target object is positioned by combining the video positioning information and the wireless signal positioning information. When the target object is obscured in the video, the obscured target object can also be positioned by the wireless signal data to improve the accuracy of positioning. At the same time, the identity of the target object identified based on the wireless signal data remains globally unchanged, so the change of the target object identity ID due to occlusion can be avoided, which is conducive to the global positioning and tracking of the target object.

[0061] Exemplary Methods

[0062] like Figure 1 As shown, an embodiment of the present invention provides a pedestrian flow global positioning method integrating wireless signals and videos. Specifically, the method includes the following steps:

[0063] Step S100: acquiring video data, and acquiring video positioning information of a target object based on the video data.

[0064] The video data is a video containing a target object (i.e., a pedestrian) obtained by a camera or other imaging device. The video positioning information is the position information corresponding to the target object obtained by detecting and identifying pedestrians in the video data, and may include the coordinates and identity information of the target object.

[0065] Step S200: Acquire wireless signal data, and acquire wireless signal positioning information of a target object based on the wireless signal data.

[0066] The wireless signal data is data corresponding to the wireless signal in the area where the target object is located (ie, the target area). In this embodiment, the wireless signal is a Wi-Fi signal.

[0067] Specifically, multiple wireless signal transmitting terminals can be pre-set in the above-mentioned target area, and the smart terminal carried by the target object (such as a mobile phone, a tablet, a smart watch, etc.) can be used as a wireless signal receiving terminal. According to the strength of the wireless signal received by the smart terminal of the target object, the distance between the target object and each wireless signal transmitting terminal can be obtained, thereby realizing the positioning of the target object. At the same time, when the smart terminal of the target object is connected to the wireless signal transmitting terminal, the corresponding smart terminal ID is unchanged, and the global identity information of the target object can be obtained based on this. Specifically, wireless data has the property of unchanged ID, that is, as long as the vision of each camera is matched with the wireless ID, the global ID of the pedestrian can be obtained. Therefore, in this embodiment, the above-mentioned wireless signal positioning information can include the wireless identity information and distance information (or wireless positioning coordinates) of the target object.

[0068] In one application scenario, the position of the wireless signal transmitter can be adjusted according to actual needs. In another application scenario, multiple wireless signal receivers can be pre-set, and the smart terminal carried by the target object can be used as the wireless signal transmitter, which can also achieve the positioning of the target object.

[0069] Step S300: positioning the target object based on the video positioning information and the wireless signal positioning information.

[0070] Specifically, in this embodiment, after obtaining the above-mentioned video positioning information and the above-mentioned wireless signal positioning information, the two are combined to locate the target object. It should be noted that when positioning, the video positioning information and the wireless signal positioning information at the same moment are combined to locate the same target object at that moment. In this way, even if the target object is blocked, its position can be obtained through the wireless positioning information. And in the process of continuous positioning, the global identity information of a target object (i.e., identity information that will not change) can be obtained by combining the two to achieve continuous tracking of the target object.

[0071] As can be seen from the above, in the scheme of the present invention, video data is obtained, and video positioning information of the target object is obtained based on the above video data; wireless signal data is obtained, and wireless signal positioning information of the target object is obtained based on the above wireless signal data; the target object is positioned based on the above video positioning information and the above wireless signal positioning information. Compared with the method of positioning based only on the video captured by the camera in the prior art, in the present invention, the target object is positioned in combination with the video positioning information and the wireless signal positioning information. When the target object is obscured in the video, the obscured target object can also be positioned by the wireless signal data to improve the accuracy of positioning. At the same time, the identity of the target object identified based on the wireless signal data remains globally unchanged, so the change of the target object identity ID due to occlusion can be avoided, which is conducive to the global positioning and tracking of the target object.

[0072] In this embodiment, Figure 2 As shown, the above step S100 specifically includes the following steps:

[0073] Step S101: acquiring video data of a target area based on a preset camera device.

[0074] Step S102: Detect and track the target object in the video data based on a preset target tracking algorithm to obtain pixel coordinates corresponding to the target object.

[0075] Step S103: transform the pixel coordinates based on a preset coordinate transformation model to obtain video positioning information of the target object, wherein the video positioning information includes the coordinates of the target object in a world coordinate system.

[0076] Among them, the above-mentioned preset camera device is a camera, the above-mentioned target area is the area where the target object is located, that is, the area where pedestrian positioning and tracking is required, and the above-mentioned video positioning information can also include the visual identity information of the target object (that is, the user identity ID obtained through video data). In this embodiment, multiple cameras are pre-set to build a multi-camera network, and video acquisition is performed on the target area together, so as to locate pedestrians from multiple angles and improve positioning accuracy. The above-mentioned preset target tracking algorithm is the yolo-deepsort algorithm, which detects and tracks pedestrians based on yolo-deepsort. This algorithm can effectively output the pixel coordinates of pedestrians and give pedestrians unique identity information. It should be noted that other algorithms can also be used in actual use, which are not specifically limited here.

[0077] It should be noted that, based on the method in this embodiment, a single frame of video data can be processed to achieve the positioning of the target object, or multiple frames of video data can be processed continuously to achieve continuous positioning and tracking. In this embodiment, the processing of a single frame of video data is used as an example for explanation. In the actual processing process, after processing one frame, the next frame is processed to achieve continuous tracking.

[0078] Through the above detection, the pixel coordinates of the target object are obtained. In this embodiment, the pixel coordinates are also transformed through a coordinate transformation model to obtain corresponding world coordinates.

[0079] Specifically, in this embodiment, Figure 3 As shown, the above step S103 includes:

[0080] Step S1031 , performing conversion between the pixel coordinate system and the image plane coordinate system based on the coordinate conversion model, and obtaining the image plane coordinates corresponding to the pixel coordinates.

[0081] Step S1032, converting between the image plane coordinate system and the camera coordinate system based on the coordinate conversion model, and obtaining the camera visual three-dimensional coordinates corresponding to the image plane coordinates, wherein the camera visual three-dimensional coordinates are coordinates in the camera coordinate system.

[0082] Step S1033: convert between the camera coordinate system and the world coordinate system based on the coordinate conversion model to obtain the world coordinates of the target object corresponding to the camera visual three-dimensional coordinates.

[0083] Among them, the above-mentioned coordinate transformation model is a pre-built and trained model. The above-mentioned coordinate transformation model is based on the camera imaging principle and the camera calibration method, and is used to realize the transformation between pixels and the world coordinate system, involving a two-dimensional pixel coordinate system, an image plane coordinate system, a three-dimensional camera coordinate system and a world coordinate system.

[0084] Specifically, the pixel coordinate system and the image plane coordinate system are both on the imaging plane, but their origins and measurement units are different. The conversion between the two is shown in the following formula (1):

[0085]

[0086] Among them, u and v represent the horizontal and vertical coordinates in the pixel coordinate system, respectively, and x and y represent the horizontal and vertical coordinates in the image plane coordinate system, respectively, that is, (x, y) represents the image plane coordinates. (u0, v0) represents the origin translation of the image plane coordinate system relative to the pixel coordinate system. Generally, the center point (origin) of the pixel coordinate system is the upper left corner of the image. The coordinates of the center of the image plane coordinate system in the pixel coordinate system can be expressed as (u0, v0). The corresponding translation is the translation of the two coordinate systems so that their origins coincide. dx and dy represent the horizontal and vertical pixel numbers that differ between the center pixel coordinates of the image and the pixel coordinates of the dot in the image, respectively.

[0087] Furthermore, the conversion between the image plane coordinate system and the three-dimensional camera coordinate system is performed based on the perspective projection relationship:

[0088]

[0089] Among them, (X c , Y c , Z c ) represents the camera visual 3D coordinates in the camera coordinate system, and f represents the intrinsic parameters of the camera. The transformation from the camera coordinate system to the world coordinate system is a rigid body transformation, that is, the coordinate system will not be deformed and mainly relies on rotation transformation and translation transformation to transform each other. Therefore, the transformation relationship can be expressed as a rigid body transformation matrix Through the conversion relationship between the above coordinate systems, a conversion relationship from the pixel coordinate system to the world coordinate system can be obtained, as shown in the following formula (3):

[0090]

[0091] Among them, (X w , Y w , Z w ) represents the world coordinates corresponding to the pixel coordinates (u, v), R represents the rotation matrix, and T represents the translation vector. Based on the above transformation relationship, the corresponding coordinate transformation model can be constructed. x and f y is the intrinsic parameter corresponding to the camera, It should be noted that in this embodiment, f, dx, dy, u0 and v0 are all camera internal parameters, f represents the focal length; (dx, dy) represents the physical size of each pixel in the u-axis and v-axis directions as dx and dy, with the unit recorded in mm / pixel; (u0, v0) is the coordinate of the center of the image plane on the pixel plane. In this embodiment, the camera internal parameters are obtained by using the camera calibration method, and a suitable origin position is selected in an indoor environment, and Z is set to w The ground elevation is fixed, the X-axis and Y-axis are oriented to the west and south respectively, a world coordinate system is constructed, reference points are arranged, and a coordinate conversion model is constructed. For the pedestrian target tracked in the image, the pixel coordinates of the sole of its foot are used as the positioning point to obtain its real trajectory coordinates under a single camera. At this time, the obtained pedestrian trajectory coordinates are independent of each other based on the coordinate systems of different cameras. In this embodiment, the optimal transformation matrix is ​​obtained to unify them.

[0092] In this embodiment, the optimal transformation matrix of multiple sets of same-name point data sets (A, B, C, ...) in the coordinate systems corresponding to different cameras is obtained. Each data set A, B and C includes the coordinates of the same-name points corresponding to different cameras of the same target object. For example, data set A includes a1, a2 and a3, which are the coordinate data in the coordinate system corresponding to camera A; data set B includes b1, b2 and b3, which are the coordinate data in the coordinate system corresponding to camera B; a1 and b1 correspond to the same point of the same target object, that is, a1 and b1 are same-name points.

[0093] Specifically, in this embodiment, the multi-coordinate system conversion formula can be expressed as shown in the following formula (4):

[0094]

[0095] Among them, data set A is the data set in the j coordinate system (that is, each point in data set A is the coordinate in the coordinate system corresponding to camera j), data set B is the data set in the m coordinate system, and data set C is the data set in the n coordinate system. is a specific point in data set A, is a specific point in data set B, is a specific point in the data set C, R B With t B is the transformation matrix from the m coordinate system to the j coordinate system, R B is a 3×3 rotation matrix, and t is the translation vector. During the transformation, the centers of the n and m coordinate systems often do not coincide, so they need to be constructed in a base coordinate system with the same origin. First, find the centroid (i.e., the average point) of the two data sets, which can be expressed as:

[0096]

[0097] Where N is the number of points in the data set. For example, the number of points in data sets A and B is the same, and centroid A and centroid B Represent the centroids of datasets A and B, respectively. Recentering the two datasets so that both centroids are at the origin of the same coordinate system temporarily eliminates translation and only needs to consider rotation. In this case, the matrix for the cumulative transformation from dataset A to B is H B definition:

[0098]

[0099] Use singular value decomposition (SVD) to transform the matrix H B Decomposed into 3 independent matrices, the rotation matrix R is obtained B Finally, the center of mass is substituted into equation (7) to obtain the translation vector t B :

[0100]

[0101] Similarly, for the remaining coordinate systems, the transformation matrix and translation vector are obtained based on the same method as above, so that the real coordinates (X m , Y m , Z m ) are integrated into a unified coordinate system (a unified coordinate system refers to the same world coordinate system, in which each camera corresponds to an independent world coordinate system), as follows, to obtain the complete trajectory of the pedestrian:

[0102]

[0103] In this embodiment, it is assumed that the coordinates of all cameras are converted to the j coordinate system, so the j coordinate system is the world coordinate system finally obtained, (X j , Y j , Z j ) is the coordinate in the unified world coordinate system finally obtained. It should be noted that X n represents the horizontal coordinate in the coordinate system n, R c Represents the rotation matrix of the coordinate system n corresponding to the dataset C, and V corresponding to the dataset C C , U C Other parameters can refer to V B , U B Thus, by establishing a coordinate transformation model according to the above method, the pixel coordinates of different cameras can be transformed into a unified world coordinate system, which is convenient for positioning and tracking the user.

[0104] Specifically, in this embodiment, Figure 4As shown, the above step S200 includes the following steps:

[0105] Step S201, obtain a preset wireless signal adaptive propagation model, wireless signals in a target area, and the number of the wireless signals, wherein the wireless signal adaptive propagation model is used to obtain the distance between the target object and the wireless signal transmitter according to the wireless signal strength corresponding to the target object.

[0106] Step S202: When the number of the wireless signals is less than 3, based on the wireless signal adaptive propagation model, the wireless identity information and distance information of the target object are obtained according to the wireless signals, and used as the wireless signal positioning information, wherein the distance information is the distance between the target object and the wireless signal transmitter.

[0107] Step S203, when the number of the wireless signals is not less than 3, based on the wireless signal adaptive propagation model, the wireless identity information and wireless positioning coordinates of the target object are obtained according to the wireless signals, and used as the wireless signal positioning information.

[0108] Specifically, in this embodiment, a flexible Wi-Fi information acquisition method is constructed, and a suitable wireless information extraction method can be selected according to the number of wireless signals in the environment. Therefore, there is no need to conduct a large number of preliminary experiments to measure the effective propagation distance of Wi-Fi, and there is no need to ensure that each location is covered by at least three wireless signals. The wireless signal adaptive propagation model in this embodiment defines the experimental area into two categories according to the number of signal access points and selects different Wi-Fi data acquisition modes. When there are less than 3 Wi-Fi coverage areas, only wireless adaptive propagation is used to obtain the distance and identity information of the receiving-transmitting end. Such data is used for self-calibration of the propagation model and providing identity information; otherwise, three-sided positioning is used to obtain coordinates and identity information. Such data is divided into wireless identity-coordinate data. In addition to the above uses, it is also used to fuse visual information to complete multi-camera trajectory output. Whether the three-sided positioning is used depends only on the number of Wi-Fi at this time node, so the number of the two types of data obtained can be adjusted by appropriately adding the total number of Wi-Fi or changing some positions for subsequent analysis.

[0109] Specifically, the above-mentioned method based on the above-mentioned wireless signal adaptive propagation model obtains the wireless identity information and wireless positioning coordinates of the above-mentioned target object according to the above-mentioned wireless signal, including: obtaining the wireless signal connection ID of the above-mentioned target object based on the above-mentioned wireless signal, and using it as the wireless identity information of the above-mentioned target object; based on the above-mentioned wireless signal adaptive propagation model, obtaining three target wireless signal transmitting ends according to the signal strength of each of the above-mentioned wireless signals at the above-mentioned target object, wherein the above-mentioned target wireless signal transmitting ends are the three wireless signal transmitting ends closest to the above-mentioned target object; based on the distance between each of the above-mentioned target wireless signal transmitting ends and the above-mentioned target object, obtaining the wireless positioning coordinates of the above-mentioned target object.

[0110] In this embodiment, the wireless signal adaptive propagation model analyzes the nonlinear relationship between the receiving strength value RSS of the Wi-Fi signal and the distance between the receiving and transmitting ends in different environments and the attenuation characteristics of the multipath effect, thereby accurately fitting the parameters of the propagation model and obtaining the distance between the signal transmitting end and the signal receiving end. Specifically, the wireless signal adaptive propagation model simulates the loss of wireless signal propagation in different environments based on the logarithmic path loss model, which is expressed as follows:

[0111]

[0112] Wherein, d represents the distance between the signal transceiver, RSS represents the signal receiving strength value (i.e., signal strength). Specifically, RSS(d) represents the signal receiving strength value when the distance between the signal transceiver is d, and RSS(d0) represents the signal receiving strength value when the distance between the signal transceiver is d0. Wherein, d0 is usually preset to 1m, and q is the environmental attenuation factor, which can be preset. The attenuation of the signal in different obstacles is different, which can be determined through calibration tests. Represents the logarithm with base e. Specifically, the conversion formula between signal strength and distance is obtained:

[0113]

[0114] In this embodiment, the RSS (d0) value and the environmental coefficient q at different distances can be fitted by the least square method to obtain the most suitable parameters of the propagation model in different scenarios, that is, to optimize the target parameters of the wireless adaptive propagation model, wherein the target parameters include the environmental attenuation factor, and then convert the signal strength into a distance value. After the wireless adaptive propagation model is constructed, three-sided positioning can be performed.

[0115] Specifically, based on the above-mentioned wireless adaptive propagation model, the actual distance from each receiver to the signal source can be obtained, and when three Wi-Fi signals are obtained, the position can be obtained by three-sided positioning. In an application scenario, the maximum likelihood estimation method can be used for positioning when the number of receivers is greater than 3. It needs to estimate the positioning result by weighted method, least square method, centroid method and other methods. The calculation is relatively complicated and cannot meet the real-time requirements. At the same time, as the distance increases, there is an error in the signal strength value of some receivers that are far away from the signal source. Therefore, in this embodiment, on the basis of three-sided positioning, the neighbor three-sided positioning method (or neighbor three-sided positioning method, that is, the nearest three sides are selected for positioning) is used for the multi-signal coverage range. Specifically, when the access point (that is, the smart terminal carried by the user) enters the multi-signal coverage range, the distance between it and each wireless signal transmitter is calculated in real time, and the three wireless signal transmitters closest to the signal access point are selected to complete the positioning.

[0116] Figure 5 is a schematic diagram of a neighbor three-side positioning provided by an embodiment of the present invention, such as Figure 5 As shown in the figure, at the current time (time node t), the signal of the unknown node's signal transmitter F can be received by more than 3 probes (i.e., Wi-Fi access points), or the unknown node's signal receiving end F can receive wireless signals from more than 3 signal transmitting ends. The coordinates of point F are represented as (x, y), and the four points J, G, H, and E are anchor nodes, representing the signal receiving ends with known locations, with coordinates of (x1, y1), (x2, y2), (x3, y3), and (x4, y4), respectively, and their distances from point F are Then there is the following formula:

[0117]

[0118] By selecting the three smallest distances in each group of data, that is, excluding the ones with larger distances at the current time node, Distance, the coordinates of node F can be obtained by least squares fitting:

[0119]

[0120] In this way, global pedestrian wireless positioning data can be obtained by combining proximity three-sided positioning with three-sided positioning (when there are only three access points, ordinary three-sided positioning is used) to obtain more accurate wireless positioning coordinates and provide identity assistance for subsequent global visual positioning.

[0121] Specifically, in this embodiment, the above-mentioned wireless signal adaptive propagation model is iteratively trained in advance based on the following steps: synchronously performing visual acquisition and wireless signal acquisition on the training objects moving in the target area in turn to obtain multiple groups of visual distances and wireless signal strengths; based on each group of the above-mentioned visual distances and the above-mentioned wireless signal strengths in turn, the above-mentioned wireless signal adaptive propagation model is iteratively trained to update the target parameters of the above-mentioned wireless signal adaptive propagation model, wherein the above-mentioned target parameters include the environmental attenuation factor.

[0122] It should be noted that, in one embodiment, the video data and wireless signal data corresponding to the training object can be obtained in advance for training. In another embodiment, training can also be performed directly based on the video data and wireless signal data corresponding to the current target object. In this embodiment, iterative training is performed by pre-acquiring the data of the training object as an example for explanation, but it is not a specific limitation. The above-mentioned target parameters are parameters that need to be iteratively trained, including the environmental attenuation factor q. In this embodiment, the above-mentioned target parameters also include the wireless signal strength RSS(d0) corresponding to the preset unit distance d0, where the preset unit distance is 1 meter.

[0123] Since the accuracy of visual ranging is higher than that of Wi-Fi positioning, in this embodiment, the parameters of the wireless signal adaptive propagation model are iteratively calibrated through the visual positioning results to improve system performance. Specifically, the frequency of wireless signal acquisition is lower than that of vision. Therefore, the visual processing frequency is calibrated based on the wireless time node. The premise of self-calibration is to obtain the relative position relationship between the camera and the current scene Wi-Fi, so that the visual positioning results can be used to calculate the distance between pedestrians and Wi-Fi in real time, and this data at time point t is recorded as the visual distance: It is also necessary to record the signal strength value of the pedestrian's position at time t, namely: Take RSS(d0)=-60 and environmental coefficient (i.e., environmental attenuation factor) q=2 as initial values. In one application scenario, the adaptive propagation model needs to obtain a large number of signal values ​​and real distance corresponding vectors to estimate various parameters. In this embodiment, by using the visual distance Instead of the actual distance that needs to be measured, it reduces the processing difficulty and improves the processing efficiency. After each time node (that is, as the value of t changes), it is equivalent to adding a piece of data that can be fitted. Repeat the above process until the pedestrian trajectory leaves the current Wi-Fi measurement range, and then a set of data that can be fitted can be obtained. Corresponding vector, substitute it into the following formula:

[0124]

[0125] in, represents the logarithm with base e, is The wireless signal strength collected at the distance (i.e., time node t). The least squares method is used for nonlinear fitting to solve the RSS (d0) and environmental coefficient of Wi-Fi in the current environment. When there are multiple pedestrians in the scene, the subsequent identity matching algorithm is required to construct the correspondence between the visual identity and the wireless identity and classify them by identity. The multi-pedestrian visual data can be divided into multiple groups of fitting data to further study the linear relationship between the parameters. The storage structure of the multi-pedestrian wireless signal propagation self-calibration database can be expressed as:

[0126]

[0127] It contains the Wi-Fi signal strength at time node (t) Visual distance Different transmitter IDs (EMI_ID) and receiver IDs (REC_ID) are classified and stored, and multiple groups of data are integrated according to the transmitter ID and receiver ID. This is used to iterate and increase the accuracy of wireless positioning, making the adaptive conversion model more adaptable and portable.

[0128] In the specific iteration process, the iteration can also be performed in sequence based on the data corresponding to different training objects. For example, by and visual distance Substitute the first result into formula (13) to replace the original RSS (d0) and environmental coefficient. Then, the Wi-Fi signal strength of pedestrian 2 is and visual distance Substitute the updated coefficients into the equation (13) to get the second iteration result, and use it to replace the RSS (d0) and environmental coefficient obtained in the first iteration. Repeating this process can increase the accuracy of subsequent results.

[0129] Specifically, in this embodiment, Figure 6 As shown, the above step S300 includes:

[0130] Step S301: Match the video positioning information and the wireless signal positioning information based on an identity matching algorithm to obtain an identity correspondence.

[0131] Step S302: Based on the identity correspondence, locate the target object with the same identity in the video positioning information and the wireless signal positioning information to obtain a positioning result.

[0132] The above-mentioned identity correspondence relationship reflects the correspondence between the identity of the target object obtained through video data and the identity of the target object obtained through wireless signal data.

[0133] Specifically, cross-camera detection may cause the problem of pedestrian identity change, that is, the same pedestrian has different IDs. In this embodiment, the Wi-Fi identity information is matched to the pedestrian tracked in the image as the unique ID for pedestrian identification, thereby unifying the identity of the same pedestrian in multi-camera scenarios. After obtaining visual and wireless positioning data, a wireless identity-coordinate database is established by matching the Wi-Fi and visual data with time tags. The method is to store, where n represents the number of access points, i∈n; Cam_ID represents the camera number, t represents the time node, Represents the visual positioning coordinates and visual identity, X I and Y I Represents the coordinates in the world coordinate system obtained through video data, represents the visual identity of the target object obtained by object detection in the video; represents a Wi-Fi positioning coordinate data set, where RSSI X n Represents the horizontal coordinate of the wireless positioning obtained through wireless signal data, RSSI Y n Represents the wireless positioning vertical coordinate, Represents the wireless signal connection ID. It should be noted that in this embodiment, by selecting a suitable origin position, the vertical coordinate Z is set to a fixed ground elevation value (for example, 1), so the obtained world coordinates and wireless positioning coordinates only need to consider the horizontal and vertical coordinates. In actual use, the vertical coordinate can also be considered according to actual needs, which is not specifically limited here.

[0134] In this embodiment, Wi-Fi and vision are integrated to perform multi-camera pedestrian tracking to achieve continuous tracking of pedestrian trajectories across cameras. The specific steps are as follows: The first step is to perform data preprocessing, including the time base of the same two devices (camera and wireless signal acquisition device), remove Wi-Fi noise values, decode and adjust video parameters, match data acquisition frequency, etc. The second step is to collect and annotate the extracted part of the visual pedestrian data, use the pedestrian annotation data to train the Darknet network to obtain the detection weight, and use the default parameter weight yolo-deepsort to track the pedestrian trajectory under a single camera. The third step is to build a coordinate transformation model for the scene. First, the internal parameters are obtained through the Zhang Zhengyou calibration method and the distortion correction of the visual data is completed. The world coordinate system is established (the world coordinate system of each camera is independent), and then multiple groups of control points with known coordinates are arranged to generate a pixel-world coordinate correspondence relationship file containing only the control points, and a coordinate transformation model is constructed to obtain the coordinates of the real visual trajectory. The fourth step is to build a vision-assisted wireless self-calibration model. First, the areas are roughly divided according to the number of detectable Wi-Fi. The areas do not overlap and the parameters of the adaptive propagation model are independent of each other. The visual positioning results and the position relationship between the camera and the Wi-Fi are used to solve the visual distance. Collect Wi-Fi signal strength data of control points through mobile devices equipped with Wi-Fi receivers Construct a multi-pedestrian wireless signal propagation self-calibration database; substitute the corresponding visual distance-signal strength into the adaptive conversion model, iterate and update the attenuation factor and environmental coefficient of the area, and update the parameters of the remaining areas in the same way. The fifth step is to establish a wireless propagation model error correction algorithm, compare the correlation coefficient of the adaptive parameters of each area with the accuracy of the wireless ranging results, and thus fine-tune the number of Wi-Fi in the area with poor accuracy. The sixth step is to use three-sided positioning to obtain the wireless identity-coordinates for the divided areas with 3 or more Wi-Fi coverage, obtain the current visual positioning results, and then construct a wireless identity-coordinate database according to the time node. The seventh step is to use the global identity matching algorithm to match the data in the wireless identity-coordinate database to obtain the corresponding relationship between wireless and visual identities. The eighth step is to fuse the coordinates of the same pedestrian trajectory under the multi-camera network through the wireless identity information, use the same-name points between the coordinate systems to obtain the best conversion matrix, and uniformly output the global pedestrian trajectory. It should be noted that a network server can also be established to upload the global positioning results in real time through the wireless network, and an interactive interface can be designed to facilitate users to obtain the positioning results in time and perform interactive operations.

[0135] In this embodiment, the pedestrian flow global positioning method integrating wireless signals and videos is also described based on a specific application scenario. Figure 7 This is a flowchart of a complex scene multi-target global trajectory positioning that integrates wireless signals and videos, provided by an embodiment of the present invention. In this specific process, a pedestrian global identity matching algorithm that integrates Wi-Fi and vision is obtained in advance; and the current video frame ope_frame, the current frame wireless positioning data set W, the current frame visual positioning data S, the pedestrian detection model yolo, and the scene-based coordinate transformation model Z are obtained. After processing by the above method, the detection box array fus_box and the current frame fusion data V after the current matching wireless information are output. Specifically, the identity is determined by comparing the Euclidean distance between the wireless data and the visual data, thereby achieving identity matching and positioning, such as Figure 7 As shown, the implementation steps are as follows:

[0136] The first step is to use the pedestrian detection model to detect pedestrians in the current video frame to obtain the detection frame structure array ope_box without Wi-Fi information: ope_box = yolo (ope_frame), and obtain the current pedestrian pixel coordinates (u i ,v i ).

[0137] The second step is to obtain the current frame visual positioning data S through the scene-based coordinate transformation model: S = Z (u i ,v i ).

[0138] Step 3: Loop through and compare the Euclidean distance Euclidean[i] between the current frame's visual positioning data S and the current frame's wireless positioning data set W: Among them, S.x and S.y represent the abscissa and ordinate located through the visual positioning data S, and W[i].x and W[i].y represent the abscissa and ordinate located through the current frame's wireless positioning data set W.

[0139] Step 4: Find the current nearest wireless data W[j]: W[j] = min(Euclidean[i]). If W[j] < Y_dist, it is determined that the wireless data W[j] and the current frame's visual positioning data S belong to the same person. Among them, Y_dist is a pre-set distance judgment threshold, which can be set and adjusted according to actual needs.

[0140] Step 5: Update the wireless data of the current detection box: ope_box.MAC = W × ope_box; fus_box = (ope_box.S, ope_box.MAC); where ope_box is an array of detection boxes containing the detection box position and visual identity information obtained through the yolo detection model, fus_box is an array of detection boxes after matching wireless information, including ope_box.S (detection box position coordinates), ope_box.MAC (wireless identity coordinates matched with the detection box position coordinates), fus_box is obtained by replacing the visual identity with the wireless identity, and W × ope_box is the replacement process.

[0141] Step 6: Return the array of the current detection box structure fus_box after matching.

[0142] Step 7: Return the current frame's fusion data V = (x i , y i , ID_w i ), where (x i , y i ) represents the coordinates of the target object after fusion positioning, and ID_w i represents the global identity ID of the target object after fusion positioning.

[0143] It should be noted that after continuous matching, the continuous pedestrian identities under a single camera can be obtained. However, due to the Wi-Fi positioning error, a small number of false matches will occur. Therefore, the Wi-Fi identity information that appears most frequently in a visual trajectory can be used as the pedestrian identity information of the trajectory. Using this as an index, the global pedestrian trajectory can be unified. At the same time, the positioning results and the complete trajectory of the pedestrian can interact online with various online platforms through wireless transmission to provide location data with unique identities. In this way, through the above method, visual positioning can be used to track pedestrians, and the visual ranging results can be continuously used to correct the parameters of the wireless adaptive propagation model. When the visual target identity cannot be matched, the wireless data is used to unify the pedestrian identity information and trace back the global trajectory of the pedestrian.

[0144] As can be seen from the above, the pedestrian flow global positioning method based on the fusion of wireless signals and videos in this embodiment can integrate the visual information of the camera network and the global identity information of pedestrians perceived by wireless signals, and realize the global unique spatial three-dimensional trajectory positioning of multiple pedestrian targets in large and complex scenes. Specifically, a pixel-world coordinate conversion model based on a multi-camera network is first designed for large public spaces, and the pedestrian detection position is automatically converted from screen pixel coordinates to precise spatial three-dimensional coordinates. Then, a self-calibrated Wi-Fi signal adaptive propagation model is established, and the spatial distance between pedestrians and wireless signal access points and the corresponding coordinates of pedestrians are calculated by the nearest neighbor trilateral positioning method to obtain the global identity ID of pedestrians, thereby providing ID and distance constraint information for global three-dimensional spatial positioning. At the same time, the subsequent global precise positioning results will be fed back to the signal propagation model for parameter iterative calibration and updating, and the ranging accuracy will be continuously improved. Finally, a global identity matching algorithm for multiple pedestrian targets that integrates vision and wireless signals is constructed, and the global spatial three-dimensional coordinates of multiple pedestrian targets in large public spaces are calculated in combination with visual position information, so as to realize the global three-dimensional trajectory positioning of large-scale crowds. The positioning results can be uploaded to the intelligent public space safety monitoring platform to serve the safety operation and maintenance of public scenes.

[0145] Exemplary Devices

[0146] like Figure 8 As shown in, corresponding to the above-mentioned pedestrian flow global positioning method integrating wireless signals and videos, an embodiment of the present invention further provides a pedestrian flow global positioning system integrating wireless signals and videos, and the above-mentioned pedestrian flow global positioning system integrating wireless signals and videos includes:

[0147] The video positioning information acquisition module 410 is used to acquire video data and acquire video positioning information of a target object based on the video data.

[0148] The video data is a video containing a target object (i.e., a pedestrian) obtained by a camera or other imaging device. The video positioning information is the position information corresponding to the target object obtained by detecting and identifying pedestrians in the video data, and may include the coordinates and identity information of the target object.

[0149] The wireless signal positioning information acquisition module 420 is used to acquire wireless signal data, and acquire wireless signal positioning information of the target object based on the wireless signal data.

[0150] The wireless signal data is data corresponding to the wireless signal in the area where the target object is located (ie, the target area). In this embodiment, the wireless signal is a Wi-Fi signal.

[0151] Specifically, multiple wireless signal transmitting terminals can be pre-set in the above-mentioned target area, and the smart terminal carried by the target object (such as a mobile phone, a tablet, a smart watch, etc.) can be used as a wireless signal receiving terminal. According to the strength of the wireless signal received by the smart terminal of the target object, the distance between the target object and each wireless signal transmitting terminal can be obtained, thereby realizing the positioning of the target object. At the same time, when the smart terminal of the target object is connected to the wireless signal transmitting terminal, the corresponding smart terminal ID is unchanged, and the global identity information of the target object can be obtained based on this. Specifically, wireless data has the property of unchanged ID, that is, as long as the vision of each camera is matched with the wireless ID, the global ID of the pedestrian can be obtained. Therefore, in this embodiment, the above-mentioned wireless signal positioning information can include the wireless identity information and distance information (or wireless positioning coordinates) of the target object.

[0152] The positioning module 430 is used to locate the target object based on the video positioning information and the wireless signal positioning information.

[0153] Specifically, in this embodiment, after obtaining the above-mentioned video positioning information and the above-mentioned wireless signal positioning information, the two are combined to locate the target object. It should be noted that when positioning, the video positioning information and the wireless signal positioning information at the same moment are combined to locate the same target object at that moment. In this way, even if the target object is blocked, its position can be obtained through the wireless positioning information. And in the process of continuous positioning, the global identity information of a target object (i.e., identity information that will not change) can be obtained by combining the two to achieve continuous tracking of the target object.

[0154] In this way, based on the above system, the target object can be located by combining the video positioning information and the wireless signal positioning information. When the target object is blocked in the video, the blocked target object can also be located by the wireless signal data to improve the accuracy of positioning. At the same time, the identity of the target object identified based on the wireless signal data remains unchanged globally, so the change of the target object ID due to occlusion can be avoided, which is conducive to the global positioning and tracking of the target object.

[0155] Based on the above embodiments, the present invention further provides an intelligent terminal, whose principle block diagram can be shown as follows: Fig. 9 As shown. The above-mentioned intelligent terminal includes a processor, a memory, a network interface and a display screen connected through a system bus. Among them, the processor of the intelligent terminal is used to provide computing and control capabilities. The memory of the intelligent terminal includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a global positioning program for pedestrian flows that integrates wireless signals and videos. The internal memory provides an environment for the operation of the operating system and the global positioning program for pedestrian flows that integrate wireless signals and videos in the non-volatile storage medium. The network interface of the intelligent terminal is used to communicate with an external terminal through a network connection. When the global positioning program for pedestrian flows that integrates wireless signals and videos is executed by the processor, the steps of any one of the above-mentioned methods for global positioning of pedestrian flows that integrate wireless signals and videos are implemented. The display screen of the intelligent terminal can be a liquid crystal display screen or an electronic ink display screen.

[0156] Those skilled in the art will understand that Fig. 9 The principle block diagram shown in the figure is only a block diagram of a partial structure related to the scheme of the present invention, and does not constitute a limitation on the smart terminal to which the scheme of the present invention is applied. The specific smart terminal may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0157] In one embodiment, a smart terminal is provided. The smart terminal includes a memory, a processor, and a pedestrian flow global positioning program that integrates wireless signals and videos and is stored in the memory and can be run on the processor. When the pedestrian flow global positioning program that integrates wireless signals and videos is executed by the processor, the following operation instructions are performed:

[0158] Acquire video data, and acquire video positioning information of a target object based on the video data;

[0159] Acquire wireless signal data, and acquire wireless signal positioning information of the target object based on the wireless signal data;

[0160] The target object is located based on the video positioning information and the wireless signal positioning information.

[0161] An embodiment of the present invention also provides a computer-readable storage medium, on which is stored a global positioning program for pedestrian flows that integrates wireless signals and videos. When the global positioning program for pedestrian flows that integrates wireless signals and videos is executed by a processor, the steps of any one of the methods for global positioning of pedestrian flows that integrates wireless signals and videos provided in an embodiment of the present invention are implemented.

[0162] It should be understood that the serial numbers of the steps in the above embodiments do not imply a sequence of execution, and the execution sequence of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0163] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the above-mentioned system can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0164] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0165] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0166] In the embodiments provided by the present invention, it should be understood that the disclosed system / terminal device and method can be implemented in other ways. For example, the system / terminal device embodiments described above are only illustrative, for example, the division of the above modules or units is only a logical function division, and in actual implementation, other division methods can be used, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0167] If the above-mentioned integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The above-mentioned computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the above-mentioned computer program includes computer program code, and the above-mentioned computer program code can be in source code form, object code form, executable file or some intermediate form. The above-mentioned computer-readable medium may include: any entity or device capable of carrying the above-mentioned computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the above-mentioned computer-readable storage medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction.

[0168] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, it should be understood by those skilled in the art that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features therein may be replaced by equivalents. However, these modifications or replacements do not deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention.

Claims

1. A global positioning method for pedestrian flow integrating wireless signals and videos, It is characterized in that The method comprises: Acquire video data, and acquire video positioning information of a target object based on the video data; Acquire wireless signal data, and acquire wireless signal positioning information of a target object based on the wireless signal data; Positioning the target object based on the video positioning information and the wireless signal positioning information; The acquiring of video data, and acquiring video positioning information of the target object based on the video data, includes: Acquire video data of the target area based on a preset camera device; Detect and track the target object in the video data based on a preset target tracking algorithm to obtain pixel coordinates corresponding to the target object; The pixel coordinates are transformed based on a preset coordinate transformation model to obtain video positioning information of the target object, wherein the video positioning information includes the coordinates of the target object in a world coordinate system; The acquiring of wireless signal data, and acquiring wireless signal positioning information of the target object based on the wireless signal data, includes: Obtaining a preset wireless signal adaptive propagation model, wireless signals in a target area, and the number of the wireless signals, wherein the wireless signal adaptive propagation model is used to obtain the distance between the target object and the wireless signal transmitting end according to the wireless signal strength corresponding to the target object; When the number of the wireless signals is less than 3, based on the wireless signal adaptive propagation model, the wireless identity information and distance information of the target object are obtained according to the wireless signal, and used as the wireless signal positioning information, wherein the distance information is the distance between the target object and the wireless signal transmitting end; When the number of the wireless signals is not less than 3, based on the wireless signal adaptive propagation model, the wireless identity information and the wireless positioning coordinates of the target object are obtained according to the wireless signals, and used as the wireless signal positioning information; The step of acquiring the wireless identity information and the wireless positioning coordinates of the target object according to the wireless signal based on the wireless signal adaptive propagation model includes: Acquire a wireless signal connection ID of the target object based on the wireless signal and use it as wireless identity information of the target object; Based on the wireless signal adaptive propagation model, three target wireless signal transmitting ends are obtained according to the signal strength of each wireless signal at the target object, wherein the target wireless signal transmitting ends are the three wireless signal transmitting ends closest to the target object; Based on the distance between each of the target wireless signal transmitting terminals and the target object, the wireless positioning coordinates of the target object are obtained.

2. According to the method for global positioning of pedestrian flow by integrating wireless signals and videos as described in claim 1, It is characterized in that The step of converting the pixel coordinates based on a preset coordinate conversion model to obtain the video positioning information of the target object includes: Performing conversion between a pixel coordinate system and an image plane coordinate system based on the coordinate conversion model to obtain image plane coordinates corresponding to the pixel coordinates; Based on the coordinate transformation model, a transformation is performed between the image plane coordinate system and the camera coordinate system to obtain the camera visual three-dimensional coordinates corresponding to the image plane coordinates, wherein the camera visual three-dimensional coordinates are coordinates in the camera coordinate system; The camera coordinate system and the world coordinate system are converted based on the coordinate conversion model to obtain the world coordinates of the target object corresponding to the camera visual three-dimensional coordinates.

3. According to claim 1, the pedestrian flow global positioning method integrating wireless signals and videos, It is characterized in that The wireless signal adaptive propagation model is iteratively trained in advance based on the following steps: Perform synchronous visual acquisition and wireless signal acquisition on the training objects moving in the target area in sequence to obtain multiple sets of visual distances and wireless signal strengths; The wireless signal adaptive propagation model is iteratively trained based on each group of the visual distance and the wireless signal strength in turn, and the target parameters of the wireless signal adaptive propagation model are updated, wherein the target parameters include an environmental attenuation factor.

4. The method for global positioning of pedestrian flow by integrating wireless signals and videos according to claim 1, It is characterized in that The positioning of the target object based on the video positioning information and the wireless signal positioning information includes: Matching the video positioning information and the wireless signal positioning information based on an identity matching algorithm to obtain an identity correspondence relationship; Based on the identity correspondence, the target object with the same identity in the video positioning information and the wireless signal positioning information is positioned to obtain a positioning result.

5. A pedestrian flow global positioning system integrating wireless signals and videos, used to implement the pedestrian flow global positioning method integrating wireless signals and videos as claimed in any one of claims 1 to 4, It is characterized in that The system comprises: A video positioning information acquisition module, used to acquire video data, and acquire video positioning information of a target object based on the video data; A wireless signal positioning information acquisition module, used to acquire wireless signal data, and acquire wireless signal positioning information of a target object based on the wireless signal data; A positioning module is used to locate the target object based on the video positioning information and the wireless signal positioning information.

6. A smart terminal, It is characterized in that The intelligent terminal includes a memory, a processor, and a global positioning program for pedestrian flows that integrates wireless signals and videos, which is stored in the memory and can be run on the processor. When the global positioning program for pedestrian flows that integrates wireless signals and videos is executed by the processor, the steps of the global positioning method for pedestrian flows that integrates wireless signals and videos are implemented as described in any one of claims 1 to 4.

7. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores a pedestrian flow global positioning program that integrates wireless signals and videos. When the pedestrian flow global positioning program that integrates wireless signals and videos is executed by a processor, the steps of the pedestrian flow global positioning method that integrates wireless signals and videos as described in any one of claims 1 to 4 are implemented.