A positioning method, a positioning device, an electronic device, and a storage medium

Abstract

The application provides a positioning method, a positioning device, electronic equipment and a storage medium, and relates to the technical field of positioning. The method comprises the following steps: receiving first monitoring data sent by a plurality of visual devices and second monitoring data sent by a plurality of base station devices; determining a first time difference of arrival set according to a first motion trail of a first object, position information of a first base station device and position information of a second base station device; in the case that the similarity between the change trend corresponding to the first time difference of arrival set and the change trend corresponding to a second time difference of arrival set determined based on the second monitoring data is greater than or equal to a preset threshold, associating the first object with a second object, and the second time difference of arrival set is a set of TDOA parameter values of the second object with respect to the first base station device and the second base station device; and positioning the first object according to a second motion trail of the second object. The application improves the positioning accuracy.

Description

Technical Field

[0001] This invention relates to the field of positioning technology, and in particular to a positioning method, positioning device, electronic device, and storage medium. Background Technology

[0002] With the promotion and deployment of 5G technology and its corresponding infrastructure, the field of indoor and outdoor high-precision positioning and navigation has developed rapidly. Related technologies utilize visual devices and 5G positioning placed on the same platform to achieve positioning fusion, essentially perceiving the platform's own location. However, in passive perception scenarios on the network side based on edge computing, visual devices perceive the location information of people or objects within the monitoring area, while 5G base stations perceive the location information of 5G terminals within their signal coverage area. Therefore, visual device positioning technology outputs the positioning trajectories of several people or objects within the monitoring range of the visual device, while 5G positioning technology outputs the positioning trajectories of several 5G terminals within the signal coverage area of ​​the 5G base station.

[0003] How to associate the identity of people or objects within the monitoring range of visual equipment with 5G terminals within the signal coverage range of 5G base stations is an urgent problem to be solved in the process of improving positioning accuracy. Summary of the Invention

[0004] This invention provides a positioning method, positioning device, electronic device, and storage medium to associate the identity of people or objects within the monitoring range of a visual device with that of 5G terminals within the signal coverage range of a 5G base station, thereby solving the problem of low positioning accuracy in the prior art.

[0005] In a first aspect, embodiments of the present invention provide a positioning method for locating a first object in a target area, wherein the target area is equipped with multiple visual devices and multiple base station devices, and the signal coverage area of ​​the multiple base station devices includes the monitoring coverage area of ​​the multiple visual devices, the method comprising:

[0006] Receive first monitoring data sent by the plurality of vision devices and second monitoring data sent by the plurality of base station devices;

[0007] Based on the first motion trajectory of the first object, the location information of the first base station device, and the location information of the second base station device, a first time difference of arrival set is determined. The first time difference of arrival set is a set of trajectory points of the first motion trajectory with respect to the time difference of arrival (TDOA) parameter values ​​of the first base station device and the second base station device. The first motion trajectory is a motion trajectory generated based on the first monitoring data. The first base station device and the second base station device are any two base station devices among the plurality of base station devices.

[0008] If the similarity between the trend corresponding to the first time difference of arrival set and the trend corresponding to the second time difference of arrival set determined based on the second monitoring data is greater than or equal to a preset threshold, the first object and the second object are associated. The second time difference of arrival set is the set of TDOA parameter values ​​of the second object with respect to the first base station device and the second base station device. The second object is the terminal device carried by the first object that communicates with the multiple base station devices.

[0009] The first object is located based on the second motion trajectory of the second object.

[0010] Optionally, before locating the first object based on the second motion trajectory of the second object, the method further includes:

[0011] In the first time difference of arrival set, the first TDOA parameter value of the first object with respect to the first base station device and the second base station device is obtained at the first moment, where the first moment is any moment when the first object moves on the first motion trajectory;

[0012] In the second time difference of arrival set, obtain the second TDOA parameter value of the second object with respect to the first base station device and the second base station device at the first time;

[0013] Based on the first TDOA parameter value and the second TDOA parameter value, the time synchronization error between the first base station device and the second base station device is determined;

[0014] The second motion trajectory of the second object is generated based on the third arrival time difference set, wherein the third arrival time difference set is obtained by correcting the second arrival time difference set based on the time synchronization error.

[0015] Optionally, locating the first object based on the second motion trajectory of the second object includes:

[0016] The first motion trajectory is spliced ​​together with the second motion trajectory to generate the third motion trajectory of the first object;

[0017] The first object is located based on the third motion trajectory.

[0018] Optionally, determining the first arrival time difference set based on the first motion trajectory of the first object, the location information of the first base station device, and the location information of the second base station device includes:

[0019] Based on the first motion trajectory of the first object, calculate the position coordinates of the first object at multiple times, determine the position coordinates of the first base station device based on the position information of the first base station device, and determine the position coordinates of the second base station device based on the position information of the second base station device.

[0020] Based on the first distance between the first coordinate and the second coordinate and the second distance between the first coordinate and the third coordinate, the ratio of the difference between the first distance and the second distance to the speed of light is calculated to determine the first arrival time difference set, wherein the first coordinate is the position coordinate of the first object at each moment, the second coordinate is the position coordinate of the first base station device, and the third coordinate is the position coordinate of the second base station device.

[0021] Optionally, associating the identity of the first object with that of the second object when the similarity between the changing trend corresponding to the first set of arrival time differences and the changing trend corresponding to the second set of arrival time differences determined based on the second monitoring data is greater than or equal to a preset threshold includes:

[0022] A first function is constructed based on the first set of arrival time differences, and a second function is constructed based on the second set of arrival time differences;

[0023] Calculate the similarity between the derivative of the first function and the derivative of the second function;

[0024] If the similarity is greater than or equal to a preset threshold, the first object is associated with the identity of the second object.

[0025] Optionally, the similarity between the derivatives of the first function and the second function is calculated according to the following formula:

[0026]

[0027] Where A represents the first object, a represents the second object, s(A,a) is the similarity between the derivatives of the first function and the second function, g'(t) is the derivative of the first function, z'(t) is the derivative of the second function, t0≤t≤t1, t is the movement time of the first and second objects, t0 is the moment when the first and second objects start moving, and t1 is the moment when the first and second objects stop moving.

[0028] Secondly, embodiments of the present invention provide a positioning device for locating a first object in a target area, wherein the target area is equipped with multiple visual devices and multiple base station devices, and the signal coverage area of ​​the multiple base station devices includes the monitoring coverage area of ​​the multiple visual devices. The device includes:

[0029] A receiving module is used to receive first monitoring data sent by the plurality of vision devices and second monitoring data sent by the plurality of base station devices;

[0030] The first determining module is used to determine a first time difference of arrival set based on the first motion trajectory of the first object, the location information of the first base station device, and the location information of the second base station device. The first time difference of arrival set is a set of trajectory points of the first motion trajectory with respect to the time difference of arrival (TDOA) parameter values ​​of the first base station device and the second base station device. The first motion trajectory is a motion trajectory generated based on the first monitoring data. The first base station device and the second base station device are any two base station devices among the plurality of base station devices.

[0031] The association module is used to associate the identity of the first object with that of the second object when the similarity between the change trend corresponding to the first time difference set and the change trend corresponding to the second time difference set determined based on the second monitoring data is greater than or equal to a preset threshold. The second time difference set is a set of TDOA parameter values ​​of the second object with respect to the first base station device and the second base station device. The second object is a terminal device carried by the first object that communicates with the multiple base station devices.

[0032] The positioning module is used to locate the first object based on the second motion trajectory of the second object.

[0033] Optionally, the device further includes:

[0034] The first acquisition module is used to acquire, from the first arrival time difference set, the first TDOA parameter value of the first object with respect to the first base station device and the second base station device at a first moment, wherein the first moment is any moment when the first object moves on the first motion trajectory;

[0035] The second acquisition module is used to acquire, from the second time difference of arrival set, the second TDOA parameter value of the second object with respect to the first base station device and the second base station device at the first time.

[0036] The second determining module is used to determine the time synchronization error between the first base station device and the second base station device based on the first TDOA parameter value and the second TDOA parameter value.

[0037] The correction module is used to generate a second motion trajectory of the second object based on a third set of arrival time differences, wherein the third set of arrival time differences is obtained by correcting the second set of arrival time differences based on the time synchronization error.

[0038] Thirdly, embodiments of the present invention provide an electronic device, comprising:

[0039] At least one processor; and

[0040] A memory communicatively connected to the at least one processor; wherein,

[0041] The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method as described in the first aspect.

[0042] Fourthly, embodiments of the present invention provide a non-transitory computer-readable storage medium storing computer instructions for causing the computer to perform the method as described in the first aspect.

[0043] In this embodiment of the invention, based on the received first and second monitoring data, a first arrival time difference set for the first object and a second arrival time difference set for the second object are determined, respectively. Based on the similarity between the changing trends corresponding to the first and second arrival time difference sets, the first and second objects are associated to achieve identity unification. This simplifies the steps of establishing the association, eliminates the need for large-scale upgrades to related base station equipment, and still improves the positioning capabilities of the base station equipment. Then, based on the first and second motion trajectories after identity association, a third motion trajectory unifying the identities of the first and second objects is generated. Thus, within the target area, the second motion trajectory can compensate for the first object's trajectory in the visual blind spots of the visual equipment, improving the completeness of the generated third motion trajectory and thereby enhancing the accuracy of the first object's positioning. Attached Figure Description

[0044] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0045] Figure 1 This is a flowchart of the positioning method provided in an embodiment of the present invention;

[0046] Figure 2 This is a schematic diagram of the structure of an edge computing node provided in an embodiment of the present invention;

[0047] Figure 3 This is a schematic diagram of the positioning device provided in an embodiment of the present invention. Detailed Implementation

[0048] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0049] The terms "first," "second," etc., used in the specification and claims of this invention are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such structures can be interchanged where appropriate so that embodiments of the invention can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, the first object can be one or more.

[0050] See Figure 1 , Figure 1 This is a flowchart of a positioning method provided in an embodiment of the present invention. The method is used to locate objects (e.g., pedestrians, vehicles, smart terminals, etc.) in a target area (which may include indoor and outdoor areas, such as urban blocks, industrial parks, etc.). The target area is equipped with multiple vision devices and multiple base station devices. In this embodiment, any object in the target area (i.e., the first object) is used as an example for illustration. The vision devices may be vision sensors such as cameras, and the base station devices may be 5G base stations.

[0051] Among them, visual devices have many blind spots and their monitoring coverage area is small. Furthermore, as citizens become more aware of privacy protection, they are increasingly resistant to visual monitoring devices such as cameras, making it difficult to deploy visual devices on a large scale in target areas. On the other hand, base station equipment has a large signal coverage area and can be deployed on a large scale, but its positioning accuracy is low and the accuracy of the generated positioning trajectory is poor.

[0052] Therefore, the present invention, by simultaneously setting up multiple vision devices and multiple base station devices in the target area, makes the signal coverage area of ​​the multiple base station devices include the monitoring coverage area of ​​the multiple vision devices, thus making up for the shortcomings of the small coverage area of ​​the vision devices by utilizing the wide signal coverage of the base station devices.

[0053] It should be noted that the number of vision devices and base station devices can be adjusted adaptively according to the actual size of the target area, and no limit is imposed here.

[0054] The method is applied to edge computing nodes and specifically includes the following steps:

[0055] Step 101: Receive first monitoring data sent by the plurality of vision devices and second monitoring data sent by the plurality of base station devices;

[0056] Multiple vision devices send the initial monitoring data they capture to the edge computing nodes. Multiple base station devices measure the uplink UL-SRS signal of terminal devices within their coverage area and generate the UL-TDOA time difference, which is then sent to the 5G core network via the NRPPA protocol. The Location Management Function (LMF) in the 5G core network is centrally located in the 5G positioning architecture; it receives measurement values ​​and auxiliary information reported by the base stations. The Access and Mobility Management Function (AMF) handles the service interface conversion. Information from the LMF measured by the base station devices can be carried using the NRPPA protocol and communicate with the AMF via the control plane interface NG-C. The 5G core network NEF forwards the NRPPA protocol to the edge computing nodes via the N33 interface. In other words, the edge computing nodes can obtain the second monitoring data sent by multiple base station devices from the core network. Figure 2 As shown, the edge computing node includes a vision processing module, an identity association module, and a positioning module. The vision processing module receives first monitoring data sent by a vision device and generates the motion trajectory of a first object based on the first monitoring data. The first monitoring data can be Real-Time Streaming Protocol (RTSP) video stream data. The identity association module receives second monitoring data sent by a base station device. The second monitoring data can be the Transient Data Output Area (TDOA) measurement value of a terminal device within the signal coverage area of ​​the base station device. The positioning module is used to locate objects in the target area.

[0057] In one example, the vision processing module obtains video stream data from the vision device via the RTSP protocol, uses computer vision algorithms to identify and locate objects in the video stream data, and then outputs a timestamp, the object's virtual identity document (ID), and the object's motion trajectory to the identity association module. The motion trajectory can be represented in the form of coordinate data.

[0058] In one example, an ID corresponding to the first object can be generated based on its feature information (e.g., height, facial features, etc.), and the first motion trajectory can be identified by the ID. It should be understood that the motion trajectories of other objects in the target area can also have corresponding IDs.

[0059] Step 102: Based on the first motion trajectory of the first object, the location information of the first base station device, and the location information of the second base station device, determine the first arrival time difference set. The first arrival time difference set is the set of trajectory points of the first motion trajectory with respect to the arrival time difference (TDOA) parameter values ​​of the first base station device and the second base station device. The first motion trajectory is a motion trajectory generated based on the first monitoring data. The first base station device and the second base station device are any two base station devices among the plurality of base station devices.

[0060] The edge computing node can generate a first motion trajectory of the first object in the spatial coordinate system of the target area based on the location information of the first object at different time points. The spatial coordinate system of the target area can be a coordinate system constructed according to the mapping relationship of the target area. Therefore, during the movement of the first object, different trajectory points correspond to different time points on the first motion trajectory. Based on the location information of each trajectory point, the location information of the first base station device, and the location information of the second base station device, a first arrival time difference set (i.e., the set of arrival time difference TDOA parameter values ​​of the trajectory points of the first motion trajectory with respect to the first base station device and the second base station device) can be determined.

[0061] In some optional implementations, step 102, determining the first arrival time difference set based on the first motion trajectory of the first object, the location information of the first base station device, and the location information of the second base station device, includes:

[0062] Based on the first motion trajectory of the first object, calculate the position coordinates of the first object at multiple times, determine the position coordinates of the first base station device based on the position information of the first base station device, and determine the position coordinates of the second base station device based on the position information of the second base station device.

[0063] Based on the first distance between the first coordinate and the second coordinate and the second distance between the first coordinate and the third coordinate, the ratio of the difference between the first distance and the second distance to the speed of light is calculated to determine the first arrival time difference set, wherein the first coordinate is the position coordinate of the first object at each moment, the second coordinate is the position coordinate of the first base station device, and the third coordinate is the position coordinate of the second base station device.

[0064] In this embodiment, both the first base station device and the second base station device are located in the target area. Therefore, in the coordinate system constructed based on the mapping relationship of the target area, the position information of the first base station device and the second base station device is known, and different trajectory points correspond to different time points on the first motion trajectory. The position coordinates of each trajectory point correspond to the position coordinates of the first object at each moment, which can be denoted as the first coordinate (x...). t yt ), where t0≤t≤t1, t is the movement time of the first object and the second object, t0 is the moment when the first object and the second object start moving, and t1 is the moment when the first object and the second object end moving; the position coordinates of the first base station device can be determined based on the position information of the first base station device, and can be denoted as the second coordinate (x2, y2); the position coordinates of the second base station can be determined based on the position information of the second base station device, and can be denoted as the third coordinate (x3, y3).

[0065] Taking a time t equal to 1 as an example, the first coordinate is (x1, y1). Using the distance formula between two points, we can calculate the first distance r1 between the first coordinate (x1, y1) and the second coordinate (x2, y2), and the second distance r2 between the first coordinate (x1, y1) and the third coordinate (x3, y3). Then, by calculating the ratio of the difference between the first distance r1 and the second distance r2 to the speed of light, we can determine the first arrival time difference of the first object when t equals 1. Similarly, when t equals other values, we can determine the first arrival time difference of the first object at the corresponding time, thus obtaining the set of first arrival time differences of the first object throughout its entire motion.

[0066] Thus, after obtaining the first arrival time difference set of the first object, step 103 is used to compare the relationship between the first arrival time difference set and the second arrival time difference set to determine whether to associate the first object with the second object.

[0067] Step 103: If the similarity between the change trend corresponding to the first time difference of arrival set and the change trend corresponding to the second time difference of arrival set determined based on the second monitoring data is greater than or equal to a preset threshold, the first object and the second object are associated. The second time difference of arrival set is the set of TDOA parameter values ​​of the second object with respect to the first base station device and the second base station device. The second object is the terminal device carried by the first object that communicates with the multiple base station devices.

[0068] The second object can be a terminal device carried by the first object that communicates with multiple base station devices. In other words, a pedestrian can carry a mobile phone, smart wearable device, or other terminal device with wireless signal transmission capabilities, and the base station devices can acquire the wireless signals transmitted by the mobile phone, smart wearable device, or other terminal. In this way, the edge computing node can locate the first object and the second object separately. After associating the identities of the first object and the second object, the first object can be further located in step 103 based on the second object's second movement trajectory.

[0069] In this example, the second monitoring data may include the TDOA measurement values ​​of the second object with respect to the first base station equipment and the second base station equipment. The second time difference of arrival set determined based on the second monitoring data can be denoted as z(t), t∈[t0, t1]. According to step 102 above, the TDOA calculation values ​​of the first object with respect to the first base station equipment and the second base station equipment can be determined, and the first time difference of arrival set can be denoted as g(t), t∈[t0, t1].

[0070] The second set of arrival time differences determined based on the second monitoring data can be seen in the following formula:

[0071] z(t) = r2(t) - r1(t) + w*c + e;

[0072] Where r1(t)=sqrt((x t -x1) 2 +(y t -y1) 2 Let r2 be the first distance from the second object to the first base station device, and r2 = sqrt((x) t -x2) 2 +(y t -y2) 2 Let be the distance from the second object to the second base station device, w be the time synchronization error between the first and second base station devices, c be the speed of light, and e be Gaussian white noise. Typically, the time synchronization error w between base stations remains constant over a short period.

[0073] From the second arrival time difference set z(t) and the first arrival time difference set g(t), it can be seen that when the first object and the second object are associated, i.e., the second object is the terminal device carried by the first object, g(t) and z(t) differ only by a constant w*c. Thus, by comparing the similarity between the changing trends corresponding to the first arrival time difference set and the changing trends corresponding to the second arrival time difference set, the identity association between the first object (i.e., the person or object seen visually) and the second object (i.e., the terminal device sensed by the 5G signal) can be completed. This simplifies the steps of establishing the association, eliminates the need for large-scale upgrades to the relevant base station equipment, and still improves the positioning capabilities of the base station equipment.

[0074] Step 104: Locate the first object based on the second motion trajectory of the second object.

[0075] Within the entire target area, visual devices inevitably have blind spots. These blind spots can be filled by the coverage of base station equipment, ensuring that the coverage area of ​​the base station equipment includes the coverage area of ​​the visual equipment. This leverages the wide signal coverage of the base station equipment to compensate for the limited coverage of the visual equipment. After associating the identities of the first and second objects, in scenarios with both visual and signal coverage, a weighted approach can be used to improve positioning accuracy. In scenarios with limited visual coverage but full signal coverage, trajectory stitching can be used to extend the positioning trajectory, thereby improving the accuracy of locating the first object.

[0076] In some optional implementations, step 104, locating the first object based on the second motion trajectory of the second object, includes:

[0077] The first motion trajectory is spliced ​​together with the second motion trajectory to generate the third motion trajectory of the first object;

[0078] The first object is located based on the third motion trajectory.

[0079] In this embodiment, within the target area, when both visual and wireless coverage are present, the first motion trajectory of the first object can be fused and optimized using the second motion trajectory, improving the accuracy of the generated third motion trajectory. When visual and wireless coverage are not present, the second motion trajectory can compensate for the missing portions of the first motion trajectory in the blind spots of the visual equipment, improving the completeness of the generated third motion trajectory. This overcomes the shortcomings of insufficient visual equipment coverage and blind spots in perception, and utilizes the characteristic of base station equipment being insensitive to line-of-sight obstacles to achieve full coverage of positioning perception, thereby improving the accuracy of locating the first object.

[0080] In this embodiment of the invention, based on the received first and second monitoring data, a first arrival time difference set for the first object and a second arrival time difference set for the second object are determined, respectively. Based on the similarity between the changing trends corresponding to the first and second arrival time difference sets, the first and second objects are associated to achieve identity unification. This simplifies the steps of establishing the association, eliminates the need for large-scale upgrades to related base station equipment, and still improves the positioning capabilities of the base station equipment. Then, based on the first and second motion trajectories after identity association, a third motion trajectory unifying the identities of the first and second objects is generated. Thus, within the target area, the second motion trajectory can compensate for the first object's trajectory in the visual blind spots of the visual equipment, improving the completeness of the generated third motion trajectory and thereby enhancing the accuracy of the first object's positioning.

[0081] In this invention, considering the time synchronization error between base station devices, the accuracy of the motion trajectory of the second object (i.e., the terminal device) output in 5G positioning technology is poor due to this error, which affects the positioning accuracy of the first object. Therefore, in this embodiment of the invention, identity association results can be used to achieve time synchronization between base station devices, as detailed below:

[0082] In some optional embodiments, before step 104, locating the first object based on the second object's second motion trajectory, the method further includes:

[0083] In the first time difference of arrival set, the first TDOA parameter value of the first object with respect to the first base station device and the second base station device is obtained at the first moment, where the first moment is any moment when the first object moves on the first motion trajectory;

[0084] In the second time difference of arrival set, obtain the second TDOA parameter value of the second object with respect to the first base station device and the second base station device at the first time;

[0085] Based on the first TDOA parameter value and the second TDOA parameter value, the time synchronization error between the first base station device and the second base station device is determined;

[0086] The second motion trajectory of the second object is generated based on the third arrival time difference set, wherein the third arrival time difference set is obtained by correcting the second arrival time difference set based on the time synchronization error.

[0087] In this embodiment, as Figure 2As shown, the edge computing node also includes a clock synchronization module. This module utilizes the output of the identity association module, taking the first motion trajectory of the first object as the true value, to calculate the time synchronization error contained in the corresponding TDOA. The first TDOA parameter value obtained from the first arrival time difference set is actually the calculated TDOA value of the first object relative to the first and second base station devices, based on the trajectory points of the first motion trajectory and the positions of the first and second base station devices. The second TDOA parameter value obtained from the second arrival time difference set is actually the measured TDOA value of the second object relative to the first and second base station devices. From the second arrival time difference set z(t) and the first arrival time difference set g(t), it can be seen that when the first object and the second object are associated, i.e., when the second object is a terminal device carried by the first object, g(t) and z(t) differ only by a constant w*c. Thus, at any time t, calculating the difference between the first and second TDOA parameter values ​​can determine the time synchronization error w between the first and second base station devices. Typically, the time synchronization error w between base station devices remains constant over a short period.

[0088] The clock synchronization module reports the calculation results to the positioning module. The positioning module can use the time synchronization error w output by the clock synchronization module to correct the TDOA measurement values ​​in the second time difference of arrival set z(t), thus obtaining the third time difference of arrival set. In this way, during the generation of the second motion trajectory of the second object based on the third time difference of arrival set, the time synchronization error between the first and second base stations is reduced, improving the accuracy of the second motion trajectory of the second object (i.e., the terminal device) output by the positioning module, thereby improving the positioning accuracy of the first object. The positioning of the second object can be achieved in real-time using a triangulation algorithm.

[0089] It should be noted that this embodiment uses the first base station device and the second base station device as examples. When there are other base station devices among the multiple base station devices, the base station devices that have not been synchronized in time can also be synchronized in pairs to achieve the same technical effect. To avoid repetition, this will not be elaborated here.

[0090] In some optional embodiments, step 103, associating the identity of the first object with that of the second object when the similarity between the changing trend corresponding to the first set of arrival time differences and the changing trend corresponding to the second set of arrival time differences determined based on the second monitoring data is greater than or equal to a preset threshold, includes:

[0091] A first function is constructed based on the first set of arrival time differences, and a second function is constructed based on the second set of arrival time differences;

[0092] Calculate the similarity between the derivative of the first function and the derivative of the second function;

[0093] If the similarity is greater than or equal to a preset threshold, the first object is associated with the identity of the second object.

[0094] In this embodiment, the motion trajectory of the first object generated by the vision processing module based on the first monitoring data sent by the vision device can be expressed as: f(t)=[x t y t ], t∈[t0,t1], where t0≤t≤t1, t is the movement time of the first object and the second object, t0 is the time when the first object and the second object start moving, and t1 is the time when the first object and the second object end moving. Based on the movement trajectory f(t) of the first object and the position coordinates of the first base station device and the second base station device, the first arrival time difference set of the first object with respect to the first base station device and the second base station device can be determined, and the constructed first function can be expressed as g(t). The second arrival time difference set determined based on the second monitoring data can be expressed as: z(t)=r2(t)-r1(t)+w*c+e; where r1(t)=sqrt((x t -x1) 2 +(y t -y1) 2 Let r2 be the first distance from the second object to the first base station device, and r2 = sqrt((x) t -x2) 2 +(y t -y2) 2 Let be the distance from the second object to the second base station device, w be the time synchronization error between the first and second base station devices, c be the speed of light, and e be Gaussian white noise. Therefore, the second function constructed based on the second arrival time difference set can also be expressed as z(t).

[0095] The second function z(t) constructed from the second set of arrival time differences, and the first function g(t) constructed from the first set of arrival time differences, show that when the first object and the second object are associated, i.e., the second object is the terminal device carried by the first object, g(t) and z(t) differ only by a constant w*c. Thus, by comparing the similarity between the changing trends corresponding to the first set of arrival time differences and the changing trends corresponding to the second set of arrival time differences, the identity association between the first object (i.e., the person or object seen visually) and the second object (i.e., the terminal device sensed by the 5G signal) can be completed.

[0096] Specifically, calculating the derivative g'(t) of the first function determines the trend corresponding to the first set of arrival time differences, and calculating the derivative z'(t) of the second function determines the trend corresponding to the second set of arrival time differences. By calculating the similarity between the derivatives of the first and second functions, the impact of the time synchronization error w is offset, simplifying the steps of establishing the correlation and eliminating the need for large-scale upgrades to the relevant base station equipment.

[0097] Then, using the output of the identity association module, and with the first object and the second object linked by identity, the time synchronization error w contained in the corresponding second arrival time difference set z(t) is calculated using the first object's first motion trajectory as the truth value. Based on the calculated time synchronization error w, the TDOA measurement values ​​in the second arrival time difference set z(t) are corrected to obtain the third arrival time difference set. Thus, in the process of generating the second motion trajectory of the second object based on the third arrival time difference set, the time synchronization error between the first and second base stations is reduced, improving the accuracy of the second motion trajectory of the second object (i.e., the terminal device) output by the positioning module, thereby improving the positioning accuracy of the first object.

[0098] The similarity between the derivatives of the first function and the derivatives of the second function is calculated according to the following formula:

[0099]

[0100] Where A represents the first object, a represents the second object, s(A,a) is the similarity between the derivatives of the first function and the second function, g'(t) is the derivative of the first function, z'(t) is the derivative of the second function, t0≤t≤t1, t is the movement time of the first and second objects, t0 is the moment when the first and second objects start moving, and t1 is the moment when the first and second objects stop moving.

[0101] In this embodiment, based on the received first monitoring data and second monitoring data, a first arrival time difference set for the first object and a second arrival time difference set for the second object are determined, respectively.

[0102] The similarity between the derivative of a first function corresponding to a first set of arrival time differences (TDOAs) and the derivative of a second function corresponding to a second set of arrival time differences (TDOAs) is calculated to associate the first and second objects, thus unifying their identities and simplifying the steps of establishing the association. Specifically, multiple TDOA measurement values ​​for the second objects can be obtained through base station equipment, allowing for the calculation of the similarity between the second and first objects separately. Among multiple similarities greater than or equal to a preset threshold for the second objects, the second object with the highest similarity is selected as the terminal device carried by the first object, thus establishing the identity association between the first and second objects. This approach improves the positioning capabilities of base station equipment without requiring large-scale upgrades. Furthermore, the association information between the first and second objects, the first motion trajectory, the second motion trajectory, and other information can be stored in a database through the identity association module.

[0103] After associating the identities of the first and second objects, the time synchronization error w in the corresponding second time difference of arrival set z(t) is calculated using the output of the identity association module and the first motion trajectory of the first object as the true value. Based on the calculated time synchronization error w, the TDOA measurement values ​​in the second time difference of arrival set z(t) are corrected to obtain the third time difference of arrival set. In this way, the time synchronization error between the first and second base stations is reduced during the generation of the second motion trajectory of the second object based on the third time difference of arrival set, thus improving the accuracy of the second motion trajectory of the second object (i.e., the terminal device) output by the positioning module.

[0104] Then, based on the first and second motion trajectories, a third motion trajectory is generated that unifies the identities of the first and second objects. In this way, within the target area, the second motion trajectory can compensate for the first object's trajectory in the visual blind spots of the vision device, improving the completeness of the generated third motion trajectory and thus enhancing the accuracy of the first object's localization.

[0105] See Figure 3 , Figure 3 This is a schematic diagram of the positioning device provided in an embodiment of the present invention. The positioning device 300 is used to locate a first object in a target area. The target area is equipped with multiple vision devices and multiple base station devices. The signal coverage area of ​​the multiple base station devices includes the monitoring coverage area of ​​the multiple vision devices. The positioning device 300 includes:

[0106] The receiving module 301 is used to receive first monitoring data sent by the plurality of vision devices and second monitoring data sent by the plurality of base station devices;

[0107] The first determining module 302 is used to determine a first time difference of arrival set based on the first motion trajectory of the first object, the location information of the first base station device, and the location information of the second base station device. The first time difference of arrival set is a set of trajectory points of the first motion trajectory with respect to the time difference of arrival (TDOA) parameter values ​​of the first base station device and the second base station device. The first motion trajectory is a motion trajectory generated based on the first monitoring data. The first base station device and the second base station device are any two base station devices among the plurality of base station devices.

[0108] The association module 303 is used to associate the identity of the first object with that of the second object when the similarity between the change trend corresponding to the first time difference set and the change trend corresponding to the second time difference set determined based on the second monitoring data is greater than or equal to a preset threshold. The second time difference set is a set of TDOA parameter values ​​of the second object with respect to the first base station device and the second base station device. The second object is a terminal device carried by the first object that communicates with the multiple base station devices.

[0109] The positioning module 304 is used to locate the first object according to the second motion trajectory of the second object.

[0110] Optionally, the device further includes:

[0111] The first acquisition module is used to acquire, from the first arrival time difference set, the first TDOA parameter value of the first object with respect to the first base station device and the second base station device at a first moment, wherein the first moment is any moment when the first object moves on the first motion trajectory;

[0112] The second acquisition module is used to acquire, from the second time difference of arrival set, the second TDOA parameter value of the second object with respect to the first base station device and the second base station device at the first time.

[0113] The second determining module is used to determine the time synchronization error between the first base station device and the second base station device based on the first TDOA parameter value and the second TDOA parameter value.

[0114] The correction module is used to generate a second motion trajectory of the second object based on a third set of arrival time differences, wherein the third set of arrival time differences is obtained by correcting the second set of arrival time differences based on the time synchronization error.

[0115] Optionally, the positioning module 304 includes:

[0116] The splicing submodule is used to splice the first motion trajectory according to the second motion trajectory to generate the third motion trajectory of the first object;

[0117] The positioning submodule is used to locate the first object based on the third motion trajectory.

[0118] Optionally, the first determining module 302 includes:

[0119] The first calculation submodule is used to calculate the position coordinates of the first object at multiple times based on the first motion trajectory of the first object, determine the position coordinates of the first base station device based on the position information of the first base station device, and determine the position coordinates of the second base station device based on the position information of the second base station device.

[0120] Based on the first distance between the first coordinate and the second coordinate and the second distance between the first coordinate and the third coordinate, the ratio of the difference between the first distance and the second distance to the speed of light is calculated to determine the first arrival time difference set, wherein the first coordinate is the position coordinate of the first object at each moment, the second coordinate is the position coordinate of the first base station device, and the third coordinate is the position coordinate of the second base station device.

[0121] Optionally, the associated module 303 includes:

[0122] A submodule is constructed to construct a first function based on the first set of arrival time differences and a second function based on the second set of arrival time differences.

[0123] The second calculation submodule is used to calculate the similarity between the derivative of the first function and the derivative of the second function;

[0124] The association submodule is used to associate the identity of the first object with that of the second object when the similarity is greater than or equal to a preset threshold.

[0125] Optionally, the similarity between the derivatives of the first function and the second function is calculated according to the following formula:

[0126]

[0127] Where A represents the first object, a represents the second object, s(A,a) is the similarity between the derivatives of the first function and the second function, g'(t) is the derivative of the first function, z'(t) is the derivative of the second function, t0≤t≤t1, t is the movement time of the first and second objects, t0 is the moment when the first and second objects start moving, and t1 is the moment when the first and second objects stop moving.

[0128] It should be noted that the positioning device 300 is capable of achieving Figure 1 To avoid repetition, the various processes in the method embodiments shown will not be described again here.

[0129] This invention also provides an electronic device, comprising:

[0130] At least one processor; and

[0131] A memory communicatively connected to the at least one processor; wherein,

[0132] The memory stores instructions that can be executed by the at least one processor. These instructions are executed by the at least one processor to enable the at least one processor to perform the various processes as described in the above-described positioning method embodiments, and to achieve the same technical effect. To avoid repetition, these will not be repeated here.

[0133] This invention also provides a non-transitory computer-readable storage medium storing computer instructions. These computer instructions are used to cause the computer to execute various processes as described in the above-described positioning method embodiments, achieving the same technical effects. To avoid repetition, they will not be described again here. The computer-readable storage medium may be, for example, ROM, RAM, a magnetic disk, or an optical disk.

[0134] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of the present invention is not limited to performing functions in the order discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0135] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0136] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of the present invention.

Claims

1. A positioning method, characterized in that, The method is used to locate a first object in a target area, wherein the target area is equipped with multiple vision devices and multiple base station devices, and the signal coverage area of ​​the multiple base station devices includes the monitoring coverage area of ​​the multiple vision devices. The system receives first monitoring data sent by the plurality of vision devices and second monitoring data sent by the plurality of base station devices; the second monitoring data includes TDOA measurement values ​​of a second object with respect to the first base station device and the second base station device. Based on the first motion trajectory of the first object, the location information of the first base station device, and the location information of the second base station device, a first time difference of arrival set is determined. The first time difference of arrival set is a set of trajectory points of the first motion trajectory with respect to the time difference of arrival (TDOA) parameter values ​​of the first base station device and the second base station device. The first motion trajectory is a motion trajectory generated based on the first monitoring data. The first base station device and the second base station device are any two base station devices among the plurality of base station devices. If the similarity between the trend corresponding to the first time difference of arrival set and the trend corresponding to the second time difference of arrival set determined based on the second monitoring data is greater than or equal to a preset threshold, the first object and the second object are associated. The second time difference of arrival set is the set of TDOA parameter values ​​of the second object with respect to the first base station device and the second base station device. The second object is the terminal device carried by the first object that communicates with the multiple base station devices. The first object is located based on the second motion trajectory of the second object.

2. The method according to claim 1, characterized in that, Before locating the first object based on the second motion trajectory of the second object, the method further includes: In the first time difference of arrival set, the first TDOA parameter value of the first object with respect to the first base station device and the second base station device is obtained at the first moment, where the first moment is any moment when the first object moves on the first motion trajectory; In the second time difference of arrival set, obtain the second TDOA parameter value of the second object with respect to the first base station device and the second base station device at the first time; Based on the first TDOA parameter value and the second TDOA parameter value, the time synchronization error between the first base station device and the second base station device is determined; The second motion trajectory of the second object is generated based on the third arrival time difference set, wherein the third arrival time difference set is obtained by correcting the second arrival time difference set based on the time synchronization error.

3. The method according to claim 1 or 2, characterized in that, The step of locating the first object based on the second motion trajectory of the second object includes: The first motion trajectory is spliced ​​together with the second motion trajectory to generate the third motion trajectory of the first object; The first object is located based on the third motion trajectory.

4. The method according to claim 1, characterized in that, The step of determining the first arrival time difference set based on the first motion trajectory of the first object, the location information of the first base station device, and the location information of the second base station device includes: Based on the first motion trajectory of the first object, calculate the position coordinates of the first object at multiple times, determine the position coordinates of the first base station device based on the position information of the first base station device, and determine the position coordinates of the second base station device based on the position information of the second base station device. Based on the first distance between the first coordinate and the second coordinate and the second distance between the first coordinate and the third coordinate, the ratio of the difference between the first distance and the second distance to the speed of light is calculated to determine the first arrival time difference set, wherein the first coordinate is the position coordinate of the first object at each moment, the second coordinate is the position coordinate of the first base station device, and the third coordinate is the position coordinate of the second base station device.

5. The method according to claim 1, characterized in that, Associating the identity of the first object with that of the second object when the similarity between the changing trend corresponding to the first set of arrival time differences and the changing trend corresponding to the second set of arrival time differences determined based on the second monitoring data is greater than or equal to a preset threshold includes: A first function is constructed based on the first set of arrival time differences, and a second function is constructed based on the second set of arrival time differences; Calculate the similarity between the derivative of the first function and the derivative of the second function; If the similarity is greater than or equal to a preset threshold, the first object is associated with the identity of the second object.

6. The method according to claim 5, characterized in that, The similarity between the derivatives of the first function and the derivatives of the second function is calculated according to the following formula: ； Where A represents the first object, a represents the second object, s(A,a) is the similarity between the derivatives of the first function and the second function, g'(t) is the derivative of the first function, z'(t) is the derivative of the second function, t0≤t≤t1, t is the movement time of the first and second objects, t0 is the moment when the first and second objects start moving, and t1 is the moment when the first and second objects stop moving.

7. A positioning device, characterized in that, For locating a first object in a target area, the target area being equipped with multiple vision devices and multiple base station devices, the signal coverage area of ​​the multiple base station devices encompassing the monitoring coverage area of ​​the multiple vision devices, the device comprising: The receiving module is configured to receive first monitoring data sent by the plurality of vision devices and second monitoring data sent by the plurality of base station devices; the second monitoring data includes TDOA measurement values ​​of a second object with respect to the first base station device and the second base station device; The first determining module is used to determine a first time difference of arrival set based on the first motion trajectory of the first object, the location information of the first base station device, and the location information of the second base station device. The first time difference of arrival set is a set of trajectory points of the first motion trajectory with respect to the time difference of arrival (TDOA) parameter values ​​of the first base station device and the second base station device. The first motion trajectory is a motion trajectory generated based on the first monitoring data. The first base station device and the second base station device are any two base station devices among the plurality of base station devices. The association module is used to associate the identity of the first object with that of the second object when the similarity between the change trend corresponding to the first time difference set and the change trend corresponding to the second time difference set determined based on the second monitoring data is greater than or equal to a preset threshold. The second time difference set is a set of TDOA parameter values ​​of the second object with respect to the first base station device and the second base station device. The second object is a terminal device carried by the first object that communicates with the multiple base station devices. The positioning module is used to locate the first object based on the second motion trajectory of the second object.

8. The apparatus according to claim 7, characterized in that, The device further includes: The first acquisition module is used to acquire, from the first arrival time difference set, the first TDOA parameter value of the first object with respect to the first base station device and the second base station device at a first moment, wherein the first moment is any moment when the first object moves on the first motion trajectory; The second acquisition module is used to acquire, from the second time difference of arrival set, the second TDOA parameter value of the second object with respect to the first base station device and the second base station device at the first time. The second determining module is used to determine the time synchronization error between the first base station device and the second base station device based on the first TDOA parameter value and the second TDOA parameter value. The correction module is used to generate a second motion trajectory of the second object based on a third set of arrival time differences, wherein the third set of arrival time differences is obtained by correcting the second set of arrival time differences based on the time synchronization error.

9. An electronic device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method as described in any one of claims 1 to 6.

10. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to perform the method as described in any one of claims 1 to 6.

Images