Indoor positioning method, system, device, equipment, medium and product

By combining communication-perceptual signals and visual-perceptual images in 5G advanced networks, and using fusion positioning technology to switch tracking, the problem of insufficient accuracy in the non-line-of-sight signal occlusion area is solved, and accurate tracking and positioning is achieved throughout the target area .

CN119006770APending Publication Date: 2024-11-22CHINA MOBILE COMM GRP CHONGQING CO LTD +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410985167.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-22
Publication Date
2024-11-22

AI Technical Summary

Technical Problem

Traditional 5G advanced networks find it difficult to accurately locate target objects in non-line-of-sight signal occlusion areas, resulting in the inability to continue to accurately track and locate target objects in some areas.

Method used

By combining communication sensing signals and visual sensing images, fusion positioning technology is adopted to switch to the corresponding tracking positioning method when the target object switches from the communication sensing positioning area to the visual sensing positioning area or vice versa to ensure accurate positioning within the entire target area. Tracking and positioning.

Benefits of technology

By integrating communication perception positioning technology and visual perception positioning technology, the target object can be accurately tracked and positioned within the entire target area, solving the problem of insufficient accuracy in the non-line-of-sight signal occlusion area of ​​traditional technology.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119006770A_ABST
    Figure CN119006770A_ABST
Patent Text Reader

Abstract

The invention discloses an indoor positioning method, system, device and equipment, a medium and a product, which are applied to the technical field of tracking and positioning. And the central unit receives the communication sensing signal and the visual sensing image, determines the moving state of the target object in the target area, and adopts corresponding tracking and positioning modes based on four conditions. When the target object moves from one positioning area to another positioning area, the positioning technology can be switched through the fused positioning technology, so that a better tracking and positioning effect is realized. According to the scheme provided by the embodiment of the invention, the visual perception equipment is arranged in the visual perception positioning area which cannot be accurately positioned by a communication perception positioning technology, and the precision of tracking and positioning the target object in the area is ensured through the visual perception positioning technology. Therefore, through fusion of the communication perception positioning technology and the visual perception positioning technology, it can be ensured that the target object can be accurately tracked and positioned in the whole target area.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application belongs to the field of tracking and positioning technology, and in particular, relates to an indoor positioning method, system, device, equipment, medium and product. Background Art

[0002] With the development of intelligence, people's demand for indoor positioning is constantly driven. Intelligent navigation and orientation, intelligent production and logistics management, intelligent buildings and homes, and industrial equipment positioning all require the support of indoor positioning. At present, there are many indoor positioning technologies, such as ultra-wideband (UWB) positioning, indoor wireless signal fingerprint positioning, etc. With the advancement of mobile communication technology, especially the fifth generation mobile communication technology (5G) and 5G Advance technology development, mobile communication networks have gradually acquired accurate indoor positioning support capabilities. The integrated network of synaesthesia (communication perception) and computing enables mobile communication systems to provide communication capabilities while being able to perceive the location of objects in their coverage area.

[0003] The synaesthesia positioning capability of the 5G advanced network uses the communication signals transmitted by the base station to improve the communication channel model by introducing elements and features related to the perception system, such as radar cross-sectional area and other information, so as to make it suitable for positioning synaesthesia scenarios. The perception positioning capability of the 5G advanced network has high positioning accuracy for objects that are not blocked by the line of sight (Light Of Sight, LOS) signal, but it is difficult to determine objects that are blocked by the non-line of sight (Non-Light Of Sight, NLOS) signal. It can be seen that the traditional synaesthesia positioning is less effective in some areas, resulting in the inability to continue to accurately track and locate the target object after the target moves to these areas. Summary of the invention

[0004] The embodiments of the present application provide an indoor positioning method, system, device, equipment, medium and product, which can accurately track and locate a target object in a target area.

[0005] On the one hand, an embodiment of the present application provides an indoor positioning method, which is applied to a central unit, and the method includes:

[0006] Receive communication perception signals in the communication perception positioning area and visual perception images in the visual perception positioning area; the communication perception signals are collected and sent by the first sending and receiving point; the visual perception images are collected by the visual perception device and forwarded by the client front-end device and the second sending and receiving point; the target area includes the communication perception positioning area and the visual perception positioning area;

[0007] Based on the communication perception signal and the visual perception image, determining the movement state of the target object in the target area; the target object is identified based on the communication perception signal and the visual perception image;

[0008] In a case where the movement state represents that the target object moves in the communication sensing and positioning area, tracking and positioning the target object based on the communication sensing signal;

[0009] In a case where the movement state represents that the target object moves in the visual perception positioning area, tracking and positioning the target object based on the visual perception image;

[0010] When the movement state represents that the target object moves from the communication perception positioning area to the visual perception positioning area, switching from the communication perception signal to the visual perception image to track and locate the target object;

[0011] When the movement state represents that the target object moves from the visual perception positioning area to the communication perception positioning area, the target object is tracked and positioned by switching from the visual perception image to the communication perception signal.

[0012] On the other hand, determining the movement state of the target object in the target area based on the communication perception signal and the visual perception image includes:

[0013] When the target object is located in the communication perception positioning area and any one or any combination of the following conditions is met, it is determined that the target object moves from the communication perception positioning area to the visual perception positioning area:

[0014] The future predicted position of the target object includes a position located in the visual perception positioning area; the future predicted position is determined according to the historical position of the target object;

[0015] Generating outliers in each position obtained by tracking and locating the target object based on the communication sensing signal;

[0016] A new target object is detected at the corresponding boundary of the visual perception positioning area.

[0017] On the other hand, determining the movement state of the target object in the target area based on the communication perception signal and the visual perception image includes:

[0018] When the target object is located in the visual perception positioning area and any one or any combination of the following conditions is met, it is determined that the target object moves from the visual perception positioning area to the communication perception positioning area:

[0019] The future predicted position of the target object includes a position located in the communication sensing positioning area; the future predicted position is determined according to the historical position of the target object;

[0020] The visual perception tracking locates the position of the lost target object;

[0021] A new target object is detected at the corresponding boundary of the communication sensing positioning area.

[0022] On the other hand, the target area is divided into a plurality of grids; each grid corresponds to a grid number;

[0023] The tracking and positioning of the target object includes:

[0024] Determining a grid where a target object is located based on the communication perception signal and / or the visual perception image;

[0025] Determine the grid number corresponding to the grid where the target object is located;

[0026] The position of the target object is determined based on the grid number.

[0027] On the other hand, the communication perception positioning area is the line-of-sight area of ​​the first sending and receiving point, and the visual perception positioning area is the non-line-of-sight area of ​​the first sending and receiving point.

[0028] On the other hand, before receiving the communication perception signal in the communication perception positioning area and the visual perception image in the visual perception positioning area, the method further includes:

[0029] Acquire a plan view of the target area, a type of the first sending and receiving point, and a location of the first sending and receiving point;

[0030] Taking the position of the first sending and receiving point as the center and based on the radiation direction corresponding to the type of the first sending and receiving point, rendering the plan view to determine the areas covered and not covered by the first sending and receiving point;

[0031] An area covered by the first transmitting and receiving point is determined as a line-of-sight area, and an area not covered by the first transmitting and receiving point is determined as a non-line-of-sight area.

[0032] On the other hand, the visual perception image forwarded by the second sending and receiving point includes the recognition result of the target object by the client front-end device;

[0033] Before receiving the communication perception signal in the communication perception positioning area and the visual perception image in the visual perception positioning area, the method further includes:

[0034] Establishing a global image recognition model; establishing a terminal model in each of the client front-end devices;

[0035] Sending the global model parameter vector of the global image recognition model to each of the client front-end devices, so that each of the client front-end devices performs model training on the global model parameter vector according to the terminal model;

[0036] Receiving parameter changes returned by each of the client front-end devices;

[0037] Determining the model parameter adjustment range based on the parameter change amount;

[0038] Updating a global image recognition model based on the global model parameter vector and the model parameter adjustment amplitude;

[0039] The updated global image recognition model is sent to each of the client front-end devices, so that the client front-end devices can recognize the target object in the visual perception image based on the global image recognition model.

[0040] On the other hand, an embodiment of the present application provides an indoor positioning system, including:

[0041] The central unit is connected to the first sending and receiving point and the second sending and receiving point respectively, and is used to receive the communication perception signal in the communication perception positioning area collected and sent by the first sending and receiving point; and receive the visual perception image of the visual perception positioning area collected by the visual perception device and forwarded by the client front-end device and the second sending and receiving point; the target area includes the communication perception positioning area and the visual perception positioning area;

[0042] The first sending and receiving point is used to collect the communication perception signal in the communication perception positioning area;

[0043] The visual perception device is used to collect the visual perception image in the visual perception positioning area;

[0044] The client front-end device is connected to the visual perception device and the second sending and receiving point respectively, and is used to forward the visual perception image collected by the visual perception device to the second sending and receiving point;

[0045] The second sending and receiving point is used to forward the visual perception image sent by the client front-end device to the central unit.

[0046] On the other hand, an embodiment of the present application provides an indoor positioning device, including:

[0047] A receiving module, used to receive the communication perception signal in the communication perception positioning area collected and sent by the first sending and receiving point, and to receive the visual perception image of the visual perception positioning area collected by the visual perception device and forwarded by the client front-end device and the second sending and receiving point; the target area includes the communication perception positioning area and the visual perception positioning area;

[0048] A determination module, configured to determine a movement state of a target object in the target area based on the communication perception signal and the visual perception image; the target object is identified based on the communication perception signal and the visual perception image;

[0049] Positioning module for:

[0050] In a case where the movement state represents that the target object moves in the communication sensing and positioning area, tracking and positioning the target object based on the communication sensing signal;

[0051] In a case where the movement state represents that the target object moves in the visual perception positioning area, tracking and positioning the target object based on the visual perception image;

[0052] When the movement state represents that the target object moves from the communication perception positioning area to the visual perception positioning area, switching from the communication perception signal to the visual perception image to track and locate the target object;

[0053] When the movement state represents that the target object moves from the visual perception positioning area to the communication perception positioning area, the target object is tracked and positioned by switching from the visual perception image to the communication perception signal.

[0054] In another aspect, an embodiment of the present application provides an indoor positioning device, the device comprising: a processor and a memory storing computer program instructions;

[0055] When the processor executes the computer program instructions, the indoor positioning method as described above is implemented.

[0056] On the other hand, an embodiment of the present application provides a computer-readable storage medium, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the indoor positioning method as described above is implemented.

[0057] On the other hand, an embodiment of the present application provides a computer program product. When instructions in the computer program product are executed by a processor of an electronic device, the electronic device executes the indoor positioning method as described above.

[0058] The indoor positioning method of the embodiment of the present application needs to locate the target object in the target area. The target area includes a communication perception positioning area and a visual perception positioning area respectively. The method is applied to the central unit. The central unit receives the communication perception signal and the visual perception image. After identifying the existence of the target object in the target area, the central unit determines the movement state of the target object in the target area based on the communication perception signal and the visual perception image. Finally, the corresponding tracking and positioning methods are adopted based on the four situations respectively. When the target object moves from one positioning area to another positioning area, the tracking and positioning based on the communication perception signal or the tracking and positioning based on the visual perception image can be switched through the fused positioning technology, so as to achieve a better tracking and positioning effect. Due to the scheme proposed in the embodiment of the present application, a visual perception device is set in the visual perception positioning area where the communication perception positioning technology cannot accurately locate, and the accuracy of tracking and positioning the target object in the area is guaranteed by the visual perception positioning technology. It can be seen that by integrating the communication perception positioning technology and the visual perception positioning technology, it can be ensured that the target object can be accurately tracked and positioned in the entire target area. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] In order to more clearly illustrate the technical solution of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0060] Figure 1 A schematic diagram of a flow chart of an indoor positioning method provided by an embodiment of the present application is shown;

[0061] Figure 2 A schematic diagram of the architecture of an adaptive bandwidth indoor positioning system provided by an embodiment of the present application is shown;

[0062] Figure 3 A flowchart of the overall solution of fusion positioning provided by an embodiment of the present application is shown;

[0063] Figure 4 A schematic diagram of a system architecture for model training provided by an embodiment of the present application is shown;

[0064] Figure 5 A schematic diagram of the structure of an indoor positioning device provided in an embodiment of the present application is shown;

[0065] Figure 6 A schematic diagram of the hardware structure of the indoor positioning device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0066] The features and exemplary embodiments of various aspects of the present application will be described in detail below. In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain the present application, rather than to limit the present application. For those skilled in the art, the present application can be implemented without the need for some of these specific details. The following description of the embodiments is only to provide a better understanding of the present application by illustrating the examples of the present application.

[0067] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "include..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0068] In order to solve the problems of the traditional solution, the embodiments of the present application provide an indoor positioning method, system, device, equipment, medium and product. The indoor positioning method provided by the embodiments of the present application is first introduced below, and the method is applied to the central unit. Figure 1 FIG. 1 is a flow chart of an indoor positioning method provided by an embodiment of the present application; Figure 1 As shown, the method includes the following steps: S101, S102, S103, S104, S105 and S106.

[0069] S101: receiving a communication perception signal in a communication perception positioning area and a visual perception image in a visual perception positioning area.

[0070] Figure 2 FIG. 1 shows a schematic diagram of the architecture of an adaptive bandwidth indoor positioning system provided by an embodiment of the present application; Figure 2 As shown, it includes a central unit 201, a first sending and receiving point 202, a second sending and receiving point 203, a client front-end device 204, a visual perception device 205, an edge computing module 206 and a core network 207.

[0071] It can be seen that the central unit 201 is connected to the first sending and receiving point 202 and the second sending and receiving point 203 respectively. The communication perception signal is collected by the first sending and receiving point 202 and sent to the central unit 201. The visual perception device 205, the client front-end device 204, the second sending and receiving point 203 and the central unit 201 are connected in sequence, and the visual perception image is collected by the visual perception device 205 and forwarded to the central unit 201 by the client front-end device 204 and the second sending and receiving point 203.

[0072] The 5G RAN architecture adopts the independent deployment of the central unit 201 (CU) and the distribution unit (DU). There are generally multiple distribution units in an indoor distribution. Each distribution unit includes a transmitting and receiving point (TRP) to provide users with wireless access signals and perform perception and positioning at the same time.

[0073] The functions of the central unit 201 and the distributed unit are divided according to the real-time nature of the processing content. The central unit 201 device mainly includes non-real-time wireless high-level protocol stack functions, and also supports the sinking of some core network 207 functions and the deployment of edge application services, while the distributed unit device mainly processes the physical layer functions and layer 2 functions with real-time requirements.

[0074] The client front-end device 204 (CPE) can provide a wifi network for the lower end and a 5G network connection for the upper end. In this proposal, it mainly connects to the 5G sending and receiving points and provides a wifi network for the visual perception device 205 (such as a camera). The client front-end device 204 can provide front-end computing resources as a distributed node for federated learning to assist the visual perception device 205 in completing local real-time computing tasks.

[0075] The edge computing module 206 can be used as a part of the central unit 201. The specific data processing process can be realized by the edge computing module 206. The edge computing module 206 is used to integrate communication perception signals and visual perception images to perform collaborative indoor positioning; provide a federated learning center node and integrate distributed node model parameters. In addition, the central unit 201 is also connected to the core network 207.

[0076] In the embodiment of the present application, the number of visual perception devices 205, client front-end devices 204, first sending and receiving points 202, and second sending and receiving points 203 can be set according to actual conditions, and the structure shown in the figure does not limit other schemes.

[0077] The target area includes a communication perception positioning area and a visual perception positioning area. The purpose of this solution is to accurately track and locate the target object in the target area. In the communication perception positioning area where communication perception can accurately locate, the communication perception signal is collected through the first sending and receiving point 202, thereby performing communication perception tracking and positioning, and achieving low-cost and high-precision tracking and positioning. In the visual perception positioning area where communication perception cannot accurately locate, this solution collects visual perception images through the visual perception device 205, thereby performing high-precision visual perception tracking and positioning in the visual perception positioning area.

[0078] S102: Determine the movement state of the target object in the target area based on the communication perception signal and the visual perception image. (When the movement state represents that the target object moves in the communication perception positioning area, enter S103. When the movement state represents that the target object moves in the visual perception positioning area, enter S104. When the movement state represents that the target object moves from the communication perception positioning area to the visual perception positioning area, enter S105. When the movement state represents that the target object moves from the visual perception positioning area to the communication perception positioning area, enter S106).

[0079] pass Figure 2 In the architecture shown, the central unit is able to receive communication perception signals in the perception positioning area and visual perception images in the visual perception positioning area, and then continue to determine the movement state of the target object in the target area based on the received communication perception signals and visual perception images.

[0080] It should be noted that before determining the movement state of the target object, the target object in the target area needs to be identified first. If the target object exists in the target area, it needs to be tracked and located. The target object is identified based on the communication perception signal and the visual perception image. The communication perception signal is used to identify the target object in the communication perception positioning area, while the visual perception image is used to identify the target object in the visual perception positioning area.

[0081] When the target object is identified based on the communication perception signal and the visual perception image, it can be realized through the central unit. In addition, in actual application, in order to reduce the computing pressure of the central unit, a global image recognition model can be established in the client front-end device, and the recognition of the visual perception image can be realized by the client front-end device. After the client front-end device identifies the target object, it can send the corresponding data to the central unit, further reducing the occupation of communication resources.

[0082] There are four types of movement states of the target object in the target area, namely, moving in the communication perception positioning area, moving in the visual perception positioning area, moving from the communication perception positioning area to the visual perception positioning area, and moving from the visual perception positioning area to the communication perception positioning area.

[0083] S103: Tracking and locating the target object based on the communication sensing signal.

[0084] S104: Tracking and locating the target object based on the visual perception image.

[0085] S105: Switch from the communication perception signal to the visual perception image to track and locate the target object.

[0086] S106: Switch from visual perception image to communication perception signal to track and locate the target object.

[0087] For S103, S104, S105 and S106, because the corresponding positioning area needs to adopt corresponding tracking and positioning technology and needs to be switched in time to ensure the accuracy of tracking and positioning, the embodiment of the present application adopts corresponding tracking and positioning technology to track and position the target object based on four different situations.

[0088] In the two cases where the target object moves in the communication perception positioning area or moves in the visual perception positioning area, the target object can be tracked and positioned based on the communication perception signal or the visual perception image respectively.

[0089] In the two cases where the target object moves from the communication perception positioning area to the visual perception positioning area, or from the visual perception positioning area to the communication perception positioning area, it is necessary to switch the tracking positioning technology. In addition, in the process of moving from one positioning area to another, the switching needs to be completed in time to avoid the loss of the target object or the position recognition error. Therefore, it is also necessary to accurately determine the moment when the target object switches between the two positioning areas so as to switch the tracking scheme in time.

[0090] The integrated network of synaesthesia and computing refers to the integration of communication and perception functions, so that the communication system has both communication and perception functions. While transmitting information through wireless channels, it actively recognizes and analyzes the characteristics of the channel to perceive the physical characteristics of the surrounding environment, and integrates edge computing resources for analysis, reducing synaesthesia latency, improving analysis accuracy, and further enhancing communication and perception functions. Visual positioning is a technology that uses visual sensors to obtain environmental images and extract feature values ​​to obtain the real-time position and posture changes of the current camera, thereby achieving positioning. Both synaesthesia positioning and visual positioning are device-free positioning, which has natural integration advantages.

[0091] This solution proposes an overall architecture of collaborative indoor positioning that integrates visual positioning and synergy computing, which realizes the architectural integration of 5Gadvanced's perceptual positioning and visual positioning, so that the located objects do not need to be configured with any tags or equipment, thus solving the problem of limitations of the located objects and further improving positioning accuracy.

[0092] The indoor positioning method of the embodiment of the present application needs to locate the target object in the target area. The target area includes a communication perception positioning area and a visual perception positioning area respectively. The method is applied to the central unit. The central unit receives the communication perception signal and the visual perception image. After identifying the existence of the target object in the target area, the central unit determines the movement state of the target object in the target area based on the communication perception signal and the visual perception image. Finally, the corresponding tracking and positioning methods are adopted based on the four situations respectively. When the target object moves from one positioning area to another positioning area, the tracking and positioning based on the communication perception signal or the tracking and positioning based on the visual perception image can be switched through the fused positioning technology, so as to achieve a better tracking and positioning effect. Due to the scheme proposed in the embodiment of the present application, a visual perception device is set in the visual perception positioning area where the communication perception positioning technology cannot accurately locate, and the accuracy of tracking and positioning the target object in the area is guaranteed by the visual perception positioning technology. It can be seen that by integrating the communication perception positioning technology and the visual perception positioning technology, it can be ensured that the target object can be accurately tracked and positioned in the entire target area.

[0093] In the above embodiments, it is mentioned that the movement state of the target object in the target area needs to be determined based on the communication perception signal and the visual perception image, and the target object has four movement states, namely, moving in the communication perception positioning area, moving in the visual perception positioning area, moving from the communication perception positioning area to the visual perception positioning area, and moving from the visual perception positioning area to the communication perception positioning area. The following mainly describes the determination methods for the latter two situations.

[0094] This embodiment provides a preset condition to filter out the situation where the target object moves from the communication perception positioning area to the visual perception positioning area. Specifically, when the target object is located in the communication perception positioning area and satisfies any one or any combination of the following conditions, it is determined that the target object moves from the communication perception positioning area to the visual perception positioning area:

[0095] The future predicted position of the target object includes a position located in the visual perception positioning area; the future predicted position is determined according to the historical position of the target object;

[0096] Outliers are generated in each position obtained by tracking and locating the target object based on the communication sensing signal;

[0097] The corresponding boundaries of the visual perception positioning area detect new target objects.

[0098] An embodiment of the present application proposes a method for determining that a target object moves from a communication perception positioning area to a visual perception positioning area, thereby timely switching from a communication perception signal to a visual perception image to track and locate the target object, thereby ensuring the accuracy of tracking and positioning.

[0099] This embodiment provides a preset condition to filter out the situation where the target object moves from the visual perception positioning area to the communication perception positioning area. Specifically, when the target object is located in the visual perception positioning area and satisfies any one or any combination of the following conditions, it is determined that the target object moves from the visual perception positioning area to the communication perception positioning area:

[0100] The future predicted position of the target object includes a position located in the communication sensing positioning area; the future predicted position is determined according to the historical position of the target object;

[0101] Visual perception tracking locates the position of the lost target object;

[0102] The corresponding boundary of the communication sensing positioning area detects a new target object.

[0103] An embodiment of the present application proposes a method for determining that a target object moves from a visual perception positioning area to a communication perception positioning area, thereby timely switching from a visual perception image to a communication perception signal to track and locate the target object, thereby avoiding losing the position of the target object.

[0104] In practical applications, the target area can be divided into multiple grids; each grid is assigned a grid number, so that the position of the target object can be obtained by simply determining the grid number where the target object is located. Correspondingly, tracking and locating the target object includes: determining the grid where the target object is located based on communication perception signals and / or visual perception images; determining the grid number corresponding to the grid where the target object is located; and determining the position of the target object based on the grid number.

[0105] The embodiment of the present application can unify the coordinate system and quickly confirm the location of the target object by rasterizing the target area.

[0106] The first sending and receiving point can achieve accurate tracking and positioning in its own line-of-sight area, but cannot be accurately tracked and positioned in the non-line-of-sight area. Therefore, the communication perception positioning area can be determined as the line-of-sight area of ​​the first sending and receiving point, and the visual perception positioning area can be determined as the non-line-of-sight area of ​​the first sending and receiving point. Visual tracking and positioning of the non-line-of-sight area of ​​the first sending and receiving point through visual perception images can ensure that the target object can be accurately tracked in the entire target area. At the same time, it can also avoid deploying too many visual perception positioning settings, which would result in excessive costs.

[0107] The above embodiment mentioned that the communication perception positioning area is the line-of-sight area of ​​the first sending and receiving point, and the visual perception positioning area is the non-line-of-sight area of ​​the first sending and receiving point. However, in actual application, the specific range of the line-of-sight area and the non-line-of-sight area of ​​the first sending and receiving point may not be known. Therefore, the embodiment of the present application provides a specific determination method, first obtaining the plan view of the target area, the type of the first sending and receiving point, and the point position of the first sending and receiving point; and taking the point position of the first sending and receiving point as the center and based on the radiation direction corresponding to the type of the first sending and receiving point, rendering the plan view (for example, changing the pixel value) to determine the area covered and uncovered by the first sending and receiving point; determining the area covered by the first sending and receiving point as the line-of-sight area, and determining the area not covered by the first sending and receiving point as the non-line-of-sight area.

[0108] The embodiment of the present application proposes a method for confirming the line-of-sight area and the non-line-of-sight area of ​​a first sending and receiving point, which can quickly determine the line-of-sight area and the non-line-of-sight area by rendering a plan view of the target area.

[0109] The above embodiment mentioned that the recognition of the target object can be realized by the client front-end device, thereby reducing the calculation pressure of the central unit. Therefore, the visual perception image forwarded by the second sending and receiving point includes the recognition result of the target object by the client front-end device. Correspondingly, before receiving the communication perception signal in the communication perception positioning area and the visual perception image of the visual perception positioning area, it is also necessary to train the global image recognition model and send it to the client front-end device so that the client front-end device can recognize the target object. The specific training process is as follows: establish a global image recognition model, send the global model parameter vector of the global image recognition model to each client front-end device, so that each client front-end device can perform model training on the global model parameter vector according to its own terminal model. Then the central unit receives the parameter change amount returned by each client front-end device, determines the model parameter adjustment amplitude based on each parameter change amount, and finally updates the global image recognition model based on the global model parameter vector and the model parameter adjustment amplitude. Then the updated global image recognition model is sent to each client front-end device so that the client front-end device can recognize the target object in the visual perception image based on the global image recognition model.

[0110] The embodiment of the present application uses a client front-end device to identify the target object of a visually perceived image, and then sends it to a central unit after identification, which can reduce the computing pressure of the central unit, thereby speeding up the processing speed. In addition, a training method for an image recognition model is also proposed, through which the target object in a visually perceived image can be quickly identified.

[0111] Here Figure 2A specific implementation scheme is provided by taking the framework as an example. It should be noted that the scheme proposed in the embodiment of the present application is one example of the present application and does not limit other schemes. Figure 3 FIG. 1 shows a flow chart of an overall solution of fusion positioning provided by an embodiment of the present application; Figure 3 As shown, the following steps are included: S301: visual perception device fills in points. S302: edge computing network global image recognition model training. S303: target object recognition and visual perception positioning data transmission. S304: communication perception signal transmission and reception. S305: communication perception positioning data transmission. S306: fusion tracking and positioning of communication perception and visual perception.

[0112] First, according to the specific indoor environment and the coverage of indoor sending and receiving points, visual perception equipment is deployed to make up for the non-line-of-sight areas covered by indoor sending and receiving points. Next, the target object to be located is determined, and the image recognition training is performed using the federated learning architecture based on the edge computing network framework of synaesthesia. The client front-end device recognizes the visual perception image sent by the visual perception device, and when it determines that the image contains the target object to be tracked and located, it sends the processed visual perception positioning data to the sending and receiving point side. The sending and receiving point sends the processed visual perception positioning data to the central unit, and the edge computing module processes the data. The sending and receiving point sends and receives communication perception signals, sends the communication perception positioning data to the central unit, and is processed by the edge computing module at the same time. Finally, the communication perception positioning data and the visual perception positioning data are integrated and calculated in the edge computing module.

[0113] For S301, visual perception equipment is used to fill in the points. Since the positioning accuracy of the sending and receiving point perception positioning is relatively insufficient for the non-line-of-sight area where the signal is blocked, it is necessary to fill in the points of visual perception equipment in the non-line-of-sight area. The area that needs to be visually perceived is determined based on the specific indoor environment and the indoor sending and receiving point coverage. The following is a specific image processing method for determining the device filling point area.

[0114] 1. First, obtain the indoor floor plan, the indoor transmitting and receiving points, and the coverage antenna type (omnidirectional or directional); if it is a directional antenna, also obtain the coverage lobe width.

[0115] 2. Binarize the floor plan, with the indoor building structure set to black with a grayscale value of 0, and the others set to white with a grayscale value of 255.

[0116] 3. With the sending and receiving points as the center, the radiation direction of the antenna is the image rendering direction. If the antenna is an omnidirectional antenna, it is rendered in the 360° direction. If it is a directional antenna, it is rendered in the radiation direction of its beam width. If a pixel is 255, it is adjusted to other values ​​(such as 128). If a pixel is 0, the rendering in that direction is stopped.

[0117] 4. The white area in the indoor plan, that is, the area with a pixel value of 255, is an area that is difficult for the communication perception signal to cover (non-line-of-sight area), and needs to be supplemented by the visual perception device. The rest of the area is the line-of-sight area.

[0118] For S302, the training process of the edge computing network global image recognition model is described below. Figure 4 A schematic diagram of a system architecture for model training provided by an embodiment of the present application is shown; Figure 4 As shown, a central unit is connected to multiple client front-end devices respectively, and the central unit further processes the data through the edge computing module.

[0119] In order to reduce unnecessary image data transmitted through 5G wireless, reduce indoor positioning costs, and improve the utilization efficiency of 5G wireless resources, the embodiment of the present application uses an edge computing network framework to perform image recognition of the target object on the client front-end device side, and only sends visual perception positioning data after the target object is detected, which greatly reduces the demand for 5G network resources. At the same time, through distributed model training and reasoning, the computing pressure of the edge computing module is relieved and the delay of image data processing is reduced.

[0120] The edge computing power of the integrated network of synergy and computing is used to train the global image recognition model. The edge computing model deploys the target object recognition central model, and the distributed model is deployed on the front-end device side of each client. The global image recognition model is managed using the federated learning architecture.

[0121] First, determine the target object to be identified and collect labeled image data of the corresponding target object for training.

[0122] Then the global image recognition model is trained. The system can choose to use the annotated pictures for separate training, or use the existing lightweight image recognition model for transfer learning. Transfer learning uses a lightweight basic model such as MobileNet. The specific process is as follows:

[0123] 1. Select the source task: Select a source task that is related to the target task (the target object to be identified). The source task is usually a task that is pre-trained on large-scale data so that the model can learn common features.

[0124] 2. Select the model architecture: Consider the computing power of the client's front-end device and choose a lightweight model architecture.

[0125] 3. Initialize the model: Use the pre-trained weights on the source task to initialize the model parameters. The model already includes the learning of the source task and can better adapt to the target task.

[0126] 4. Model structure adjustment: According to the target objects identified in the previous article, adjust the fully connected layer and output layer of the model so that the number of recognized types matches the number of target object types.

[0127] 5. Transfer training: Based on the labeled image data in the previous article, fine-tune the training on the data of the target task. Back-propagate the error on the target task and update the model parameters to adapt it to the specific characteristics of the target task.

[0128] Next, conduct federated learning on each node.

[0129] Input: local image data E of the client front-end device distribution node i , federated learning round R, local iteration number N, learning rate η.

[0130] Output: global model parameter vector w T , the local model parameter vector

[0131] 1. Global model node execution:

[0132] 2. Initialize the global model parameters to the parameters generated in the previous article, set the vector to w 0 .

[0133] 3. Cycle (3.1 to 3.3): t = 0, 1, ..., R-1.

[0134] 3.1. Send the global model parameter vector w T To each end user distribution node.

[0135] 3.2. Receive the global model parameter vector change from the distributed nodes after completing local training and feedback

[0136] 3.3. Assume that the total number of samples of the distributed nodes of each client front-end device is M, and calculate the total amount of change of the global model parameter vector in this round according to the proportion of the sample size of each distributed node:

[0137] 4. Update the global model parameter vector w t+1 ←AVG(w t ,ΔW,η), and the averaging method is used to update the parameters.

[0138] 5. Output the updated global model parameter vector w T .

[0139] 6. The client front-end device performs local training, local iteration number E, outputs and feedbacks the training results to the global model.

[0140] After completing the above iterative training, the edge computing module will contain the final parameter w T The global model is deployed to the client front-end device side to provide target object recognition and reasoning capabilities.

[0141] For S303, target object recognition and visual perception positioning data are sent. The client front-end device uses the global image recognition model obtained above to recognize and detect the target object in the image sent by the visual perception device. When the target object is detected, the target object area is framed and the target object type and detection time are recorded. t , the corresponding target object type S t,i and the detection time t is sent to the edge computing module, that is, [P t ,S t,i ,t].

[0142] For S304, the communication perception signal is sent and received. Considering the downlink communication scenario where the indoor distribution is a multi-antenna transmission and reception point and a single-antenna user equipment UE, the transmission and reception point transmits data symbols to the objects in the indoor distribution, and at the same time collects the reflected signals through a multi-antenna co-located radar receiver for detection and tracking. The transmission and reception point adopts a multi-input and multi-output MIMO uniform linear array with a transmitting antenna and a receiving antenna. Given the precoder f, the perception signal model across N subcarriers can be expressed as:

[0143] Among them, x n is an OFDM signal, is the additive white Gaussian noise AWGN, H n It is the feature of synaesthesia positioning.

[0144] For S305, communication sensing positioning data is sent, and the distance, speed, angle, etc. of the target object are obtained by processing the sensing signal. For distance measurement: For the self-transmitting and self-receiving sensing working mode, the time delay t between the transmitting wave and the target echo is measured to obtain the distance R between the target and the measuring point = Ct / 2, where the speed of light C = 3×10 8m / s. For speed measurement: the Doppler effect generated by the target movement is used to derive the target speed by measuring the Doppler frequency of the sensing signal, where the Doppler frequency fd = v / λ, where v is the target movement speed and λ is the signal wavelength. For angle measurement: the target angle is determined by the intensity difference between the sensing signals of different beams or the phase difference of the sensing signals of the multi-phase center.

[0145] Further, the distance, speed, angle, and perception time of the target object are set to [d t ,v t ,a t ,t] is sent to the edge computing module.

[0146] Finally, through S306, the fusion tracking and positioning of communication perception and visual perception is performed: first, the indoor position coordinate system is unified, and the position calculations are performed for the communication perception positioning data and the video positioning data respectively, and the two types of position data are further fused according to the scene conditions.

[0147] First, unify the indoor position coordinate system, grid the indoor area according to the positioning accuracy requirements, and number each grid to determine the corresponding position of the number in the room. For each visual perception device, define its positioning range, that is, the specific grid number it contains; at the same time, determine the positioning range of the communication perception device as the entire indoor area excluding the area of ​​the visual perception device positioning range.

[0148] Then the communication perception positioning and visual perception positioning are calculated. Communication perception positioning: Using the [d t ,v t ,a t ,t] calculate the specific location, that is, the location of the sending and receiving point and the distance d between the target object and the sending and receiving point t , combined with the azimuth angle a formed with the sending and receiving points t Get the grid and grid number where the target object is located.

[0149] Visual perception positioning: Based on the visual perception positioning data obtained in the previous article [P t ,S t,i ,t] for processing. First, for each frame, the area of ​​the target object is calculated, that is, the width and height of the bounding box of the target object are used to calculate; the area information of the target is further used in combination with the internal reference information of the visual perception device to estimate the position of the target in the coordinate system of the visual perception device; according to the position, orientation and geometric relationship of the visual perception device with the scene, the position in the coordinate system of the visual perception device is converted to the grid position determined in the previous text, and the grid number is obtained.

[0150] Then, scene-based positioning fusion is performed. For target objects moving in the communication perception positioning area or the visual positioning area, they are positioned according to the previous method. For target objects moving from the communication perception positioning area to the visual positioning area or from the visual positioning area to the communication perception positioning area, since the switching of positioning technology occurs, it is necessary to fuse the two positioning technologies to improve positioning accuracy.

[0151] When the target object moves from the communication perception positioning area to the visual perception positioning area, the following three preset conditions are given:

[0152] 1. First, based on the latest i position data of the target object and combined with the indoor distribution scene structure, time series prediction is performed to obtain the positions of the future j samples, among which there are positions belonging to the visual positioning area.

[0153] 2. Detect the change of the position data of communication-aware positioning and determine the existence of outliers. (When performing communication-aware positioning, the target object moves from the LOS area to the NLOS area. Due to the great change in the channel model, the positioning changes greatly. The position determined in the NLOS state is significantly different from the position determined previously by the LOS.)

[0154] 3. Detect related visual positioning boundary areas and find that the target object is newly added.

[0155] When conditions 1, 3, or 2, 3, or 1, 2, 3 are met, the system stops positioning the target object on the communication perception positioning side, and the visual perception positioning side provides the positioning of the target object.

[0156] When the target object moves from the visual perception positioning area to the communication perception positioning area, the following three preset conditions are given:

[0157] 1. First, based on the latest i position data of the target object and combined with the indoor distribution scene structure, time series prediction is performed to obtain the positions of the future j samples, among which there are positions belonging to the communication perception positioning area.

[0158] 2. The target object was not found during visual inspection.

[0159] 3. Detect related communication perception and positioning boundary areas and find that the target object is newly added.

[0160] If 1, 2 and 3 are satisfied at the same time, the system stops positioning the target object on the visual perception positioning side, and the communication perception positioning side provides the positioning of the target object.

[0161] In order to solve the above technical problems, the embodiment of the present application further provides an indoor positioning system, including:

[0162] The central unit is connected to the first sending and receiving point and the second sending and receiving point respectively, and is used to receive the communication perception signal in the communication perception positioning area collected and sent by the first sending and receiving point; and receive the visual perception image of the visual perception positioning area collected by the visual perception device and forwarded by the client front-end device and the second sending and receiving point; the target area includes the communication perception positioning area and the visual perception positioning area;

[0163] A first sending and receiving point is used to collect communication perception signals in the communication perception positioning area;

[0164] A visual perception device, used for collecting visual perception images in a visual perception positioning area;

[0165] The client front-end device is connected to the visual perception device and the second sending and receiving point respectively, and is used to forward the visual perception image collected by the visual perception device to the second sending and receiving point;

[0166] The second sending and receiving point is used to forward the visual perception image sent by the client front-end device to the central unit.

[0167] The system provided in the embodiment of the present application is the same as the method in the above embodiment, so the two have the same embodiments and beneficial effects, which will not be repeated here.

[0168] In order to solve the above technical problems, the present application also provides an indoor positioning device. Figure 5 The structure diagram of the indoor positioning device provided in the embodiment of the present application is shown; Figure 5 As shown, the device includes the following modules:

[0169] The receiving module 501 is used to receive the communication perception signal in the communication perception positioning area collected and sent by the first sending and receiving point, and receive the visual perception image of the visual perception positioning area collected by the visual perception device and forwarded by the client front-end device and the second sending and receiving point; the target area includes the communication perception positioning area and the visual perception positioning area;

[0170] A determination module 502 is used to determine the movement state of a target object in a target area based on the communication perception signal and the visual perception image; the target object is identified based on the communication perception signal and the visual perception image;

[0171] The positioning module 503 is used to:

[0172] When the mobile state represents that the target object moves in the communication sensing positioning area, the target object is tracked and positioned based on the communication sensing signal;

[0173] When the moving state represents that the target object moves in the visual perception positioning area, the target object is tracked and positioned based on the visual perception image;

[0174] When the moving state represents that the target object moves from the communication perception positioning area to the visual perception positioning area, switching from the communication perception signal to the visual perception image to track and locate the target object;

[0175] When the movement state represents that the target object moves from the visual perception positioning area to the communication perception positioning area, the target object is tracked and positioned by switching from the visual perception image to the communication perception signal.

[0176] In some embodiments, the determination module 502 is specifically configured to:

[0177] When the target object is located in the communication perception positioning area and any one or any combination of the following conditions is met, it is determined that the target object moves from the communication perception positioning area to the visual perception positioning area:

[0178] The future predicted position of the target object includes a position located in the visual perception positioning area; the future predicted position is determined according to the historical position of the target object;

[0179] Outliers are generated in each position obtained by tracking and locating the target object based on the communication sensing signal;

[0180] The corresponding boundaries of the visual perception positioning area detect new target objects.

[0181] In some embodiments, the determination module 502 is specifically configured to:

[0182] When the target object is located in the visual perception positioning area and any one or any combination of the following conditions is met, it is determined that the target object moves from the visual perception positioning area to the communication perception positioning area:

[0183] The future predicted position of the target object includes a position located in the communication sensing positioning area; the future predicted position is determined according to the historical position of the target object;

[0184] Visual perception tracking locates the position of the lost target object;

[0185] The corresponding boundary of the communication sensing positioning area detects a new target object.

[0186] In some embodiments, the target area is divided into a plurality of grids; each grid corresponds to a grid number;

[0187] The positioning module 503 is specifically used for:

[0188] Determine the grid where the target object is located based on the communication perception signal and / or the visual perception image;

[0189] Determine the grid number corresponding to the grid where the target object is located;

[0190] Determines the location of the target object based on the grid number.

[0191] In some embodiments, the indoor positioning device further includes:

[0192] An acquisition module, used to acquire a plan view of a target area, a type of a first sending and receiving point, and a position of the first sending and receiving point before receiving a communication sensing signal in a communication sensing positioning area and a visual sensing image in a visual sensing positioning area;

[0193] A rendering module, used to render the plan view with the position of the first sending and receiving point as the center and based on the radiation direction corresponding to the type of the first sending and receiving point, so as to determine the areas covered and uncovered by the first sending and receiving point;

[0194] The determination module 502 is further configured to determine the area covered by the first transmitting and receiving point as the line-of-sight area, and determine the area not covered by the first transmitting and receiving point as the non-line-of-sight area.

[0195] In some embodiments, the visual perception image forwarded by the second sending and receiving point includes the recognition result of the target object by the client front-end device;

[0196] The indoor positioning device also includes:

[0197] A module is established, which is used to establish a global image recognition model before receiving the communication perception signal in the communication perception positioning area and the visual perception image in the visual perception positioning area; a terminal model is established in each client front-end device;

[0198] A sending module, used to send the global model parameter vector of the global image recognition model to each client front-end device, so that each client front-end device can perform model training on the global model parameter vector according to the terminal model;

[0199] The receiving module 501 is also used to receive the parameter change amount returned by each client front-end device;

[0200] The determination module 502 is further used to determine the adjustment range of the model parameters based on the change amount of each parameter;

[0201] An adjustment module, specifically used to update the global image recognition model based on the global model parameter vector and the model parameter adjustment amplitude;

[0202] The sending module is also used to send the updated global image recognition model to each client front-end device, so that the client front-end device can recognize the target object in the visual perception image based on the global image recognition model.

[0203] The device provided in the embodiment of the present application is the same as the method in the above embodiment, so both have the same embodiments and beneficial effects, which will not be repeated here.

[0204] Figure 6 FIG. 1 shows a schematic diagram of the hardware structure of the indoor positioning device provided in an embodiment of the present application; Figure 6 As shown, the indoor positioning device may include a processor 601 and a memory 602 storing computer program instructions.

[0205] Specifically, the processor 601 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0206] The memory 602 may include a large capacity memory for data or instructions. By way of example and not limitation, the memory 602 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive or a combination of two or more of these. In appropriate cases, the memory 602 may include a removable or non-removable (or fixed) medium. In appropriate cases, the memory 602 may be inside or outside the integrated gateway disaster recovery device. In a specific embodiment, the memory 602 is a non-volatile solid-state memory.

[0207] The memory 602 may include a read-only memory (ROM), a random access memory (RAM), a magnetic disk storage media device, an optical storage media device, a flash memory device, an electrical, optical or other physical / tangible memory storage device. Thus, in general, the memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., a memory device) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the method according to an aspect of the present disclosure.

[0208] The processor 601 implements any one of the indoor positioning methods in the above embodiments by reading and executing computer program instructions stored in the memory 602 .

[0209] In one example, the indoor positioning device may further include a communication interface 603 and a bus 604. The processor 601, the memory 602, and the communication interface 603 are connected via the bus 604 and communicate with each other.

[0210] The communication interface 603 is mainly used to implement communication between various modules, devices, units and / or equipment in the embodiments of the present application.

[0211] The bus 604 includes hardware, software or both, coupling the components of the indoor positioning device to each other. For example and not limitation, the bus may include an accelerated graphics port (AGP) or other graphics bus, an enhanced industrial standard architecture (EISA) bus, a front-end bus (FSB), a hypertransport (HT) interconnect, an industrial standard architecture (ISA) bus, an infinite bandwidth interconnect, a low pin count (LPC) bus, a memory bus, a microchannel architecture (MCA) bus, a peripheral component interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a serial advanced technology attachment (SATA) bus, a video electronics standard association local (VLB) bus or other suitable buses or a combination of two or more of these. Where appropriate, the bus 604 may include one or more buses. Although the present application embodiment describes and illustrates a specific bus, the present application considers any suitable bus or interconnect.

[0212] In addition, in combination with the indoor positioning method in the above embodiments, the embodiments of the present application may provide a computer storage medium for implementation. The computer storage medium stores computer program instructions; when the computer program instructions are executed by a processor, any one of the indoor positioning methods in the above embodiments is implemented.

[0213] An embodiment of the present application also provides a computer program product, including a computer program, which implements any one of the indoor positioning methods in the above embodiments when the computer program is processed and executed.

[0214] It should be clear that the present application is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present application is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present application.

[0215] The functional blocks shown in the above-described block diagram can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present application are programs or code segments that are used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. The code segment can be downloaded via a computer network such as the Internet, an intranet, etc.

[0216] It should also be noted that the exemplary embodiments mentioned in this application describe some methods or systems based on a series of steps or devices. However, this application is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiment, or in a different order from the embodiment, or several steps can be performed simultaneously.

[0217] The above describes various aspects of the present disclosure with reference to the flowcharts and / or block diagrams of an indoor positioning method, system, device, equipment, medium and product according to an embodiment of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of boxes in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device to produce a machine so that these instructions executed by the processor of the computer or other programmable data processing device enable the implementation of the functions / actions specified in one or more boxes of the flowchart and / or block diagram. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor or a field programmable logic circuit. It can also be understood that each box in the block diagram and / or flowchart and the combination of boxes in the block diagram and / or flowchart can also be implemented by dedicated hardware that performs the specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions.

[0218] The above is only a specific implementation of the present application. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. It should be understood that the protection scope of the present application is not limited to this. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in this application, and these modifications or replacements should be included in the protection scope of this application.

Claims

1. An indoor positioning method, characterized in that: Applied to a central unit, the method comprises: Receive communication perception signals in the communication perception positioning area and visual perception images in the visual perception positioning area; the communication perception signals are collected and sent by the first sending and receiving point; the visual perception images are collected by the visual perception device and forwarded by the client front-end device and the second sending and receiving point; the target area includes the communication perception positioning area and the visual perception positioning area; Based on the communication perception signal and the visual perception image, determining the movement state of the target object in the target area; the target object is identified based on the communication perception signal and the visual perception image; In a case where the movement state represents that the target object moves in the communication sensing and positioning area, tracking and positioning the target object based on the communication sensing signal; In a case where the movement state represents that the target object moves in the visual perception positioning area, tracking and positioning the target object based on the visual perception image; When the movement state represents that the target object moves from the communication perception positioning area to the visual perception positioning area, switching from the communication perception signal to the visual perception image to track and locate the target object; When the movement state represents that the target object moves from the visual perception positioning area to the communication perception positioning area, the target object is tracked and positioned by switching from the visual perception image to the communication perception signal.

2. The indoor positioning method according to claim 1, characterized in that: Determining the movement state of the target object in the target area based on the communication perception signal and the visual perception image includes: When the target object is located in the communication perception positioning area and any one or any combination of the following conditions is met, it is determined that the target object moves from the communication perception positioning area to the visual perception positioning area: The future predicted position of the target object includes a position located in the visual perception positioning area; the future predicted position is determined according to the historical position of the target object; Generating outliers in each position obtained by tracking and locating the target object based on the communication sensing signal; A new target object is detected at the corresponding boundary of the visual perception positioning area.

3. The indoor positioning method according to claim 1, characterized in that: Determining the movement state of the target object in the target area based on the communication perception signal and the visual perception image includes: When the target object is located in the visual perception positioning area and any one or any combination of the following conditions is met, it is determined that the target object moves from the visual perception positioning area to the communication perception positioning area: The future predicted position of the target object includes a position located in the communication sensing positioning area; the future predicted position is determined according to the historical position of the target object; The visual perception tracking locates the position of the lost target object; A new target object is detected at the corresponding boundary of the communication sensing positioning area.

4. The indoor positioning method according to any one of claims 1 to 3, characterized in that: The target area is divided into a plurality of grids; each grid has a corresponding grid number; The tracking and positioning of the target object includes: Determining a grid where a target object is located based on the communication perception signal and / or the visual perception image; Determine the grid number corresponding to the grid where the target object is located; The position of the target object is determined based on the grid number.

5. The indoor positioning method according to claim 1, characterized in that: The communication perception positioning area is a line-of-sight area of ​​the first sending and receiving point, and the visual perception positioning area is a non-line-of-sight area of ​​the first sending and receiving point.

6. The indoor positioning method according to claim 5, characterized in that: Before receiving the communication perception signal in the communication perception positioning area and the visual perception image in the visual perception positioning area, the method further includes: Acquire a plan view of the target area, a type of the first sending and receiving point, and a location of the first sending and receiving point; Taking the position of the first sending and receiving point as the center and based on the radiation direction corresponding to the type of the first sending and receiving point, rendering the plan view to determine the areas covered and not covered by the first sending and receiving point; An area covered by the first transmitting and receiving point is determined as a line-of-sight area, and an area not covered by the first transmitting and receiving point is determined as a non-line-of-sight area.

7. The indoor positioning method according to claim 1, characterized in that: The visual perception image forwarded by the second sending and receiving point includes a recognition result of the target object by the client front-end device; Before receiving the communication perception signal in the communication perception positioning area and the visual perception image in the visual perception positioning area, the method further includes: Establishing a global image recognition model; establishing a terminal model in each of the client front-end devices; Sending the global model parameter vector of the global image recognition model to each of the client front-end devices, so that each of the client front-end devices performs model training on the global model parameter vector according to the terminal model; Receiving parameter changes returned by each of the client front-end devices; Determining the model parameter adjustment range based on the parameter change amount; Updating a global image recognition model based on the global model parameter vector and the model parameter adjustment amplitude; The updated global image recognition model is sent to each of the client front-end devices, so that the client front-end devices can recognize the target object in the visual perception image based on the global image recognition model.

8. An indoor positioning system, characterized in that: include: The central unit is connected to the first sending and receiving point and the second sending and receiving point respectively, and is used to receive the communication perception signal in the communication perception positioning area collected and sent by the first sending and receiving point; and receiving a visual perception image of a visual perception positioning area collected by a visual perception device and forwarded by a client front-end device and the second sending and receiving point; the target area includes the communication perception positioning area and the visual perception positioning area; The first sending and receiving point is used to collect the communication perception signal in the communication perception positioning area; The visual perception device is used to collect the visual perception image in the visual perception positioning area; The client front-end device is connected to the visual perception device and the second sending and receiving point respectively, and is used to forward the visual perception image collected by the visual perception device to the second sending and receiving point; The second sending and receiving point is used to forward the visual perception image sent by the client front-end device to the central unit.

9. An indoor positioning device, characterized in that: include: A receiving module, used to receive the communication perception signal in the communication perception positioning area collected and sent by the first sending and receiving point, and to receive the visual perception image of the visual perception positioning area collected by the visual perception device and forwarded by the client front-end device and the second sending and receiving point; the target area includes the communication perception positioning area and the visual perception positioning area; A determination module, configured to determine a movement state of a target object in the target area based on the communication perception signal and the visual perception image; the target object is identified based on the communication perception signal and the visual perception image; Positioning module for: In a case where the movement state represents that the target object moves in the communication sensing and positioning area, tracking and positioning the target object based on the communication sensing signal; In a case where the movement state represents that the target object moves in the visual perception positioning area, tracking and positioning the target object based on the visual perception image; When the movement state represents that the target object moves from the communication perception positioning area to the visual perception positioning area, switching from the communication perception signal to the visual perception image to track and locate the target object; When the movement state represents that the target object moves from the visual perception positioning area to the communication perception positioning area, the target object is tracked and positioned by switching from the visual perception image to the communication perception signal.

10. An indoor positioning device, characterized in that: The device comprises: a processor and a memory storing computer program instructions; When the processor executes the computer program instructions, the indoor positioning method according to any one of claims 1 to 7 is implemented.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer program instructions, and when the computer program instructions are executed by the processor, the indoor positioning method according to any one of claims 1 to 7 is implemented.

12. A computer program product, characterized in that When the instructions in the computer program product are executed by a processor of an electronic device, the electronic device executes the indoor positioning method as described in any one of claims 1 to 7.