Map update method and apparatus
Patent Information
- Application Number
- PCT/CN2025/085953
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2025-03-28
- Publication Date
- 2026-10-01
Smart Images
Figure CN2025085953_01102026_PF_FP_ABST
Abstract
Description
A map updating method and apparatus Technical Field
[0001] This application relates to the field of wireless communication technology, and in particular to a map updating method and apparatus. Background Technology
[0002] Wireless local area network (WLAN) sensing is the process by which devices with WLAN sensing capabilities use received wireless signals in a given environment to determine the characteristics of a predetermined target (such as an object, animal, or person). These characteristics include the target's distance, orientation, speed, movement, and behavior.
[0003] In WLAN sensing scenarios for indoor environments, accurate indoor environmental information can improve environmental perception performance. For example, acquiring information including the distribution of obstacles in the indoor environment allows for more accurate location of predetermined targets. Currently, the common method for acquiring indoor environmental information is to collect indoor environmental images and then obtain indoor environmental information based on these images. However, due to the dynamic nature of the indoor environment—such as the movement of furniture or adjustments to the layout of items—the indoor environmental information can change, potentially leading to inaccuracies when based on static images. Summary of the Invention
[0004] This application provides a map updating method and apparatus for updating maps of indoor environments to obtain accurate indoor environment information.
[0005] In a first aspect, embodiments of this application provide a map updating method. This method can be applied to a map updating device, a module (e.g., a circuit, chip, or chip system) within the map updating device, or a logical node, logical module, or software capable of implementing all or part of the functions of the map updating device. The map updating device can be a server, or it can be deployed on a server, which can be a cloud server or a physical server. Taking its application to a map updating device as an example, the method includes: the map updating device acquiring a first map of a target environment, the first map describing the target environment; the map updating device determining the movement trajectory of an object in the target environment based on sensing data sent by sensing devices in the target environment; and the map updating device updating the first map based on the movement trajectory.
[0006] Using the above method, the map updating device can determine the movement trajectory of an object in the target environment based on the received sensing data, and update the first map of the target environment according to the movement trajectory. Thus, the map updating device can update the first map of the target environment based on the object's movement trajectory, obtaining an updated second map of the target environment. When performing sensing and positioning based on this second map, accurate positioning results can be obtained.
[0007] In one possible implementation, the map updating device determines whether there are obstacles on the movement trajectory based on the first map; when there are obstacles on the movement trajectory, the map updating device updates the first map.
[0008] Using the above method, when the map updating device determines that there are obstacles on the movement trajectory, it can determine that the position of the obstacles in the current target environment has changed and the first map needs to be updated, so as to obtain a map that correctly describes the distribution of obstacles in the target environment.
[0009] In one possible implementation, the map updating device determines at least one trajectory point in the first map, the trajectory point being a location point on the first map mapped from the movement trajectory; the map updating device determines whether there are obstacles on the movement trajectory based on the at least one trajectory point.
[0010] Using the above method, the map updating device can accurately determine whether there are obstacles on the movement trajectory based on at least one trajectory point in the first map.
[0011] In one possible implementation, the map updating device determines a first parameter corresponding to the region where each trajectory point is located, the first parameter being used to characterize the probability that the corresponding region includes obstacles; the map updating device determines whether there are obstacles on the movement trajectory based on the first parameter corresponding to the region where each trajectory point is located.
[0012] Using the above method, the map update device can accurately determine whether there are obstacles on the movement trajectory based on the probability of obstacles in the area where the trajectory point is located.
[0013] In one possible implementation, the map updating device obtains a first parameter corresponding to each region in the first map, the first parameter being used to characterize the probability that the corresponding region includes obstacles; the map updating device adjusts the first parameter corresponding to each region according to whether each region includes trajectory points, the trajectory points being the position points of the movement trajectory mapped onto the first map; the map updating device determines the image information corresponding to each region based on the adjusted first parameter corresponding to each region; and the map updating device generates a second map updated to the first map based on the image information corresponding to each region.
[0014] Using the above method, the map updating device can update the first map to obtain a second map that can accurately describe the distribution of obstacles in the target environment.
[0015] In one possible implementation, the map updating device generates a trajectory map corresponding to the target environment based on the movement trajectory, the trajectory map being used to describe the number of times the trajectory is covered for each area in the target environment; the map updating device predicts an obstacle distribution map of the target environment based on the trajectory map; and the map updating device updates the first map based on the obstacle distribution map.
[0016] Using the above method, the map updating device can accurately predict the obstacle distribution map of the target environment based on the trajectory map, and obtain a second map that can accurately describe the obstacle distribution in the target environment when the first map needs to be updated.
[0017] In one possible implementation, the map update device inputs the trajectory map into a trained first network model to obtain an obstacle distribution map of the target environment output by the first network model; wherein the first network model is trained based on a training sample set, the training sample set including multiple training samples, each training sample including a trajectory sample map and an obstacle distribution sample map.
[0018] Using the above method, the map update device can obtain a fast and accurate obstacle distribution map of the target environment based on the trained network model.
[0019] In one possible implementation, if there is a difference between the obstacle distribution map and the first map, the first map is updated.
[0020] Using the above method, the map updating device can update the first map to obtain a second map that can accurately describe the distribution of obstacles in the target environment.
[0021] Secondly, this application provides a map updating device that has the functions of the first aspect mentioned above. For example, the map updating device includes modules, units or means corresponding to the operations involved in the first aspect mentioned above. These modules, units or means can be implemented by software, hardware or a combination of software and hardware.
[0022] For example, the map updating apparatus of this application embodiment includes a communication unit and a processing unit; wherein the communication unit is used to communicate with a sensing device in the target environment, and for example, to receive sensing data sent by the sensing device; the processing unit is used to acquire a first map of the target environment, the first map being used to describe the target environment, and for example, the first map being used to describe the distribution of obstacles in the target environment; to determine the movement trajectory of an object in the target environment based on the sensing data sent by the sensing device in the target environment; and to update the first map based on the movement trajectory.
[0023] In one possible implementation, the processing unit is configured to determine whether there are obstacles on the movement trajectory based on the first map; and to update the first map when there are obstacles on the movement trajectory.
[0024] In one possible implementation, a processing unit is configured to determine at least one trajectory point in the first map, the trajectory point being a location point on the first map mapped from the movement trajectory; and to determine whether there are obstacles on the movement trajectory based on the at least one trajectory point.
[0025] In one possible implementation, a processing unit is configured to determine a first parameter corresponding to the region where each trajectory point is located, the first parameter being used to characterize the probability that the corresponding region includes an obstacle; and to determine whether there is an obstacle on the movement trajectory based on the first parameter corresponding to the region where each trajectory point is located.
[0026] In one possible implementation, a processing unit is configured to obtain a first parameter corresponding to each region in a first map, the first parameter being used to characterize the probability that the corresponding region includes obstacles; adjust the first parameter corresponding to each region according to whether each region includes trajectory points, the trajectory points being the location points of the movement trajectory mapped onto the first map; determine the image information corresponding to each region based on the adjusted first parameter corresponding to each region; and generate a second map updated to the first map based on the image information corresponding to each region.
[0027] In one possible implementation, a processing unit is configured to generate a trajectory map corresponding to the target environment based on the movement trajectory, the trajectory map describing the number of times the trajectory covers each area in the target environment; predict an obstacle distribution map of the target environment based on the trajectory map; and update the first map based on the obstacle distribution map.
[0028] In one possible implementation, a processing unit is configured to input the trajectory map into a trained first network model to obtain an obstacle distribution map of the target environment output by the first network model; wherein the first network model is trained based on a training sample set, the training sample set including multiple training samples, each training sample including a trajectory sample map and an obstacle distribution sample map.
[0029] In one possible implementation, the processing unit is configured to update the first map if there is a difference between the obstacle distribution map and the first map.
[0030] Thirdly, this application provides a map updating apparatus, which includes at least one processor and at least one memory. The at least one processor is coupled to the memory. The memory is used to store part or all of the computer program or instructions necessary to implement the functions involved in the first aspect above. The at least one processor can execute the computer program or instructions, and when the computer program or instructions are executed, the map updating apparatus causes it to implement the methods in any possible design or implementation of the first aspect above.
[0031] The aforementioned map update device may be a server, a module in the server (such as a circuit, chip, or chip system), or a logical node, logical module, or software that can implement all or part of the server's functions.
[0032] Fourthly, this application provides a computer-readable storage medium storing a computer program or instructions that, when executed, implement the method in any of the possible designs of the first aspect described above.
[0033] Fifthly, this application provides a computer program product comprising a computer program or instructions that, when executed, implement any of the possible design methods of the first aspect described above.
[0034] For the various aspects of the second to fifth aspects mentioned above, and the technical effects that each aspect may achieve, please refer to the above description of the technical effects that various possible solutions can achieve for any aspect of the first aspect, which will not be repeated here. Attached Figure Description
[0035] Figure 1 is a schematic diagram of a network architecture provided in an embodiment of this application;
[0036] Figure 2 is a schematic diagram of the architecture of a wireless fidelity (Wi-Fi) communication system provided in an embodiment of this application;
[0037] Figure 3 is a schematic diagram of a communication protocol architecture for a star-flash communication technology provided in an embodiment of this application;
[0038] Figure 4 is a schematic diagram of the architecture of a Star Flash Communication System provided in an embodiment of this application;
[0039] Figure 5 is a flowchart illustrating a map updating method provided in an embodiment of this application;
[0040] Figure 6 is a flowchart illustrating a model-driven algorithm based on path length change rate (PLCR) provided in an embodiment of this application for determining the movement trajectory of an object in a target environment.
[0041] Figure 7 is a flowchart illustrating a data-driven algorithm based on PLCR provided in an embodiment of this application for determining the movement trajectory of an object in a target environment.
[0042] Figure 8 is a schematic diagram of a map to be processed that includes multiple trajectory data provided in an embodiment of this application;
[0043] Figure 9 is a schematic diagram of a divided map to be processed provided in an embodiment of this application;
[0044] Figure 10 is a schematic diagram of a trajectory map provided in an embodiment of this application;
[0045] Figure 11 is a schematic diagram of the cumulative trajectory coverage times of a data collection object provided in an embodiment of this application;
[0046] Figure 12 is a schematic diagram of a map updating device provided in an embodiment of this application;
[0047] Figure 13 is a schematic diagram of a map updating device provided in an embodiment of this application. Detailed Implementation
[0048] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the embodiments of this application will be further described in detail below with reference to the accompanying drawings.
[0049] In the embodiments of this application, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, a and b, a and c, b and c, or a, b, and c, where each of a, b, and c can be an element itself or a set containing one or more elements.
[0050] In this application, terms such as "exemplary," "in some embodiments," and "in other embodiments" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" in this application should not be construed as being more preferred or advantageous than other embodiments or designs. Rather, the term "exemplary" is used to present the concept in a specific manner.
[0051] In this application, the terms "of," "corresponding (relevant)," and "corresponding" may sometimes be used interchangeably. It should be noted that, unless a distinction is emphasized, their intended meanings are consistent. Similarly, in the embodiments of this application, "communication" and "transmission" may sometimes be used interchangeably. It should be noted that, unless a distinction is emphasized, their intended meanings are consistent. For example, transmission can include sending and / or receiving, and can be a noun or a verb.
[0052] It should be noted that the terms "first" and "second" used in the embodiments of this application are used only for the purpose of distinguishing descriptions and should not be construed as indicating or implying relative importance or order.
[0053] To facilitate understanding of this solution, the application scenario of this application embodiment is first introduced. As shown in Figure 1, the network architecture includes a local network 10 and a map updating device 20. The local network 10 can be deployed in the target environment, and the map updating device 20 is used to update a first map of the target environment. The local network 10 includes at least one access point device and at least one terminal device connected to the access point device. Figure 1 uses access point device 101 and access point device 102 as an example; access point device 101 is connected to terminal device 1 and terminal device 2, and access point device 102 is connected to terminal device 3. The access point device or terminal device in the local network 10 can collect sensing data in the target environment, such as channel state information (CSI) data. For example, terminal device 1 can act as a sensing transmitter, sending sensing signals; access point device 101 can act as a sensing receiver, receiving the sensing signals sent by the sensing transmitter and measuring the sensing signals to obtain sensing data. For example, access point device 102 can act as a sensing transmitter, sending sensing signals; terminal device 3 can act as a sensing receiver, receiving the sensing signals sent by the sensing transmitter and measuring the sensing signals to obtain sensing data. By measuring the sensing signals to obtain sensing data, objects in the target environment can be sensed, such as people or specific items in the target environment.
[0054] After receiving the sensing data, the access point device or terminal device in the local network 10 reports the measured sensing data to the map update device 20. The map update device 20 updates the first map of the target environment based on the sensing data.
[0055] It should be understood that the map update device 20 in the network architecture shown in Figure 1 can be any device with computing power, or the map update device 20 can be deployed on a device with computing power. For example, the map update device 20 can be a server, or deployed on a server; wherein the server can be a physical server or a cluster of physical servers, or the server can also be a cloud server. The embodiments of this application do not limit the form of the server.
[0056] In the network architecture shown in Figure 1, access point devices and terminal devices in the local network can establish connections through various communication methods.
[0057] For example, a short-range communication connection can be established between the access point device and the terminal device; for instance, a WLAN communication connection or a Starlink communication connection can be established between the access point device and the terminal device. When establishing a WLAN communication connection between the access point device and the terminal device, the terminal device can be a station (STA), and the access point device can be an access point (AP); when establishing a Starlink communication connection between the access point device and the terminal device, the terminal device can be a terminal (T) node, and the access point device can be a grant (G) node.
[0058] For example, a 3GPP communication connection can be established between the access point device and the terminal device. When establishing a 3GPP communication connection between the access point device and the terminal device, the terminal device can use a cellular network to connect to the access point device. Therefore, the terminal device can be of various types, and the access point device can be of various types of base stations.
[0059] The network architecture described in this application uses the establishment of a Starlink connection or WLAN connection between a terminal device and an access point device as an example. Those skilled in the art will readily understand that the various aspects involved in this application can be extended to other communication networks employing various standards or protocols, such as high-performance radio local area networks (HIPERLANs), wireless wide area networks (WWANs), wireless personal area networks (WPANs), or other networks currently known or developed in the future. Therefore, regardless of the coverage area and wireless access protocol used, the various aspects provided in this application can be applied to any suitable wireless network.
[0060] The technical solutions of this application embodiment can also be applied to various communication systems or networks, such as: Long Term Evolution (LTE) systems, LTE Frequency Division Duplex (FDD) systems, LTE TDD systems, Universal Mobile Telecommunication System (UMTS) systems, Worldwide Interoperability for Microwave Access (WiMAX) communication systems, 5th Generation (5G) systems or New Radio (NR) systems, Future Communication Systems, Internet of Things (IoT) networks, or Vehicle-to-Everything (V2X) networks, etc. The communication systems applicable to this application described above are merely illustrative examples; the application is not limited to these examples. These examples are uniformly described here and will not be repeated below.
[0061] In the aforementioned communication systems, devices with communication capabilities can be called terminal devices, nodes, communication nodes, or communication equipment. For example, terminal devices can include independent devices such as handheld terminals, vehicles, in-vehicle equipment, network-side equipment, user equipment, access terminals, user units, user stations, mobile stations, mobile stations, remote stations, remote terminals, wireless communication equipment, user agents, or user devices, or components (such as chips or integrated circuits) included in independent devices. Terminal devices can be any possible smart terminal device (such as a mobile phone), IoT device, smart transportation equipment (such as vehicles, drones, etc.), smart manufacturing equipment, smart home devices (such as large screens, speakers, etc.), and IoT terminals, etc.
[0062] The terminal devices in this application embodiment can be applied to various application scenarios, such as the following: IoT, Industrial Internet of Things (IIoT), Mobile Internet (MI), Industrial Control, Self-driving, Transportation Safety, Smart City, or Smart Home. In certain application scenarios or network types, devices with similar communication capabilities may have other names, and this application does not impose any restrictions on this.
[0063] This application supports the SparkLink / NearLink standard protocols; it can also support IEEE protocols, such as IEEE 802.11be / Wi-Fi 7 / EHT, IEEE 802.11bn / UHR / Wi-Fi 8, IEEE Integrated mmWave / IMMW, IEEE 802.15 / UWB, or IEEE 802.11bf / sensing.
[0064] Currently, sensing technologies enable the identification of specific targets (such as objects, animals, and people) within a given environment. This includes sensing the target's distance, location, speed, movement, and behavior. In indoor environment sensing scenarios, accurate indoor environmental information can improve environmental perception performance. For example, acquiring information including the distribution of obstacles within the indoor environment allows for more accurate location of predetermined targets. Currently, the common method for acquiring indoor environmental information is to collect indoor environmental images and then obtain indoor environmental information based on these images. However, due to the dynamic nature of indoor environments—such as the movement of furniture or adjustments to the layout of items—information obtained from static indoor environmental images may be inaccurate.
[0065] Based on this, embodiments of this application provide a map updating method that can be used in a map updating device. In describing the map updating method of this application, the following example uses a map updating device performing a first map update of a target environment. A sensing device (e.g., a terminal device or access point device) in the target environment can sense objects in the target environment and obtain sensing data, where the sensing data can characterize the location of the objects in the target environment. The sensing device sends the sensing data to the map updating device, which can determine the movement trajectory of the objects in the target environment based on the sensing data received within a first time period, and update the first map of the target environment based on the movement trajectory. Thus, the map updating device can update the first map of the target environment based on the movement trajectory of the objects in the target environment, obtaining an updated second map of the target environment. When performing sensing and positioning based on this second map, accurate positioning results can be obtained.
[0066] The relevant technologies, terms, and features involved in the embodiments of this application will be explained below. It should be noted that these explanations are intended to make the embodiments of this application easier to understand and should not be regarded as limiting the scope of protection claimed by this application.
[0067] 1. Sensing technology:
[0068] The following section uses the 802.11bf scenario to introduce the relevant content of sensing technology.
[0069] Sensing measurement, also known as wireless sensing or WLAN sensing, refers to the process by which a transmitting and receiving end detects or determines the status of a target by transmitting signals. For example, it involves using wireless signals to sense environmental information (called sensing information). This information includes the distribution, size, quantity, and temperature of objects in the environment, human actions and behaviors, and even human breathing rate and heart rate. After sensing the environment, various other technologies, such as AI, can be combined for subsequent processing to reconstruct the physical environment, analyze the environment, identify and analyze people and objects within it, and trigger subsequent actions, etc.
[0070] WLAN sensing refers to a station (STA) with WLAN sensing capabilities using received WLAN signals to detect characteristic information of expected targets in a given environment. For example, characteristic information includes one or more of the following: distance, speed, angle, motion, presence or proximity, gestures, etc. Targets include one or more of the following: objects, people, animals, etc. Environment includes one or more of the following: rooms, houses, vehicles, businesses, etc.
[0071] For example, the transmitting end can send a signal for sensing and measurement to the receiving end, which can measure the signal to obtain a channel estimation result, such as Channel State Information (CSI). The receiving end can then perform sensing based on the CSI. Alternatively, the receiving end can send the channel estimation result back to the transmitting end, which can then perform target sensing or target state sensing based on the channel estimation result. For example, either the receiving end or the transmitting end can process the CSI to determine whether a moving target exists in the environment.
[0072] In the sensing and measurement process, the devices involved in sensing mainly consist of the following roles:
[0073] Sensing initiator (SI): The device that initiates the sensing measurement process and sends a sensing measurement establishment request. For non-DMG devices, the sensing initiator is the device that sends the sensing measurement establishment request frame. For DMG devices, the sensing initiator is the device that sends the DMG sensing measurement establishment request frame.
[0074] Sensing responder (SR): A device that responds to the sensing process initiated by the sensing initiator and sends a sensing measurement response. For non-DMG devices, the sensing responder refers to the device that sends a sensing measurement establishment response frame. For DMG devices, the sensing responder refers to the device that sends a DMG sensing measurement establishment response frame.
[0075] Sensing transmitter: A device that transmits sensing signals. These sensing signals can refer to signals used for sensing measurements, such as physical layer protocol data units (PPDUs). For example, sensing could be WLAN sensing or DMG sensing.
[0076] Sensing receiver: A device that receives sensing signals sent by a sensing transmitter. For example, sensing could be WLAN sensing or DMG sensing.
[0077] In the embodiments of this application, sensing can refer to WLAN sensing (such as low-frequency sub7GHz sensing) or (E)DMG sensing (i.e., high-frequency sensing, such as 60GHz sensing). Here, DMG can refer to high-frequency sensing, and this application does not limit the specific protocol used.
[0078] 2. Sensing data:
[0079] Sensing data can be data obtained by measuring a sensed signal; for example, sensing data can be data obtained by the receiver measuring a sensed signal transmitted by the transmitter. Types of sensing data include, but are not limited to: channel state information (CSI) data, path length change rate (PLCR), signal strength data (such as received signal strength indicator (RSSI), signal-to-noise ratio (SNR), etc.), and network performance data (such as throughput, bandwidth utilization, latency, packet loss rate, etc.).
[0080] 3. Edge-Cloud Collaboration: A system computing model based on collaboration between the edge and the cloud. Traditional edge-cloud collaboration primarily involves edge devices generating data or task requests, which are then uploaded to an edge computing center (comprising one or more edge servers) via the edge network. The edge computing center then executes the computational tasks. For computationally intensive and complex tasks, the edge computing center migrates these tasks upwards through the core network to the cloud computing center. After the cloud computing center completes big data analysis, the results and data are stored there. Alternatively, at least one of the following can be distributed: computation results, optimized business rules, or models. The edge computing center then transmits these results downwards through the edge network to the edge devices. The edge computing center then performs business execution and optimization based on the business rules distributed by the cloud computing center, thus achieving edge-cloud collaboration.
[0081] The edge-cloud collaboration involved in the embodiments of this application mainly refers to the edge-cloud collaboration to complete the update of indoor maps. For example, the edge collects the perception data of the indoor environment and reports it to the cloud, and the cloud updates the indoor map based on the perception data.
[0082] 4. Cloud: A cloud service cluster used to provide remote cloud services such as computing power and storage. In this embodiment, the cloud is used to update the indoor map based on the perception data reported by the edge devices. In this embodiment, the cloud can consist of one or more physical devices, which can be referred to as cloud devices or cloud servers.
[0083] 5. Edge Device: Devices located close to the user, such as terminal devices, IoT terminals, edge servers, access point devices, etc. In this embodiment, the edge device is used to collect sensing data of the indoor environment. In this embodiment, the edge device can consist of one or more physical devices, which can be called edge devices or edge device devices, such as one or more access point devices or one or more terminal devices.
[0084] In this embodiment, the target environment includes at least one access point device and at least one terminal device connected to each access point device. The access point device in the target environment can act as a sensing receiver to obtain sensing data from the target environment; or the terminal device in the target environment can act as a sensing receiver to obtain sensing data from the target environment. The access point device and the terminal device in the target environment can establish a connection using short-range communication technology. For example, the short-range communication technology in this embodiment may include: Wi-Fi communication technology, NFC communication technology, Bluetooth communication technology, RFID communication technology, NFC technology, etc. This embodiment uses Wi-Fi communication technology or NFC communication technology as examples for illustration.
[0085] 1. Wi-Fi communication technology:
[0086] Wi-Fi communication technology is a wireless communication technology based on the IEEE 802.1b standard. Commonly known as wireless broadband, it can operate in the 2.4GHz and 5GHz frequency bands. The 2.4GHz band has better penetration capabilities, making it suitable for wireless networks with wider coverage; while the 5GHz band offers higher transmission rates and better interference resistance, making it suitable for high-density wireless network environments.
[0087] A Wi-Fi communication system includes an access point (AP) and one or more STAs, as shown in Figure 2. Taking a Wi-Fi communication system with one AP and two STAs as an example, the STA associated with the AP can receive frames sent by the AP and can also send frames to the AP. This application embodiment will describe communication between the AP and STAs as an example. It is understood that this application embodiment can also be applied to communication between APs, such as communication between APs through a distributed system (DS), and can also be applied to communication between STAs.
[0088] An access point (AP) can be an access point for terminal devices (such as mobile phones) to access wired (or wireless) networks. An AP acts as a bridge connecting wired and wireless networks, its main function being to connect various wireless network clients together and then connect the wireless network to an Ethernet network. For example, an AP can be a terminal device (such as a mobile phone) with a Wi-Fi chip or a network device (such as a router). In this embodiment, the AP can be a device supporting the 802.11be standard, or it can be a device supporting various WLAN standards such as 802.11ax, 802.11ay, 802.11ac, 802.11n, 802.11g, 802.11b, 802.11a, 802.11be, 802.11bn, or the next-generation 802.11 series.
[0089] A STA can be a wireless communication chip, wireless sensor, or wireless communication terminal, and can also be referred to as a user. For example, a STA can be a mobile phone supporting Wi-Fi communication, a tablet computer supporting Wi-Fi communication, a set-top box supporting Wi-Fi communication, a smart TV supporting Wi-Fi communication, a smart wearable device supporting Wi-Fi communication, an in-vehicle communication device supporting Wi-Fi communication, and a computer supporting Wi-Fi communication, etc. Optionally, a STA can support the 802.11be standard, or it can support various WLAN standards such as 802.11ax, 802.11ay, 802.11ac, 802.11n, 802.11g, 802.11b, 802.11a, 802.11be, 802.11bn, or the next-generation 802.11 series.
[0090] Understandably, the number of APs and STAs shown in Figure 2 is just an example, and there could be more or fewer.
[0091] For example, in the embodiments of this application, the terminal device can be a STA and the access point device can be an AP.
[0092] 2. Starlight Communication Technology:
[0093] Sparklink communication technology includes Sparklink Basic (SLB) access technology and Sparklink Low Energy (SLE) access technology. Figure 3 is a schematic diagram of the communication protocol architecture of the Sparklink communication technology involved in the embodiments of this application. As shown in Figure 3, the protocol architecture includes a basic application layer, a basic service layer, and a Sparklink access layer (also called the access layer). The basic application layer and the basic service layer can be collectively referred to as the Sparklink upper layer. The various layers in the protocol architecture are described below.
[0094] Basic application layer: includes various general frameworks; in order to enable communication between different devices on different platforms, the basic application layer has defined frameworks for various possible and universally applicable application scenarios.
[0095] The basic service layer includes the control plane and the data plane. The control plane primarily provides services such as device discovery and management. The data plane includes channel control data, broadcast data, service management data, real-time data, and reliable data, as well as transmission control adaptation protocols, transmission control protocol / internet protocol (TCP / IP), and transparent transmission protocols.
[0096] Starflash Access Layer: Includes SLB module and SLE module. The SLB module can also be called SLB access layer, and the SLE module can also be called SLE access layer.
[0097] It is understood that the communication protocol architecture shown above is only one possible example, and other possible protocol layers may also be included in the communication protocol architecture. This application embodiment does not limit this.
[0098] As exemplified, Figure 4 shows a schematic diagram of the StarSpark communication system architecture. This system may include a G node and at least one T node. The G node and each accessing T node can communicate using SLB or SLE access technology. In this embodiment, the terminal device can be a T node, and the access point device can be a G node.
[0099] It should be understood that the network architecture and business scenarios described in the embodiments of this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided in the embodiments of this application. As those skilled in the art will know, with the evolution of network architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.
[0100] Figure 5 is a schematic flowchart of a map updating method provided in an embodiment of this application. The map updating method mainly includes the following steps 500-502. It is understood that the steps and execution order shown in Figure 5 are only an example. In actual implementation, some of the steps may be executed or the remaining steps may also be executed. Similarly, the execution order of the steps may also be adjusted, and this embodiment of the application does not limit this.
[0101] Step 500: The map update device acquires the first map of the target environment.
[0102] In the following description of the embodiments of this application, the map updating device updates the first map of the target environment as an example. In the embodiments of this application, the first map of the target environment can also be updated by other types of servers, such as a physical server cluster updating the first map of the target environment. The physical server cluster may include one or more physical servers.
[0103] Optionally, the target environment in this application embodiment can be a specific physical environment, such as a specific area. For example, the target environment can be an indoor environment, which can be the environment inside a building; for example, a home indoor environment, an office indoor environment, or a shopping mall indoor environment, etc.
[0104] A first map of the target environment is used to describe the target environment. Optionally, the first map can be used to describe the distribution of objects in the target environment. For example, the first map can be used to describe the distribution of stationary objects in the target environment; for example, if the target environment is a home interior environment, then the objects in the target environment can be home furnishings (such as sofas, coffee tables, TV cabinets, dining tables, study desks, beds, cabinets, chairs, air conditioners, televisions, air purifiers, etc.).
[0105] In this embodiment, objects in the target environment can be referred to as obstacles, and the first map can be used to describe the distribution of obstacles in the target environment. Optionally, the first map can be used to describe the distribution of obstacles in the space of the target environment, or the first map can be used to describe the distribution of obstacles on the ground of the target environment, or the first map can be used to describe the distribution of obstacles in both the space and on the ground of the target environment. For example, if the target environment is a home interior environment, the first map can be a floor plan of the home interior environment, or the first map can be a layout map of the home interior environment.
[0106] In one possible implementation, the map updating device can acquire a first map of the target environment configured by the user, which can be an initial environment map of the target environment.
[0107] For example, the map update device may include an API interface, through which a user terminal can configure a first map of the target environment to the map update device.
[0108] The first map of the target environment can be a plan view of the target environment captured by a camera device, or it can be a three-dimensional model generated by scanning the target environment using scanning technology.
[0109] Step 501: The map update device determines the movement trajectory of the object in the target environment based on the sensing data sent by the sensing devices in the target environment.
[0110] The map update device can receive sensing data sent by one or more sensing devices in the target environment. Taking the sensing data obtained by the sensing devices based on WLAN sensing technology as an example, the STA or AP in the target environment can obtain the sensing data and send it to the map update device.
[0111] When the map update device receives sensing data sent by the AP in the target environment, the STA connected to the AP can send a sensing signal, and the AP can measure the received sensing signal to obtain sensing data; for example, the sensing data can be CSI data obtained by the AP measuring the sensing signal.
[0112] When the map update device receives sensing data sent by the STA in the target environment, the AP can send a sensing signal, and the STA connected to the AP can measure the received sensing signal to obtain sensing data; for example, the sensing data can be CSI data obtained by the AP measuring the sensing signal.
[0113] After receiving sensing data from one or more sensing devices in the target environment, the map updating device can locate objects in the target environment. Based on the sensing data sent by one or more sensing devices over a certain period of time, the map updating device can determine the movement trajectory of objects in the target environment.
[0114] The objects in this application embodiment can be people, animals, or objects or devices that can move in the environment (such as robots, such as robot vacuum cleaners).
[0115] In the embodiments of this application, the movement trajectory of an object in the target environment can be one or more movement trajectories of one or more objects in the target environment.
[0116] In this embodiment of the application, after obtaining perception data over a certain period of time, the map updating device can determine the movement trajectory of an object in the target environment in a variety of different ways.
[0117] For example, the map updating device can determine the movement trajectory of an object in the target environment based on a passive trajectory tracking method using sensed data. Alternatively, the map updating device can also determine the movement trajectory of an object in the target environment based on a Wi-Fi fingerprint positioning method.
[0118] The following section uses the passive trajectory tracking method based on sensing data of the map update device as an example for introduction.
[0119] Passive trajectory tracking method 1: Map update device based on PLCR model-driven algorithm to determine the movement trajectory of the object in the target environment.
[0120] The following section, with reference to Figure 6, describes the process by which the map update device uses a PLCR-based model-driven algorithm to determine the movement trajectory of an object in the target environment.
[0121] In this passive trajectory tracking method 1, CSI data is used as the sensing data as an example. For instance, as shown on the left side of Figure 6, the target environment includes multiple sensing devices (APs and terminal devices as shown in Figure 6) and one or more objects (people in the target environment as shown in Figure 6). The sensing devices can collect sensing data from the target environment, and based on this sensing data, they can sense the objects in the target environment. The sensing devices in the target environment (terminal devices or APs as shown in Figure 6) send the collected sensing data to the map updating device.
[0122] The map updating device preprocesses CSI data over a certain time period (e.g., performing noise reduction and phase calibration). For example, it analyzes the phase change over time in the preprocessed CSI data, and by calculating the frequency of this change, the Doppler frequency shift can be obtained. Furthermore, the map updating device can calculate the path length change rate (PLCR) based on the CSI data. For instance, in the feature analysis stage shown in Figure 6, the map updating device can determine the CSI ratio of the CSI data over a certain time period. Since the CSI data contains amplitude and phase information of multiple subcarriers, this CSI bit can represent the CSI ratio between different subcarriers. In the feature analysis stage shown in Figure 6, the map updating device can filter the determined CSI ratio using a filter and perform a short-time Fourier transform (STFT) to obtain the path length change rate (PLCR).
[0123] The map update device analyzes the amplitude of CSI and determines the time when objects in the target environment begin to move and the time when they stop moving based on the CSI amplitude.
[0124] As shown in Figure 6, the map updating device determines the object's moving speed based on the path length change rate (PLCR) and Doppler shift. It also determines the subcarrier delay using CSI data and estimates the object's moving direction based on this delay. Based on the determined object's moving speed, moving direction, and the times when the object starts and stops moving, the map updating device updates the object's position in the target environment. The map updating device corrects for errors using Kalman filtering, thereby obtaining the object's trajectory in the target environment.
[0125] Passive trajectory tracking method 2: The map update device uses a data-driven algorithm based on PLCR to determine the movement trajectory of the object in the target environment.
[0126] For example, this data-driven algorithm can be understood as determining the movement trajectory of an object in a target environment based on a trained second network model.
[0127] In this passive trajectory tracking method 1, CSI data is used as the sensing data as an example. For instance, as shown on the left side of Figure 7, the target environment includes multiple sensing devices (APs and terminal devices in Figure 7) and one or more objects (people in the target environment as shown in Figure 7). The sensing devices can collect sensing data from the target environment, and based on this sensing data, they can sense the objects in the target environment. The sensing devices in the target environment (terminal devices or APs in Figure 7) send the collected sensing data to the map updating device.
[0128] The map updating device preprocesses CSI data over a certain time period (e.g., performing noise reduction and phase calibration). For example, it analyzes the phase change over time in the preprocessed CSI data, and calculates the Doppler frequency shift by determining the frequency of this change. Furthermore, the map updating device can calculate the path length change rate (PLCR) based on the CSI data. For instance, in the feature analysis stage shown in Figure 7, the map updating device can determine the CSI ratio of the CSI data over a certain time period. Since the CSI data contains amplitude and phase information of multiple subcarriers, this CSI bit can represent the CSI ratio between different subcarriers. In the feature analysis stage shown in Figure 7, the map updating device can filter the determined CSI ratio using a filter and perform a short-time Fourier transform (STFT) to obtain the path length change rate (PLCR).
[0129] The map update device calls the trained second network model, inputs the path length change rate (PLCR) and the initial position of the object into the second network model, and obtains the movement trajectory of the object in the target environment output by the second network model.
[0130] For example, the second network model can be a long short-term memory (LSTM) network model.
[0131] During the training of the second network model, PLCR-Location datasets with different device layouts and user trajectories can be collected to improve the model's generalization ability; the second network model can be trained to learn the model's mapping relationship to improve tracking accuracy; and the training dataset can be used to train the network model to fit the motion characteristics of the hypercurve model.
[0132] It should be noted that the two methods for determining the movement trajectory of an object in a target environment described above are merely examples of embodiments of this application. The methods for determining the movement trajectory of an object in a target environment in this application are not limited to the examples described above. For example, a map update device can also determine the movement trajectory of an object in a target environment based on a Wi-Fi fingerprint positioning method. In addition, the CSI data and PLCR data used in the two methods for determining the movement trajectory of an object in a target environment described above are just examples. Embodiments of this application can also use other types of sensing data when determining the movement trajectory of an object in a target environment.
[0133] Step 502: The map updating device updates the first map according to the movement trajectory.
[0134] In this embodiment, after determining the movement trajectory of an object in the target environment, the map updating device can update the first map using various different methods. For example, the map updating device can adjust the image information of the first map based on the trajectory points mapped onto the first map according to the movement trajectory, thereby updating the first map. As another example, the map updating device can generate a trajectory map of the target environment based on the movement trajectory, and predict the obstacle distribution map of the target environment based on the trajectory map, thereby updating the first map based on the predicted obstacle distribution map.
[0135] The following section details different methods for updating the first map.
[0136] Option 1: The map updating device adjusts the image information of the first map based on the trajectory points mapped onto the first map according to the movement trajectory, and updates the first map.
[0137] In Scheme 1 of this application embodiment, after determining the movement trajectory of an object in the target environment, the map update device of this application embodiment can determine whether the first map needs to be updated.
[0138] In one possible implementation, the map updating device determines whether there are obstacles on the movement trajectory based on the first map; if there are obstacles on the movement trajectory, the first map is updated.
[0139] Since the first map in this embodiment is used to describe the target environment, for example, the first map is used to describe the distribution of obstacles in the target environment. Based on the first map and the movement trajectory of the object, it can be determined whether there are obstacles on the movement trajectory. When the map updating device determines that there are obstacles on the movement trajectory, it can determine that the first map needs to be updated.
[0140] The following section details the scheme by which the map update device determines whether the first map needs to be updated.
[0141] Optionally, the map updating device determines at least one trajectory point in the first map, wherein the trajectory point is a location point on the first map mapped from the movement trajectory; the map updating device determines whether there is an obstacle on the movement trajectory based on the at least one trajectory point.
[0142] In this embodiment of the application, when the map updating device determines the movement trajectory of an object in the target environment, it generates the movement trajectory based on the object's location obtained by locating the object.
[0143] In some embodiments, the map updating device can locate an object based on each sensing data to obtain an object location; the map updating device can obtain multiple object locations based on multiple sensing data within a certain time period, and can obtain the movement trajectory of the object in the target environment based on the multiple object locations.
[0144] Based on this, the movement trajectory includes multiple object locations, and the map updating device can determine that each object location in the movement trajectory is mapped to a trajectory point on the first map. After determining at least one trajectory point in the first map, the map updating device can determine whether there are obstacles on the movement trajectory based on at least one trajectory point in the first map.
[0145] In some embodiments, after acquiring a first map of the target environment, the map updating device may perform an initialization process on the first map. During the initialization process, the map updating device may determine a first parameter corresponding to each image region in the first map, wherein the first parameter is used to characterize the probability that the environment corresponding to the image region includes obstacles.
[0146] In specific implementation, after acquiring a first map of the target environment, the map updating device identifies obstacles in the target environment. The map updating device can divide the first map into multiple image regions. For each image region, it determines a first parameter corresponding to that image region based on whether the environment corresponding to that image region contains obstacles. For example, when the environment corresponding to an image region contains obstacles, the map updating device sets the first parameter of that image region to a first value; when the environment corresponding to an image region does not contain obstacles, the map updating device sets the first parameter of that image region to a second value. For instance, the map updating device sets the first parameter of an image region in the first map whose environment contains obstacles to 0.5, indicating that the environment corresponding to that image region contains obstacles; the map updating device sets the first parameter of an image region in the first map whose environment does not contain obstacles to 0.25, indicating that the environment corresponding to that image region does not contain obstacles.
[0147] When determining whether there are obstacles on a movement trajectory, the map updating device may determine whether there are obstacles on the movement trajectory based on the first parameter corresponding to the image area where each trajectory point is located.
[0148] Optionally, when the map update device determines that the average value of the first parameter corresponding to the image area where at least one trajectory point is located is greater than a set threshold, it determines that there is an obstacle on the movement trajectory.
[0149] For example, when the map updating device determines n trajectory points in the first map based on the movement trajectory, where n is an integer greater than or equal to 1, the map updating device determines the first parameter P(x,y) corresponding to the image region where each trajectory point is located; the map updating device determines the average parameter corresponding to the n trajectory points. The average parameter represents the average probability that the environment corresponding to the image region where the n trajectory points are located includes obstacles.
[0150] Wherein, P(x i ,y i Let be the first parameter of the image region containing the i-th trajectory point out of n trajectory points, where n is the number of trajectory points corresponding to the movement trajectory. Let be the average parameter corresponding to n trajectory points.
[0151] In this embodiment of the application, the map updating device determines the average parameters corresponding to n trajectory points. When the value exceeds a set threshold, the map update device determines that there is an obstacle on the movement trajectory. For example, the set threshold can be 0.45.
[0152] When the map updating device determines that there are obstacles on the movement trajectory, it updates the first map.
[0153] The following describes the scheme for updating the first map using the map updating device in Scheme 1.
[0154] In one possible implementation, the map updating device obtains the first parameter corresponding to each image region in the first map, and adjusts the first parameter corresponding to each image region according to whether each image region includes trajectory points; the map updating device determines the image information corresponding to each image region based on the adjusted first parameter corresponding to each image region; the map updating device generates a second map updated to the first map based on the image information corresponding to each image region.
[0155] The following details the solution for updating the first map.
[0156] 1. The map updating device obtains the first parameter corresponding to each image region in the first map, and adjusts the first parameter corresponding to each image region according to whether each image region includes trajectory points.
[0157] The map updating device adjusts the first parameter corresponding to each image region based on whether each image region in the first map contains trajectory points.
[0158] For an image region in the first map that includes trajectory points, the map updating device reduces the first parameter corresponding to the image region that includes trajectory points.
[0159] For example, the map updating device adjusts the first parameter corresponding to the image region including the trajectory points according to the following formula: P′(x,y)=P(x,y)×0.5;
[0160] Wherein, P(x,y) is the first parameter corresponding to the image region containing the trajectory points in the first map before adjustment, and P′(x,y) is the first parameter corresponding to the image region containing the trajectory points in the first map after adjustment.
[0161] For image regions in the first map that do not include trajectory points, the map updating device adjusts the first parameter corresponding to the image region based on whether the adjacent image regions of the image region include trajectory points.
[0162] For example, the more first image regions there are among multiple adjacent image regions that do not include trajectory points, the smaller the first parameter corresponding to the image region that does not include trajectory points; wherein the first image region is the image region that includes trajectory points among the adjacent image regions.
[0163] For example, for image regions in the first map that do not include trajectory points:
[0164] If none of the adjacent image regions of the image region contain trajectory points, then the first parameter corresponding to the image region can remain unchanged;
[0165] If the number of adjacent image regions containing trajectory points is 1, the map updating device can adjust the first parameter corresponding to the image region according to the following formula: P′(x,y)=P(x,y)×0.8;
[0166] Wherein, P(x,y) is the first parameter corresponding to the image region in the first map before adjustment that does not include the trajectory points, and P′(x,y) is the first parameter corresponding to the image region in the first map after adjustment that does not include the trajectory points.
[0167] If the number of adjacent image regions containing trajectory points is 2, the map updating device can adjust the first parameter corresponding to the image region according to the following formula: P′(x,y)=P(x,y)×0.8 2 ;
[0168] Wherein, P(x,y) is the first parameter corresponding to the image region in the first map before adjustment that does not include the trajectory points, and P′(x,y) is the first parameter corresponding to the image region in the first map after adjustment that does not include the trajectory points.
[0169] If the number of adjacent image regions containing trajectory points is greater than or equal to 3, the map update device can adjust the first parameter corresponding to the image region according to the following formula: P′(x,y)=P(x,y)×0.5;
[0170] Wherein, P(x,y) is the first parameter corresponding to the image region in the first map before adjustment that does not include the trajectory points, and P′(x,y) is the first parameter corresponding to the image region in the first map after adjustment that does not include the trajectory points.
[0171] 2. The map updating device determines the image information corresponding to each image region based on the adjusted first parameter corresponding to each image region.
[0172] Optionally, the image information can be grayscale values.
[0173] The map updating device can determine the grayscale value corresponding to each image region based on the adjusted first parameter corresponding to each image region in the first map.
[0174] For example, the gray value corresponding to each image region and the adjusted first parameter can have the following relationship: G′(x,y)=1-P′(x,y);
[0175] Where G′(x,y) is the gray value corresponding to the image region, and P′(x,y) is the adjusted first parameter corresponding to the region.
[0176] After the map update device determines the grayscale value corresponding to each image region in the first map, it can compare the grayscale value with the grayscale threshold. If the grayscale value corresponding to the image region is greater than the grayscale threshold, the grayscale value corresponding to the image region is assigned a value of 1, indicating that there are no obstacles in the environment corresponding to the image region. If the grayscale value corresponding to the image region is not greater than the grayscale threshold, the grayscale value corresponding to the image region is assigned a value of 0, indicating that there are obstacles in the environment corresponding to the image region.
[0177] 3. The map updating device generates a second map that updates the first map based on the image information corresponding to each image area.
[0178] When the image information is in grayscale, the map updating device generates an updated second map based on the grayscale value corresponding to each image region in the first image after determining the grayscale value corresponding to each image region.
[0179] Option 2: The map updating device predicts the obstacle distribution map of the target environment and updates the first map based on the predicted obstacle distribution map.
[0180] In one possible implementation, the map updating device generates a trajectory map corresponding to the target environment based on the movement trajectory, wherein the trajectory map is used to describe the number of times the trajectory is covered for each area in the target environment; the map updating device predicts the obstacle distribution map of the target environment based on the trajectory map; and the map updating device updates the first map based on the obstacle distribution map.
[0181] The following details the specific scheme for updating the first map using Scheme 2.
[0182] First, the map updating device generates a trajectory map corresponding to the target environment based on the movement trajectory.
[0183] After determining the movement trajectory of an object in the target environment, the map updating device of this application can map the object's movement trajectory onto a first map to obtain a map to be processed containing trajectory data. For example, as shown in FIG8, the map to be processed includes multiple trajectory data.
[0184] After obtaining the map to be processed containing trajectory data, the map updating device can perform image segmentation processing on the map to be processed, dividing it into multiple image regions. For example, the map updating device can evenly divide the map to be processed into multiple image regions, as shown in Figure 9, where each image region is a grid in Figure 9.
[0185] The map updating device generates a trajectory map corresponding to the target environment based on the unprocessed map containing multiple image regions; wherein, the trajectory map is used to describe the number of times the trajectory covers each region in the target environment, and each image region in the trajectory map corresponds to a region in the target environment.
[0186] When generating a trajectory map corresponding to a target environment, the map updating device can count the number of times the trajectory is covered in each image region of the map to be processed, and generate a trajectory map corresponding to the target environment based on the number of times the trajectory is covered in each image region. For example, as shown in Figure 10, the trajectory map corresponding to the target environment generated by the map updating device contains the number of times the trajectory is covered in each image region.
[0187] Then, the map update device predicts the obstacle distribution map of the target environment based on the trajectory map.
[0188] In this embodiment of the application, the map update device can predict the obstacle distribution map of the target environment based on the trained first network model.
[0189] Optionally, the map update device inputs the trajectory map into the trained first network model to obtain the obstacle distribution map of the target environment output by the first network model; wherein, the first network model is trained based on a training sample set, which includes multiple training samples, each training sample including a trajectory sample map and an obstacle distribution sample map.
[0190] In this embodiment, the map updating device can acquire one or more movement trajectories of objects in the target environment. These multiple movement trajectories can be the movement trajectories of one or more objects in the target environment. The map updating device generates a trajectory map corresponding to the target environment based on these one or more movement trajectories (e.g., the trajectory map shown in Figure 10). The map updating device uses this trajectory map as input to a trained first network model, performs inference based on the trained first network model, and obtains an obstacle distribution map predicted from the trajectory map.
[0191] For example, the first network model in this application embodiment can be an image input-image output model (such as U-Net) network model.
[0192] After obtaining an obstacle distribution map of the target environment based on a trained first network model, the map updating device in this application can determine whether to update the first map based on the obstacle distribution map output by the network model and the first map.
[0193] Optionally, if there are differences between the obstacle distribution map and the first map, the first map is updated.
[0194] In some embodiments, the map updating device may determine whether there is a difference between the obstacle distribution map predicted by the first network model and the first map using various different methods. For example, the map updating device may calculate the similarity between the obstacle distribution map and the first map, and determine that there is a difference between the obstacle distribution map predicted by the first network model and the first map when the similarity is less than a first threshold.
[0195] The training process of the first network model is described below. For example, in this embodiment, the training process of the first network model can be performed by a training device; optionally, the training device can be a map update device, or the training device can be deployed on the map update device, or the training device can be a device independent of the map update device, or the training device can be deployed on other devices besides the map update device.
[0196] The training process of the first network model in this application embodiment may include multiple stages; for example, the training process of the first network model includes a training sample acquisition stage and a model training stage.
[0197] 1. Training device acquires training samples stage.
[0198] During the training sample acquisition phase, the training device can collect obstacle distribution sample maps and trajectory sample maps corresponding to multiple different environments, thereby obtaining a training sample set including multiple training samples. Each training sample includes obstacle distribution sample maps and trajectory sample maps corresponding to the same environment.
[0199] For a given environment (e.g., an indoor environment), the training device acquires perception data reported by sensing devices in that environment over a certain period of time, and determines multiple movement trajectories of one or more objects within that environment. Optionally, to improve the accuracy of the obstacle distribution map prediction by the trained first network model, the multiple movement trajectories acquired by the training device can cover the entire obstacle-free area of the environment.
[0200] The training device acquires an initial map corresponding to the environment. The initial map is used to represent the distribution of obstacles in the environment; the initial map can also be called the obstacle distribution sample map of the environment.
[0201] After the training device determines multiple movement trajectories of an object in the environment, it maps these trajectories onto an initial map, obtaining a map sample containing the trajectory data. The training device then divides this map sample into multiple image regions and counts the number of trajectory coverages in each region. Based on the trajectory coverage counts in each region, it generates trajectory samples corresponding to that environment. It should be understood that the process by which the training device generates trajectory samples for a given environment is the same as the process described above where the map updating device generates a trajectory map corresponding to the target environment.
[0202] The training device obtains a training sample for an environment. The training sample includes an obstacle distribution sample map and a trajectory sample. The obstacle distribution sample map in the training sample can be the initial map corresponding to the environment.
[0203] Based on this, the training device can obtain multiple training samples for different environments, and these multiple training samples can form a training sample set. The training device can then train the initial first network model based on this training sample set, thereby obtaining a trained first network model.
[0204] 2. The initial stage of training the network model using the training device.
[0205] The training device will input multiple training samples from the training sample set obtained during the training sample acquisition phase into the initial first network model to train the initial first network model.
[0206] During the training of the first network model, the training device predicts the predicted obstacle distribution map for each trajectory sample in the training sample set. Based on a defined loss function, the training device determines the loss value between the predicted obstacle distribution map and the obstacle distribution sample map predicted by the first network model. The training device can adjust the model parameters of the first network model based on the determined loss value. After adjusting the model parameters of the first network model, the training device can retrain the adjusted first network model. After multiple rounds of training and adjustment, when the training device determines that the loss value between the predicted obstacle distribution map and the obstacle distribution sample map meets the conditions, the training device can obtain the trained first network model.
[0207] During the training of the first network model, in order to improve the accuracy of the first network model in predicting the obstacle distribution map, the training device uses a suitable loss function to train the first network model. For example, since the first network model's prediction of the obstacle distribution map is a binary classification problem, that is, determining whether each region in the image is the location of an obstacle, the loss function in this embodiment can be a binary cross-entropy loss function. When updating the model parameters of the first network model, the training device can use the gradient calculated by the backpropagation algorithm and select a suitable optimizer to update the model parameters of the first network model.
[0208] In some embodiments, the training device can determine the loss value between the predicted obstacle distribution map and the obstacle distribution sample map based on a defined loss function.
[0209] For example, for each sampling point in the predicted obstacle distribution map, a first value P′ of the sampling point in the predicted obstacle distribution map is determined. The first value P′ represents the probability that the environmental location corresponding to the sampling point is an obstacle, as predicted by the first network model. If the environmental location corresponding to the sampling point is predicted to be an obstacle, the first value P′ is 1; if the environmental location corresponding to the sampling point is predicted to be free of obstacles, the first value P′ is 0. A second value P of the sampling point in the obstacle distribution map is also determined. The second value P represents the probability that the environmental location corresponding to the sampling point is an obstacle in the actual environment. If the environmental location corresponding to the sampling point is an obstacle in the actual environment, the second value P is 1; if the environmental location corresponding to the sampling point is free of obstacles in the actual environment, the second value P is 0.
[0210] The training device uses the first value P′ of each sampling point in the predicted obstacle distribution map and the second value P in the obstacle distribution sample map to determine the loss value between the predicted obstacle distribution map and the obstacle distribution sample map.
[0211] For example, the loss function is defined as follows:
[0212] Where L(P,P′) is the cross-entropy loss function corresponding to each sampling point, and M is the number of sampling points in the predicted obstacle distribution map or obstacle distribution sample map.
[0213] It should be understood that each sampling point in the predicted obstacle distribution map or obstacle distribution sample map can be a single pixel or an image region containing multiple pixels.
[0214] The definition of L(P,P′) can be as follows:
[0215] L(P,P′)=-(Pln(P′)+(1-P)ln(1-P′); The definitions of P and P′ can be found in the introduction above.
[0216] In this embodiment, when adjusting the model parameters of the first network model, the training device adjusts the model parameters based on the selected optimizer. Through multiple rounds of training and adjustments, the loss value of the loss function is continuously reduced, thereby achieving convergence of the first network model. For example, the optimizer used to adjust the model parameters of the first network model in this embodiment can be an adaptive moment estimation (Adam) optimizer or a stochastic gradient descent (SGD) optimizer.
[0217] To further optimize the trained first network model and avoid missed detections and false alarms in the predicted obstacle distribution map, the following measures are taken: For example, a missed detection occurs when the first network model identifies an area not covered by a trajectory sample as an obstacle region, even though this area might be an obstacle edge or an area rarely moved by the object. A false alarm occurs when, due to positioning or trajectory tracking errors, the object's trajectory passes through an obstacle, and the first network model classifies this obstacle-covered area as an obstacle-free area.
[0218] To avoid the aforementioned issues of missed detections and false alarms, the training device counts the cumulative number of times an object's trajectory is covered within a certain period when acquiring trajectory samples. In practice, the training device can use either a fixed time window or a sliding time window to collect the cumulative number of times an object's trajectory is covered. For example, as shown in Figure 11, for each image region in the obstacle distribution sample map, the cumulative number of times an object's trajectory is covered within that image region over a certain period (Figure 11 uses 5 days as an example).
[0219] In training the first network model, in order to improve the model accuracy and reduce the probability of missed detections and false alarms, the embodiments of this application can formulate corresponding constraint rules; the constraint rules stipulate that the duration and the cumulative number of trajectory coverages are used as the basis for determining whether to update the map.
[0220] For example, the constraint rules define the image regions that do not need to be updated, and the image regions that need to be updated.
[0221] For example, constraint rules define image regions that do not need to be updated:
[0222] For an image region without obstacles in the obstacle distribution sample map, if the cumulative number of trajectory coverages within a first time period is not greater than a second threshold, then the image region is determined to be an obstacle-free region. The first time period is not greater than a third threshold, and the first time period can be a relatively short period of time. For example, if the first time period is 2 days and the second threshold is 3, then if the cumulative number of trajectory coverages within 2 days is not greater than 3, then the image region is determined to be an obstacle-free region.
[0223] For an image region containing obstacles in the obstacle distribution sample map, if the cumulative number of trajectory coverages within the first time period is not greater than the fourth threshold, then the image region is determined to be an obstacle region. For example, if the first time period is 2 days and the second threshold is 2, then if the cumulative number of trajectory coverages within 2 days is not greater than 2, then the image region is determined to be an obstacle region.
[0224] For example, constraint rules define the image regions that need to be updated:
[0225] For image regions without obstacles in the obstacle distribution sample map, if the cumulative trajectory coverage count is 0 within a second time period, the image region is determined to be an obstacle region. The second time period is not less than the fifth threshold, and the second time period can be a relatively long period of time, which is longer than the first time period. For example, if the second time period is 4 days, then if the cumulative trajectory coverage count is 0 within 4 days, it means that no object has passed through the region within 4 days, and the image region is determined to be an obstacle region.
[0226] For image regions with obstacles in the obstacle distribution sample map, if the cumulative number of trajectory coverages within the second time period is greater than the sixth threshold, then the image region is determined to be an obstacle-free region. Here, the second time period is not less than the fifth threshold, the second time period can be a relatively long period of time, the second time period is greater than the first time period, and the sixth threshold is a large value, indicating that no object has passed through the region for a relatively long period of time; for example, if the second time period is 4 days and the sixth threshold is 4, it means that the region has more than 4 trajectory coverages within 4 days, and the image region is determined to be an obstacle-free region.
[0227] The embodiments of this application can incorporate the above-mentioned constraint rules in various ways. These will be described in detail below.
[0228] Rule integration method 1:
[0229] In this integration method, the training device can incorporate constraint rules during the training sample acquisition stage.
[0230] During the training sample acquisition phase, the training device can generate training samples according to the aforementioned constraint rules. When generating the obstacle distribution sample map in the training samples, the training device marks whether there are obstacles in the image region based on the cumulative trajectory coverage times within a certain time period in each image region.
[0231] As described in the constraint rules above, if an image region in the obstacle distribution sample map satisfies the definition of an image region that does not need to be updated, such as an image region in the obstacle distribution sample map that is free of obstacles and whose cumulative trajectory coverage count within the first time period is no greater than the second threshold, then that image region is marked as "obstacle-free". For example, for an image region in the obstacle distribution sample map, if the cumulative trajectory coverage count within 2 days is no greater than 3, then that image region in the obstacle distribution sample map is marked as "obstacle-free".
[0232] If an image region in the obstacle distribution sample map meets the constraints defined as an image region that needs to be updated; for example, if an image region in the obstacle distribution sample map is an image region with obstacles, and the cumulative number of trajectory coverages within the second time period is greater than the sixth threshold, then the image region is marked as "no obstacles"; for example, for an image region in the obstacle distribution sample map, if the image region has more than 4 trajectory coverages within 4 days, then the image region in the obstacle distribution sample map is marked as "no obstacles".
[0233] Rule integration method 2:
[0234] In this integration method, the training device can be designed with a loss function that includes the above-mentioned constraint rules, so that the output of the first network model is more in line with actual applications.
[0235] For example, by adding a rule-based penalty term to the regular loss function, a penalty is imposed on the model's predictions when the cumulative number of trajectory coverages is low within a short time period but the model is predicted as an obstacle region, or when the cumulative number of trajectory coverages is close to 0 within a longer time period but the model is not predicted as an obstacle. The importance of the rule-based loss term can be adjusted through weights, thereby forcing the first network model to satisfy the above-mentioned constraint rules during training.
[0236] Rule integration method 3:
[0237] In this integration method, the training device adjusts the model architecture of the first network model so that the adjusted first network model can satisfy the aforementioned constraint rules. For example, an adjustment module based on constraint rules is added to the first network model. This adjustment module can adjust the obstacle distribution map predicted or inferred by the first network model based on the defined constraint rules.
[0238] Based on this integration method, before the first network model acquires the trajectory map of the input target environment for inference or prediction and outputs the obstacle distribution map, the adjustment module determines, based on the aforementioned constraint rules, whether to update each image region in the obstacle distribution map. For example, if an image region in the obstacle distribution map is defined by the constraint rules as an image region that does not need to be updated, then that image region does not need to be updated; if an image region in the obstacle distribution map is defined by the constraint rules as an image region that needs to be updated, then that image region needs to be updated.
[0239] Rule integration method 4:
[0240] In this integration approach, the first network model is trained using a generative adversarial network (GAN) training method. During training, the generator generates a predicted obstacle distribution map based on the trajectory sample map, and the discriminator determines whether the generated predicted obstacle distribution map conforms to the aforementioned constraint rules. Based on this training method, the generator can continuously optimize its output, ensuring that the predicted obstacle distribution map output by the generator satisfies the constraint rules of the discriminator.
[0241] Figure 12 illustrates a possible exemplary block diagram of the map updating device involved in the embodiments of this application. As shown in Figure 12, the map updating device 1200 may include modules or units for implementing the method embodiments described above. In one possible design, the map updating device 1200 includes a processing unit 1201 and a communication unit 1202. Optionally, the map updating device 1200 may further include a storage unit 1203 for storing device program code and / or data.
[0242] The map update device 1200 can be a map update device side device in the above embodiments, such as a map update device, a module (e.g., circuit, chip or chip system) in the map update device, or a logic node, logic module or software that can realize all or part of the functions of the map update device.
[0243] For example, in one embodiment, the communication unit 1202 is used to communicate with a sensing device in the target environment. The processing unit 1201 is used to acquire a first map of the target environment, the first map being used to describe the target environment; determine the movement trajectory of an object in the target environment based on sensing data sent by the sensing device in the target environment; and update the first map based on the movement trajectory.
[0244] In one possible implementation, the processing unit 1201 is configured to determine whether there are obstacles on the movement trajectory based on the first map; and update the first map when there are obstacles on the movement trajectory.
[0245] In one possible implementation, the processing unit 1201 is configured to determine at least one trajectory point in the first map, the trajectory point being a location point on the first map mapped from the movement trajectory; and to determine whether there is an obstacle on the movement trajectory based on the at least one trajectory point.
[0246] In one possible implementation, the processing unit 1201 is used to determine a first parameter corresponding to the region where each trajectory point is located, the first parameter being used to characterize the probability that the corresponding region includes an obstacle; and to determine whether there is an obstacle on the movement trajectory based on the first parameter corresponding to the region where each trajectory point is located.
[0247] In one possible implementation, the processing unit 1201 is configured to obtain a first parameter corresponding to each region in the first map, the first parameter being used to characterize the probability that the corresponding region includes obstacles; adjust the first parameter corresponding to each region according to whether each region includes trajectory points, the trajectory points being the position points of the movement trajectory mapped onto the first map; determine the image information corresponding to each region according to the adjusted first parameter corresponding to each region; and generate a second map updated to the first map according to the image information corresponding to each region.
[0248] In one possible implementation, the processing unit 1201 is configured to generate a trajectory map corresponding to the target environment based on the movement trajectory, the trajectory map being used to describe the number of times the trajectory is covered for each area in the target environment; predict an obstacle distribution map of the target environment based on the trajectory map; and update the first map based on the obstacle distribution map.
[0249] In one possible implementation, the processing unit 1201 is used to input the trajectory map into a trained first network model to obtain an obstacle distribution map of the target environment output by the first network model; wherein the first network model is trained based on a training sample set, the training sample set includes multiple training samples, and each training sample includes a trajectory sample map and an obstacle distribution sample map.
[0250] In one possible implementation, the processing unit 1201 is used to update the first map if there is a difference between the obstacle distribution map and the first map.
[0251] It is understood that the division of units in the above-described device is merely a logical functional division. One function can correspond to one functional unit, or two or more functions can be integrated into one functional unit. In actual implementation, all or some units can be integrated onto a single physical entity, or distributed across different physical entities. Furthermore, the aforementioned functional units can be implemented in hardware, software, or a combination of both. Whether a function is executed in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for specific applications, but such implementations should not be considered beyond the scope of this application.
[0252] In one example, the functional unit in any of the above devices may be one or more integrated circuits configured to implement the above methods, such as: one or more application-specific integrated circuits (ASICs), or one or more central processing units (CPUs), one or more microcontroller units (MCUs), one or more digital signal processors (DSPs), or one or more field-programmable gate arrays (FPGAs), or a combination of at least two of these integrated circuit forms.
[0253] In one example, storage unit 1203 may include random access memory, flash memory, read-only memory, programmable read-only memory or electrically erasable programmable memory and / or registers, etc.
[0254] Figure 13 illustrates a possible exemplary block diagram of a map updating apparatus according to an embodiment of this application. The map updating apparatus 1300 shown in Figure 13 includes a processor 1310 and a communication interface 1320. The processor 1310 and the communication interface 1320 are coupled to each other. It is understood that the communication interface 1320 can be a transceiver or an input / output interface. Optionally, the map updating apparatus 1300 may further include a memory 1330 for storing instructions executed by the processor 1310, or storing input data required for the processor 1310 to execute instructions, or storing data generated after the processor 1310 executes instructions.
[0255] When the map update device 1300 is used to implement the above method embodiment, the processor 1310 is used to implement the function of the processing unit 1201, and the communication interface 1320 is used to implement the function of the communication unit 1202.
[0256] It is understood that the processor in the embodiments of this application can be a CPU, or other general-purpose processors, DSPs, ASICs, FPGAs, or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. A general-purpose processor can be a microprocessor or any conventional processor.
[0257] The method steps in the embodiments of this application can be implemented in hardware or by a processor executing software instructions. The software instructions can consist of corresponding software modules, which can be stored in random access memory, flash memory, read-only memory, programmable read-only memory, erasable programmable read-only memory, electrically erasable programmable read-only memory, registers, hard disks, portable hard disks, compact disc read-only memory (CD-ROM), or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and storage medium can reside in an ASIC. Alternatively, the ASIC can reside in a map updating device. Of course, the processor and storage medium can also exist as discrete components in the map updating device.
[0258] This application also provides a computer-readable storage medium storing a computer program or instructions for implementing the method executed by the map updating device in the above method embodiments.
[0259] For example, when the computer program or instructions are executed by the computer, the computer can implement the method performed by the map updating device in the above method embodiments.
[0260] This application also provides a computer program product containing a computer program or instructions, which, when executed by a computer, causes the computer to implement the method performed by the map updating device in the above method embodiments.
[0261] This application also provides a communication system, which includes the map update device, access point device and terminal device described in the above embodiments.
[0262] This application also provides a chip device, including a processor, for calling computer programs or computer instructions stored in the memory to cause the processor to execute the method provided in any of the above embodiments.
[0263] In one possible implementation, the input of the chip device corresponds to the receiving operation in any of the above embodiments, and the output of the chip device corresponds to the sending operation in any of the above embodiments.
[0264] Optionally, the processor is coupled to the memory via an interface.
[0265] Optionally, the chip device may also include a memory in which computer programs or instructions are stored.
[0266] The processor mentioned above can be a general-purpose central processing unit, a microprocessor, an ASIC, or one or more integrated circuits used to control the execution of a program that controls the methods provided in any of the above embodiments. The memory mentioned above can be read-only memory (ROM) or other types of static storage devices capable of storing static information and instructions, such as random access memory (RAM).
[0267] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer programs or instructions. A computer program is a set of instructions that directs each step of an action of an electronic computer or other device with message processing capabilities. It is typically written in a programming language and runs on a target architecture. When the computer program or instructions are loaded and executed on a computer, the processes or functions described in the embodiments of this application are performed, in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer program or instructions can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another. For example, the computer program or instructions can be transferred from one website, computer, server, or data center to another website, computer, server, or data center via wired or wireless means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium, such as a floppy disk, hard disk, or magnetic tape; it can also be an optical medium, such as a digital video optical disc; or it can be a semiconductor medium, such as a solid-state drive. The computer-readable storage medium can be volatile or non-volatile, or it can include both types of storage media.
[0268] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, optical storage, etc.) containing computer-usable program code.
[0269] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in one or more blocks of the flowchart illustrations and / or one or more blocks of the block diagrams.
[0270] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement the functions specified in one or more flowcharts and / or one or more block diagrams.
[0271] These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, thereby providing steps for implementing the functions specified in one or more flowcharts and / or one or more block diagrams.
[0272] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.
Claims
1. A map updating method, characterized in that, The method includes: Obtain a first map of the target environment, the first map being used to describe the target environment; Based on the sensing data sent by the sensing devices in the target environment, determine the movement trajectory of the object in the target environment; The first map is updated based on the movement trajectory.
2. The method as described in claim 1, characterized in that, The step of updating the first map based on the movement trajectory includes: Based on the first map, determine whether there are obstacles on the movement trajectory; When an obstacle is present on the movement trajectory, the first map is updated.
3. The method as described in claim 2, characterized in that, Determining whether there are obstacles on the movement trajectory based on the first map includes: Determine at least one trajectory point in the first map, wherein the trajectory point is the location point of the movement trajectory mapped onto the first map; Based on the at least one trajectory point, determine whether there is an obstacle on the movement trajectory.
4. The method as described in claim 3, characterized in that, Determining whether there is an obstacle on the movement trajectory based on the at least one trajectory point includes: Determine a first parameter corresponding to the region where each trajectory point is located, the first parameter being used to characterize the probability that the corresponding region includes obstacles; Based on the first parameter corresponding to the region where each trajectory point is located, it is determined whether there are obstacles on the movement trajectory.
5. The method according to any one of claims 1 to 4, characterized in that, The updating of the first map includes: Obtain the first parameter corresponding to each region in the first map. The first parameter is used to characterize the probability that the corresponding region includes obstacles. Depending on whether each region includes trajectory points, the first parameter corresponding to each region is adjusted, wherein the trajectory points are the location points of the movement trajectory mapped onto the first map; Based on the adjusted first parameter corresponding to each region, determine the image information corresponding to each region; Based on the image information corresponding to each region, a second map is generated that updates the first map.
6. The method as described in claim 1, characterized in that, The step of updating the first map based on the movement trajectory includes: Based on the movement trajectory, a trajectory map corresponding to the target environment is generated, and the trajectory map is used to describe the number of times the trajectory covers each area in the target environment; Based on the trajectory map, predict the obstacle distribution map of the target environment; The first map is updated based on the obstacle distribution map.
7. The method as described in claim 6, characterized in that, The step of predicting the obstacle distribution map of the target environment based on the trajectory map includes: The trajectory map is input into the trained first network model to obtain the obstacle distribution map of the target environment output by the first network model. The first network model is trained based on a training sample set, which includes multiple training samples, each of which includes a trajectory sample map and an obstacle distribution sample map.
8. The method as described in claim 6 or 7, characterized in that, The step of updating the first map based on the obstacle distribution map includes: If there is a difference between the obstacle distribution map and the first map, then the first map is updated.
9. A map updating device, characterized in that, Includes a module for performing the method according to any one of claims 1 to 8.
10. A map updating device, characterized in that, It includes a processor and a memory; the processor is configured to execute instructions stored in the memory to cause the map updating device to perform the method as described in any one of claims 1 to 8.
11. A computer program product, characterized in that, The computer program product includes a computer program or instructions that, when executed, implement the method described in any one of claims 1 to 8.
12. A computer-readable storage medium, characterized in that, The storage medium stores a computer program or instructions, which, when executed, implement the method described in any one of claims 1 to 8.