Training data labeling method and device

Abstract

The invention provides a training data marking method and device, and the method comprises the steps: obtaining the channel state information data of a wireless signal of a target region, and enabling the target region to be covered by a wireless communication network; obtaining visual perception data of the target area, wherein the visual perception data is obtained based on an output result obtained after reasoning prediction is conducted on video frame data collected from the target area based on a computer vision model; and marking the channel state information data by taking the visual perception data as a label to obtain marked training data which is used for training a perception model based on the channel state information data. According to the marking method of the training data, the computer vision model is adopted to perform visual perception on the target area covered by the wireless communication network, and the channel state information data of the target area is automatically marked by utilizing the visual perception result, so that the manpower cost of marking the channel state information data is greatly reduced; and the marking efficiency of the channel state information data is improved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence (AI) technology, and in particular to a method and apparatus for labeling training data. Background Technology

[0002] With the widespread deployment of wireless fidelity (Wi-Fi) hotspots, Wi-Fi sensing technology based on channel state information (CSI) has broad prospects in fields such as presence detection and behavior recognition. However, obtaining training datasets for Wi-Fi sensing presents challenges, and how to perform large-scale, efficient, and accurate annotation of Wi-Fi sensing data is an urgent problem to be solved. Summary of the Invention

[0003] The embodiments of this application provide a training data annotation method and apparatus, which realizes automated annotation of CSI data, greatly reduces the manpower cost of CSI data annotation, improves the efficiency of CSI data annotation, and enables efficient acquisition of large amounts of CSI annotation data (such as WIFI sensing annotation data).

[0004] In a first aspect, this application provides a method for labeling training data, including acquiring CSI data of wireless signals in a target area, the target area being covered by a wireless communication network; acquiring visual perception data of the target area, the visual perception data being obtained based on the output result of a computer vision (CV) model after inference and prediction based on video frame data collected from the target area; using the visual perception data as labels to label the CSI data to obtain labeled training data, the labeled training data being used to train a perception model based on CSI data.

[0005] The training data annotation method of this application uses a CV model to perform visual perception of the target area covered by the wireless communication network, and uses the visual perception results to automatically annotate the CSI data of the target area, which greatly reduces the manpower cost of CSI data annotation and improves the efficiency of CSI data annotation.

[0006] In one possible implementation, the CSI data has a first timestamp indicating the time when the wireless signal was received; the visual perception data has a second timestamp indicating the time when the corresponding video frame data was acquired; and the visual perception data is used as a label to annotate the CSI data to obtain labeled training data. A specific implementation involves aligning the CSI data and visual perception data based on the first and second timestamps to obtain multiple sets of data, each set including CSI data with the same timestamp and visual perception data; and using the visual perception data within the same set as a label to annotate the CSI data within the same set.

[0007] By precisely matching timestamps, CSI data and visual perception data can be automatically aligned, facilitating the subsequent rapid and accurate labeling of CSI data with visual perception data.

[0008] In another possible implementation, CSI data is obtained based on the wireless signal received by the wireless receiving device, and video frame data is obtained based on the image acquisition of the target area by the camera device; the annotation method for training data provided in this application also includes performing clock synchronization operation on the wireless receiving device and the camera device to make the time of the wireless receiving device and the camera device consistent.

[0009] In this possible implementation, clock synchronization is first performed on the wireless receiving device and the camera device to align their system times. This ensures that the timestamps of the CSI data collected by the wireless receiving device and the video frame data collected by the camera device are consistent at the same time, which is beneficial for the accurate alignment of subsequent timestamp-based CSI data and visual perception data.

[0010] In another possible implementation, a specific method for obtaining visual perception data of the target area is as follows: receiving visual perception data sent by a camera device, obtaining visual perception data based on the output of a CV model deployed on the camera device after processing video frame data.

[0011] For camera devices (such as webcams) with video memory or sufficient video memory, a CV model can be deployed on the camera device. After the camera device captures video frames in the target area, the captured video frame data is used as the input of the CV model. The CV model outputs visual perception data, and the camera device sends the visual perception data to the labeler. The labeler obtains the video perception data. Optionally, the labeler can be deployed on a wireless receiving device or a computing device.

[0012] In another possible implementation, a specific implementation of obtaining visual perception data of the target area is as follows: receiving visual perception data sent by the execution device, obtaining visual perception data, which is obtained based on the output result of the computer vision model deployed on the execution device after processing the video frame data received from the camera device, wherein the execution device is a computing device that is communicatively connected to the camera device.

[0013] In other words, for camera devices lacking sufficient video memory, the CV model can be deployed on an additional execution device with video memory. After the camera device captures video frames of the target area, it sends the captured video frame data to the execution device. The execution device uses the captured video frame data as input to the CV model deployed on it. The CV model outputs visual perception data, which the execution device then sends to an annotator. The annotator obtains the video perception data. The execution device can be any device with a certain amount of video memory, such as a server or a fiber-to-the-room (FTTR) system.

[0014] In another possible implementation, the visual perception data includes one or more of the following: presence detection results, crowd counting results, action recognition results, and fall detection results in the target area. The presence detection results indicate whether there is a living person (e.g., a person) in the target area; the crowd counting results indicate how many people are in the target area; the action recognition results indicate the actions of the target person in the target area (e.g., waving, squatting, jumping, nodding, etc.); and the fall detection results indicate whether a person in the target area has fallen.

[0015] This application uses multiple CV models to output various visual perception data (such as presence detection results, crowd counting results, action recognition results, and stacking detection results), and uses these multiple visual perception data to annotate CSI data. This enables the labeling of CSI data once to be suitable for multiple perception tasks, which not only improves the comprehensiveness and reusability of data annotation, but also provides rich label information for different intelligent analysis tasks.

[0016] Optionally, the wireless signal in this application can be a WIFI signal, and the labeled training data can be used to train the WIFI sensing model. The trained WIFI sensing model can output sensing results based on the CSI data of the WIFI signal. The sensing results can include one or more of the following: presence detection results, crowd counting results, action recognition results, and stacking detection results.

[0017] Secondly, this application also provides a training data annotation device, which includes a first acquisition module, a second acquisition module, and an annotation module. The first acquisition module is used to acquire channel state information data of wireless signals in a target area, the target area being covered by a wireless communication network. The second acquisition module is used to acquire visual perception data of the target area, the visual perception data being obtained based on the output result of a computer vision model after inference and prediction based on video frame data collected from the target area. The annotation module is used to use the visual perception data as labels to annotate the channel state information data, thereby obtaining annotated training data. The annotated training data is used to train a perception model based on the channel state information data.

[0018] In one possible implementation, the CSI data has a first timestamp indicating the time when the CSI data was acquired; the visual perception data has a second timestamp indicating the time when the corresponding video frame data was acquired; the annotation module is specifically used to: align the CSI data and visual perception data based on the first and second timestamps to obtain multiple sets of data, each set of data including CSI data and visual perception data with the same timestamp; and use the visual perception data in the same set of data as labels to annotate the CSI data in the same set of data.

[0019] In another possible implementation, CSI data is obtained based on the wireless signal received by the wireless receiving device, and video frame data is obtained based on the image acquisition of the target area by the camera device; the annotation device for training data provided in this application also includes a synchronization module, which is used to perform clock synchronization operation on the wireless receiving device and the camera device to make the time of the wireless receiving device and the camera device consistent.

[0020] In another possible implementation, the second acquisition module is specifically used to: receive visual perception data sent by the camera device, obtain visual perception data, and obtain the visual perception data based on the output result of the CV model deployed on the camera device after processing the video frame data.

[0021] In another possible implementation, the second acquisition module is specifically used to: receive visual perception data sent by the execution device, obtain the visual perception data, which is obtained based on the output result of the computer vision model deployed on the execution device after processing the video frame data received from the camera device, wherein the execution device is a computing device that is communicatively connected to the camera device.

[0022] In another possible implementation, the visual perception data includes one or more of the following: presence detection results in the target area, crowd counting results, action recognition results, and stacking detection results.

[0023] Optionally, the wireless signal in this application can be a WIFI signal, and the labeled training data can be used to train the WIFI sensing model. The trained WIFI sensing model can output sensing results based on the CSI data of the WIFI signal. The sensing results can include one or more of the following: presence detection results, crowd counting results, action recognition results, and stacking detection results.

[0024] The training data annotation device provided in this application, through modular software deployment, can process and fuse data of different modalities (i.e., CSI modal data and video frame modal data), exhibiting extremely high adaptability and flexibility. Regardless of different environmental conditions, scene layouts, or diverse application requirements, the training data annotation device of this application can provide stable and reliable data annotation services through automated CV models and precise timestamp synchronization mechanisms. This flexibility allows the training data annotation device to be easily integrated into various intelligent systems, meeting the needs of intelligent perception and data analysis in different scenarios.

[0025] Thirdly, embodiments of this application provide a computing device, including a memory and a processor, wherein the memory stores instructions that, when executed by the processor, cause the method described in the first aspect or any possible implementation of the first aspect to be implemented.

[0026] Fourthly, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, causes the method described in the first aspect or any possible implementation thereof to be implemented.

[0027] Fifthly, embodiments of this application also provide a computer program or computer program product, which includes instructions that, when executed, cause a computer to perform the method described in the first aspect or any possible implementation thereof.

[0028] In a sixth aspect, embodiments of this application also provide a chip including at least one processor and a communication interface, the processor being configured to perform the method described in the first aspect or any possible implementation thereof. Attached Figure Description

[0029] Figure 1 A schematic diagram of a wireless sensing system is shown.

[0030] Figure 2 This illustration shows an application scenario diagram of the training data annotation method provided in an embodiment of this application;

[0031] Figure 3A system architecture diagram of an annotation system applicable to the annotation method for training data provided in the embodiments of this application is shown.

[0032] Figure 4 A system architecture diagram of an annotation system applicable to the annotation method for training data provided in the embodiments of this application is shown.

[0033] Figure 5 A flowchart illustrating a training data annotation method provided in an embodiment of this application;

[0034] Figure 6 This is a schematic diagram of the structure of a training data annotation device provided in an embodiment of this application;

[0035] Figure 7 A schematic diagram of the structure of a computing device provided in an embodiment of this application. Detailed Implementation

[0036] The term "and / or" used in this article describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. The symbol " / " in this article indicates that the related objects are in an "or" relationship; for example, A / B means A or B.

[0037] The terms "first" and "second," etc., used in the specification and claims herein are used to distinguish different objects, not to describe a specific order of objects. It should be understood that such terms are interchangeable where appropriate; this is merely a way of distinguishing objects with the same properties in the description of embodiments of this application. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, so that a process, method, system, product, or apparatus that comprises a list of elements is not necessarily limited to those elements, but may include other elements not expressly listed or inherent to such processes, methods, systems, products, or apparatus.

[0038] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0039] In the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more, for example, multiple processing units means two or more processing units, multiple elements means two or more elements, etc.

[0040] Figure 1 A schematic diagram of a wireless sensing system is shown. Figure 1 The wireless sensing system shown includes a wireless transmitter and a wireless receiver. The wireless transmitter emits wireless signals, which propagate to the receiver via three main paths: The first path is the line-of-sight (LOS) path, where the signal reaches the receiver directly without reflection from obstacles (such as walls, floors, ceilings, or other static objects); the second path is the static object reflection path, where the signal reaches the receiver after reflection from static obstacles in the environment; and the third path is the dynamic object reflection path, where the signal reaches the receiver after reflection from moving objects such as humans or animals. Of these three propagation paths, the first two are relatively stable, while the third path, due to variations in human position and movement, results in different propagation paths for the wireless signal, leading to different signal characteristics (e.g., different CSI) of the wireless signal received by the receiver after reflection from the human body. Therefore, a perception model based on CSI data can be obtained by modeling the mapping relationship between CSI data of wireless signals in an environment covered by wireless communication networks and perceived information in the environment (such as human motion recognition, stacking detection, presence detection, and crowd calculation).

[0041] To accurately model a perception model based on CSI data, a large amount of labeled CSI data is required. In existing technologies, most CSI data labeling is done manually, which is time-consuming and labor-intensive, resulting in high CSI labeling costs and poor accuracy.

[0042] To address the aforementioned issues, this application provides a method and apparatus for labeling training data. It uses visual perception technology to perceive visual perception data of a target area covered by a wireless communication network, and then uses the visual perception data to label CSI data generated within the target area, obtaining labeled CSI data. This achieves automated labeling of CSI data, effectively improving the efficiency and reducing the cost of CSI data labeling. The labeled CSI data can be used as training data for a perception model based on CSI data, facilitating the training of such a model.

[0043] The following detailed description, with reference to the accompanying drawings, details the specific implementation of a training data annotation method provided in this application.

[0044] Figure 2 This illustration shows an application scenario diagram of the training data annotation method provided in an embodiment of this application. For example...Figure 2 As shown, in this embodiment of the application, the wireless receiving device extracts CSI data from the received wireless signal using a CSI data extraction tool (e.g., CSI Tool), acquiring CSI data in real time or periodically. A camera device with visual perception capabilities is used to collect video frame data within a target area (covered by a wireless communication network, such as a room covered by a WIFI network). The video frame data is processed by a CV model deployed on the camera device to output visual perception data. Then, the visual perception data is used to label the CSI data, resulting in a labeled CSI dataset. This dataset can be used as a training dataset for a perception model based on CSI data, training a perception model with satisfactory prediction accuracy.

[0045] A wireless transmitting device is a device capable of transmitting wireless signals. It establishes a wireless communication connection with a wireless receiving device to send wireless signals. The wireless receiving device can receive the wireless signals transmitted by the wireless transmitting device and extract CSI data from the received wireless signals. In one example, the wireless receiving device may include a network interface card (NIC) to extract the CSI data from the wireless signals.

[0046] Wireless transmitting or receiving devices can be smart home devices, such as lights, refrigerators, washing machines, air conditioners, smart TVs, smart speakers, wireless routers, and wireless access points (APs) with wireless communication capabilities, or terminals such as smartphones, tablets, and desktop computers. For example, the wireless transmitting device could be a Wi-Fi router, and the wireless receiving device could be a smartphone, with both communicating via Wi-Fi.

[0047] The trained perception model based on CSI data can be applied in a variety of scenarios, such as:

[0048] Fall detection application scenarios:

[0049] Typically, when there are elderly people or children in the home, for their safety, a trained perception model based on CSI data can be deployed in wireless receiving devices (such as smart home appliances or smart speakers) in rooms where the elderly or children frequently move around. This perception model can use CSI data to detect whether the elderly or children in the room have fallen.

[0050] The training dataset for the perception model based on CSI data in this scenario includes multiple training samples, each of which includes CSI data and a label indicating whether a fall has occurred.

[0051] The annotation method for training data of the CSI-based perception model applied to fall detection scenarios is as follows:

[0052] By deploying wireless receiving devices (such as smart speakers) and visual perception cameras (with a fall detection CV model mounted on them) in a room, the wireless receiving devices extract CSI data from the received wireless signals using a CSI data extraction tool, acquiring CSI data in real time or periodically. The cameras collect video frame data from within the room and use this video frame data as input to the CV model, which outputs fall detection results. Then, the visual perception data is used to label the CSI data, resulting in a CSI dataset with fall-related labels. This dataset is then used to supervise the training of the CSI-based perception model, resulting in a CSI-based perception model with fall detection capabilities.

[0053] Application scenarios of human motion control:

[0054] To meet users' personalized needs, human body movements can be used to control terminal devices, such as smartphones, tablets, and smart wearable devices, as well as smart home devices, such as televisions, water heaters, refrigerators, lights, and curtains.

[0055] Users can set human body movements and the corresponding operations. By deploying a trained perception model based on CSI data in a wireless receiving device, the system can identify and perceive human body movements through CSI data, and then perform corresponding operations based on those movements.

[0056] For example, a trained perception model based on CSI data can be deployed on a smart TV. The smart TV extracts CSI data from the received wireless signals and inputs the CSI data into the perception model. The perception model then identifies human actions, such as waving. If the action is identified as waving, the smart TV executes the corresponding operation. For example, if waving corresponds to changing channels, the smart TV will perform the channel change operation, allowing users to control the smart TV through specific actions, greatly improving the user experience.

[0057] Of course, in other scenarios, the trained perception model based on CSI data can also be deployed in other wireless receiving devices, such as smart speakers. When the smart speaker recognizes human movements through the CSI data-based perception model deployed on it, it sends the recognition result to the smart TV. The smart TV then controls the smart TV according to the operation corresponding to the recognized movement.

[0058] In this scenario, each training sample in the training dataset of the perception model based on CSI data includes CSI data and human motion labels.

[0059] The annotation method for training data of CSI-based perception models applied to human motion scenarios is as follows:

[0060] By deploying wireless receiving devices (such as smart TVs) and visually perceptive cameras (with motion recognition (CV) models mounted on them) within a room, the wireless receiving devices extract CSI data from the received wireless signals using a CSI data extraction tool, acquiring CSI data in real-time or periodically. The cameras capture video frame data from within the room and use this data as input to the CV model, which outputs human motion recognition results. Then, the visually perceptive data is used to label the CSI data, resulting in a CSI dataset with motion labels. This dataset is then used for supervised training of the CSI-based perceptual model, resulting in a CSI-based perceptual model with motion recognition capabilities.

[0061] Indoor security application scenarios:

[0062] Typically, for indoor security reasons, users need to monitor their homes for intrusions when no one is home to prevent theft. In this case, a trained sensing model based on CSI data can be deployed on the wireless receiving device in the home. This sensing model can detect whether someone is present in the room through the CSI data of the wireless signal it receives. If someone is detected, it can be determined that someone has broken into the home.

[0063] In this scenario, the training dataset for the perception model based on CSI data includes CSI data and a label indicating whether or not a person is present in each training sample.

[0064] The annotation method for training data of the perception model based on CSI data for indoor security applications is as follows:

[0065] By deploying wireless receiving devices (such as Wi-Fi routers) and cameras with visual perception capabilities (with a CV model for object detection) within a room, the wireless receiving devices extract CSI data from the received wireless signals using a CSI data extraction tool, acquiring CSI data in real-time or periodically. The cameras capture video frame data from within the room and use this data as input to the CV model, which outputs the object detection result: whether someone is in the room. Then, the visual perception data is used to label the CSI data, resulting in a CSI dataset with presence / absence labels. This dataset is then used for supervised training of the CSI-based perception model, resulting in a CSI-based perception model with presence / absence detection capabilities.

[0066] Crowd counting application scenarios:

[0067] In shopping malls and other commercial venues, it is necessary to understand the flow of people to enable businesses to make informed decisions. A trained perception model based on CSI data can be deployed on wireless receiving devices (such as wireless Wi-Fi routers with transceiver capabilities) within the mall. This model can perceive the flow of people (i.e., the number of people in the mall) through the CSI data of the received wireless signals.

[0068] In this scenario, the training dataset for the perception model based on CSI data includes CSI data and quantity labels for each training sample.

[0069] The annotation method for training data of the perception model based on CSI data for indoor security applications is as follows:

[0070] By deploying wireless receiving devices (such as Wi-Fi routers) within the shopping mall and visually perceptive cameras (with crowd counting models mounted on them) on each floor, the wireless receiving devices extract CSI data from the received wireless signals using a CSI data extraction tool, acquiring CSI data in real-time or periodically. The cameras collect video frame data from within the mall and use this data as input to the CV model, which outputs a crowd count, indicating the number of people in the mall. Then, the visually perceptive data is used to label the CSI data, resulting in a CSI dataset with quantity labels. This dataset is then used for supervised training of the CSI-based perception model, resulting in a CSI-based perception model with crowd counting capabilities.

[0071] It is understood that the above examples of application scenarios are merely possible implementations of the embodiments of this application, and are not exhaustive, nor do they constitute a limitation on the embodiments of this application.

[0072] As described above, the training dataset obtained by the annotation in this application is obtained from a specific application scenario. The perception model trained using this training dataset does not need to obtain another batch of labeled data for incremental training in a specific environment, thereby improving training efficiency and providing the prediction accuracy of the trained perception model.

[0073] Figure 3 A system architecture diagram of an annotation system applicable to the annotation method for training data provided in the embodiments of this application is shown. Figure 3 As shown, the annotation system includes a wireless receiving device and a camera device, which are wirelessly connected, for example, via Wi-Fi. The wireless receiving device contains a CSI data extraction module, a signal processing module, a clock synchronization module, and a data interaction module, while the camera device contains an image acquisition module, a CV model, a clock synchronization module, and a data interaction module.

[0074] The wireless receiver module's CSI data extraction module extracts CSI data from the received wireless signal. The signal processing module preprocesses the extracted CSI data, performing operations such as removing DC components, performing a Fast Fourier Transform (FFT) to obtain frequency domain information, and using a Kalman filter to smooth the signal and reduce noise. The clock synchronization module synchronizes the system time of the wireless receiver and the camera, ensuring their system times are consistent at the same moment. The data exchange module receives tag data sent by the camera. The wireless receiver uses the tag data to annotate the CSI data.

[0075] The image acquisition module in the camera equipment is used to acquire video frame data within the target area. The CV model is used for inference and prediction based on the video frame data to obtain visual perception data. The clock synchronization module is used to perform clock synchronization operations to synchronize the system time of the wireless receiving device and the camera equipment, ensuring that their system times are consistent at the same moment. The data exchange module is used to send the visual perception data as tags to the wireless receiving device, so that the wireless receiving device can use the tag data to label the CSI data.

[0076] It should be noted that deploying CV models within camera equipment and using CV models for inference and prediction requires a certain amount of video memory and computing power. Figure 3 The architecture of the annotation system shown is suitable for camera devices with sufficient video memory and computing power, such as those with a graphics processing unit (GPU) chip. When the camera device lacks video memory or has insufficient video memory, the CV model needs to be deployed on a device with video memory, such as a FTTR, server, or other computing device.

[0077] Figure 4 A system architecture diagram of an annotation system applicable to the annotation method for training data provided in the embodiments of this application is shown. Figure 4 As shown, the annotation system includes a wireless receiving device, a camera device, and an execution device. The wireless receiving device and the camera device are connected via communication, as are the camera device and the execution device. The wireless receiving device is equipped with a CSI data sampling module, a CSI data and tag data alignment module, and a tag receiving module. The camera device is equipped with an image acquisition module, a transmission module, and a tag transceiver module. The execution device is equipped with a CV model.

[0078] The wireless receiving module's CSI data sampling module extracts CSI data from the received wireless signal. The tag receiving module receives tag data sent by the camera device. The CSI data and tag data alignment module aligns the CSI data and tag data to ensure accurate labeling of the CSI data using the tag book.

[0079] The image acquisition module in the camera device is used to collect video frame data tags within the target area. The visual perception module acts as an agent, sending the acquired video frame data to the execution device, which then performs the actual inference calculations. The tag transceiver module is used to receive tag data sent by the execution device and to send tag data to the wireless receiving device.

[0080] The transceiver module in the execution device receives video frame data sent by the camera device. The CV model is used to perform inference and prediction based on the video frame data to obtain visual perception data. The transceiver module is used to send the visual perception data output by the CV model as tags to the camera device.

[0081] Figure 5 This is a flowchart illustrating a training data annotation method provided in an embodiment of this application. This method can be executed by any device, equipment, platform, or cluster of devices with computing capabilities. This application does not specifically limit the specific computing device executing the method; a suitable computing device can be selected as needed. For example, it can be implemented on a wireless receiving device, meaning the training data annotation method provided in this application is implemented on a terminal device; it can also be implemented on a computing device (such as a terminal device or server) that is communicatively connected to the wireless receiving device. Figure 5 As shown, the training data annotation method provided in this application embodiment includes at least steps S501 to S503.

[0082] In step S501, CSI data of the wireless signal in the target area is acquired.

[0083] In a scenario where the annotation device that can implement the annotation method for training data provided in the embodiments of this application is deployed in a wireless receiving device, the acquisition of CSI data of the wireless signal is obtained by the wireless receiving device using a CSI data extraction tool (e.g., CSI Tool) to extract the CSI data from the received wireless signal.

[0084] For example, the labeling device can be deployed within a wireless receiving device in a target area covered by a wireless communication network. The wireless communication network includes a wireless transmitting device and a wireless receiving device. The wireless transmitting device is a device capable of transmitting wireless signals and establishes a wireless communication connection with the wireless receiving device to send wireless signals to it. The wireless receiving device can receive the wireless signals transmitted by the wireless transmitting device and extract CSI data from the received wireless signals.

[0085] Wireless transmitting or receiving devices can be smart home devices, such as lights, refrigerators, washing machines, air conditioners, smart TVs, smart speakers, wireless routers, wireless access points (APs), etc., which have wireless communication functions, or terminals such as smartphones, tablets, and desktop computers.

[0086] Wireless transmitting and receiving devices can communicate via Bluetooth, Wi-Fi, infrared data association (IrDA), ZigBee wireless communication, and ultra-wide bandgap (UWB) wireless communication. For example, a Wi-Fi router can be the wireless transmitting device, and a smartphone can be the wireless receiving device; the two can connect via Wi-Fi.

[0087] Smartphones obtain CSI data by extracting CSI data from received wireless signals in real time or periodically using CSI data extraction tools (such as CSI Tool).

[0088] For example, a wireless receiving device can also be a WIFI router with transceiver capabilities. This WIFI router has a wireless network card (such as the Intel 5300 series). The wireless network card uses the CSI Tool to extract CSI data from the received wireless signal in real time, accurately obtaining the CSI data in the target area.

[0089] The target area is the scenario where the CSI data-based perception model will be deployed. For example, if the application scenario of the CSI data-based perception model is to iteratively detect elderly people living alone in a room, then the target area is the room where the elderly person lives, and the wireless receiving device can be a device with wireless communication capabilities in the elderly person's room, such as a smart speaker.

[0090] To save on storage and computing power costs for wireless receiving devices, the annotation apparatus that implements the training data annotation method provided in this application can also be deployed on a computing device that is communicatively connected to the wireless receiving device. For example, the annotation apparatus can be deployed on a server (which can be a physical server or a cloud server). The server is communicatively connected to the wireless receiving device. The wireless receiving device extracts CSI data from the received wireless signal using a CSI data extraction tool and sends the CSI data to the server. The server receives the CSI data sent by the wireless receiving device and obtains the CSI data.

[0091] The following is a specific implementation of the annotation method for receiving training data provided in this application embodiment, with the annotation device deployed on a wireless receiving device as an example. The implementation of the annotation device deployed on other devices is similar and can be implemented by referring to it. This application embodiment will not be described again.

[0092] In step S502, visual perception data of the target area is acquired.

[0093] This application implements a method of setting up a camera device, such as a webcam, in a target area. This webcam has communication capabilities and is connected to a wireless receiving device. The webcam is used to collect video frame data from the target area. Then, a CV (Virtual Character Recognition) model is used to process the video frame data to obtain visual perception data of the target area. This data may include one or more of the following: presence detection results, crowd counting results, action recognition results, and fall detection results. The presence detection results indicate whether a living being (e.g., a person) exists in the target area; the crowd counting results indicate the number of people in the target area; the action recognition results indicate the actions of the target person in the target area (e.g., waving, squatting, jumping, nodding, etc.); and the fall detection results indicate whether a person has fallen in the target area.

[0094] For example, a high-resolution webcam (such as the Logitech Brio) can be used, and the webcam's position and viewing angle can be adjusted so that the webcam can capture images of the target area.

[0095] Taking a fall detection application scenario as an example, the target area is a room frequently used by the elderly or children. A suitable location is found in the room to install a webcam, and the camera's angle is adjusted so that it can capture all or as much of the room's content as possible. The camera captures clear video frame data of the room in real time. Then, a computer vision (CV) model is used to process the acquired video frame data to obtain the fall detection result. For example, the video frame data captured by the camera can be used as input to the CV model, and the output can indicate whether an elderly person or child in the room has fallen.

[0096] In one example, the camera has sufficient video memory and computing power to directly deploy the CV model into its video memory. After acquiring video frame data of the target area, the camera inputs the video frame data into the deployed CV model, which performs inference and prediction, outputting a fall detection result. The camera then sends the fall detection result as tag data to a wireless receiving device, which obtains the visual perception data of the target area.

[0097] In another example, if the camera lacks dedicated video memory or has insufficient video memory, the CV model can be deployed on a device with video memory (i.e., the execution device), such as an FTTR, server, or workstation. For instance, the CV model can be deployed on a server equipped with an NVIDIA GPU chip, enabling rapid inference computation of the CV model. The server communicates with the camera. After the camera acquires video frame data of the target area, it sends the video frame data to the server with the deployed CV model. Upon receiving the video frame data, the server processes the video frame data using the CV model to obtain a fall detection result, which is then sent to the camera. After receiving the fall detection result from the server, the camera sends the fall detection result to a wireless receiving device, which then acquires the visual perception data of the target area.

[0098] In another example, where the camera lacks dedicated video memory or has insufficient video memory, the CV model is deployed on a device with video memory, such as an FTTR (Flexible Transmission Tester), server, or workstation. For instance, the CV model could be deployed on a server equipped with an NVIDIA GPU chip, enabling rapid inference computation. The server communicates with both the wireless receiving device and the camera. After acquiring video frame data of the target area, the camera sends the data to the server with the deployed CV model. Upon receiving the video frame data, the server processes it using the CV model to obtain a fall detection result, which is then sent to the wireless receiving device. The wireless receiving device then acquires the visual perception data of the target area.

[0099] It should be noted that the camera can communicate with the execution device in various ways. For example, the camera and the execution device can communicate through wireless or wired communication. Alternatively, the camera and the execution device can communicate through direct or indirect communication (for example, the camera can communicate indirectly with the execution device through an intermediate network device). This application does not limit the specific connection method between the camera and the execution device, and a suitable communication connection method can be selected according to actual needs.

[0100] In some other examples, to ensure data reusability, visual perception data can include multiple perception results, such as presence detection results, crowd counting results, action recognition results, and stacking detection results. In this case, multiple types of CV models or CV models capable of outputting multiple perception results can be deployed in cameras or servers.

[0101] For example, a CV model with presence detection (i.e., a CV model that can output presence detection results based on input video frame data), a CV model with crowd counting, a CV model with action recognition, and a CV model with stacking detection are deployed in a camera. After the camera acquires video frame data of the target area, the video frame data is used as the basis for the various types of CV models deployed on it (including CV models with presence detection, crowd counting, action recognition, and stacking detection), resulting in multiple perception results: presence detection result, crowd counting result, action recognition result, and stacking detection result. Using these multiple perception results as labels for CSI data allows for labeling CSI data once, making it suitable for multiple perception tasks. This not only improves the comprehensiveness and reusability of data labeling but also provides rich label information for different intelligent analysis tasks.

[0102] For example, when training a perception model based on CSI data with presence detection capabilities, a "presence / absence" label is selected from multiple labels in the CSI data. The CSI dataset with the "presence / absence" label is then used as the training sample set to train the perception model, resulting in a perception model with presence detection capabilities. Similarly, when training a perception model based on CSI data with fall detection capabilities, a "fall" or "cash" label is selected from multiple labels in the CSI data. The CSI dataset with the "fall" or "cash" label is then used as the training sample set to train the perception model, resulting in a perception model with fall detection capabilities. Likewise, when training a perception model based on CSI data with action recognition capabilities, multiple labels in the CSI data are selected from multiple labels in the CSI data. Among the various labels, human action labels are selected, and the CSI dataset with human action labels is used as the training sample set to train the CSI data-based perception model, resulting in a CSI data-based perception model with action recognition function. When it is necessary to train a CSI data-based perception model with crowd counting function, a quantity label is selected from the various labels of CSI data, and the CSI dataset with quantity labels is used as the training sample set to train the CSI data-based perception model, resulting in a CSI data-based perception model with crowd counting function. In this way, the CSI dataset labeled in this application embodiment can be applied to the training of various types of CSI data-based perception models, improving the reusability of the CSI dataset labeled in this application embodiment and further reducing the labeling cost of CSI data.

[0103] In step S503, the visual perception data is used as a label to annotate the CSI data, thereby obtaining the annotated training data.

[0104] After obtaining the CSI data and visual perception data of the target area through the above steps, the visual perception data is used as a label to annotate the CSI data.

[0105] In one example, before labeling the CSI data, the CSI data and visual perception data are aligned. Then, based on the aligned data, the visual perception data is used to label the aligned CSI data.

[0106] The alignment in this embodiment is a temporal alignment, which means aligning CSI data and visual perception data at the same time to form a set of data. For example, a set of data includes CSI data at time T1 and visual perception data at time T1. Then, the visual perception data at time T1 is used as a label to annotate the CSI data at time T1, resulting in labeled CSI data. The labeled CSI data can then be used as a training sample to train the perception model based on the CSI data.

[0107] It is understandable that the visual perception data at time T1 is the visual perception data output by the CV model after processing the video frame data at time T1.

[0108] To align CSI data and visual perception data in the time dimension, it is necessary to utilize the time of the wireless signal corresponding to the CSI data and the time when the video frame was acquired. Therefore, the wireless receiving device in this embodiment performs a timestamp operation on the acquired CSI data during the acquisition process to indicate the time when the wireless signal corresponding to the CSI data was generated. For example, the wireless receiving device records the time of the received wireless signal, then uses a CSI extraction tool to extract CSI data from the wireless signal, generates a timestamp using the recorded time of the wireless signal reception, and associates the timestamp with the CSI data.

[0109] The camera's visual perception data is timestamped to indicate when the corresponding video frame was captured. For example, the camera captures an image of a target area to obtain video frame data, records the time of the captured video frame, generates a timestamp using this time, and then uses a CV model to process the video frame data to obtain visual perception data. The timestamp is then associated with the visual perception data.

[0110] Based on the timestamps of CSI data and visual perception data, we perform time-alignment operations on the CSI data and visual perception data to obtain time-aligned CSI data and visual perception data.

[0111] The timestamps of CSI data are generated using the system time of the wireless receiving device, while the timestamps of visual perception data are generated using the system time of the camera. Therefore, the consistency of the system time on the wireless receiving device and the system time of the camera is crucial to the accuracy of the time-dimensional alignment of CSI data and visual perception data. Thus, in this embodiment, before acquiring the CSI data of the wireless signal in the target area and the visual perception data of the target area—that is, before the annotation method for the training data provided in this embodiment is executed, or in the first step of the method—a clock synchronization operation is first performed on the wireless receiving device and the camera device to align their system times. This ensures that the timestamps of the CSI data acquired by the wireless receiving device and the video frame data acquired by the camera device are consistent at the same time, achieving accurate alignment of CSI data and visual perception data based on timestamps.

[0112] Based on the same concept as the aforementioned embodiment of a training data annotation method, this application also provides a training data annotation device 600. This training data annotation device can automate the annotation of CSI data, effectively reducing the annotation cost of CSI data and improving the annotation efficiency and accuracy of CSI data. The training data annotation device 600 includes components for implementing... Figure 5 The units or modules in the labeling method for training data shown.

[0113] Figure 6 This is a schematic diagram of a training data annotation device provided in an embodiment of this application. Figure 6 As shown, the annotation device 600 for the training data includes a first acquisition module 601, a second acquisition module 602, and an annotation module 603. The first acquisition module 601 is used to acquire channel state information data of wireless signals in the target area, which is covered by a wireless communication network. The second acquisition module 602 is used to acquire visual perception data of the target area, which is obtained based on the output of a computer vision model after inference and prediction based on video frame data collected from the target area. The annotation module 603 is used to use the visual perception data as labels to annotate the channel state information data, thereby obtaining annotated training data. The annotated training data is used to train a perception model based on the channel state information data.

[0114] In one possible implementation, the CSI data has a first timestamp indicating the time when the CSI data was acquired; the visual perception data has a second timestamp indicating the time when the video frame data corresponding to the visual perception data was acquired; the annotation module 603 is specifically used to: align the CSI data and the visual perception data based on the first and second timestamps to obtain multiple sets of data, each set of data including CSI data and visual perception data with the same timestamp; and use the visual perception data in the same set of data as labels to annotate the CSI data in the same set of data.

[0115] In another possible implementation, CSI data is obtained based on the wireless signal received by the wireless receiving device, and video frame data is obtained based on the image acquisition of the target area by the camera device; the training data annotation device 600 provided in this application also includes a synchronization module 604, which is used to perform clock synchronization operation on the wireless receiving device and the camera device to make the time of the wireless receiving device and the camera device consistent.

[0116] In another possible implementation, the second acquisition module 602 is specifically used to: receive visual perception data sent by the camera device, obtain visual perception data, and obtain the visual perception data based on the output result of the CV model deployed on the camera device after processing the video frame data.

[0117] In another possible implementation, the second acquisition module 602 is specifically used to: receive visual perception data sent by the execution device, and obtain the visual perception data, which is obtained based on the output result of the computer vision model deployed on the execution device after processing the video frame data received from the camera device, wherein the execution device is a computing device that is communicatively connected to the camera device.

[0118] In another possible implementation, the visual perception data includes one or more of the following: presence detection results in the target area, crowd counting results, action recognition results, and stacking detection results.

[0119] Optionally, the wireless signal in this application can be a WIFI signal, and the labeled training data can be used to train the WIFI sensing model. The trained WIFI sensing model can output sensing results based on the CSI data of the WIFI signal. The sensing results can include one or more of the following: presence detection results, crowd counting results, action recognition results, and stacking detection results.

[0120] The training data annotation device 600 provided in this application, through modular software deployment, can process and fuse data of different modalities (i.e., CSI modal data and video frame modal data), exhibiting extremely high adaptability and flexibility. Regardless of different environmental conditions, scene layouts, or diverse application requirements, the training data annotation device 600 can provide stable and reliable data annotation services through automated CV models and precise timestamp synchronization mechanisms. This flexibility allows the training data annotation device to be easily integrated into various intelligent systems, meeting the needs of intelligent perception and data analysis in different scenarios.

[0121] The training data annotation apparatus 600 according to the embodiments of this application can correspond to the execution of the method described in the embodiments of this application, and the above and other operations and / or functions of each module in the training data annotation apparatus 600 are respectively for implementing Figure 5 For the sake of brevity, the corresponding processes of each method in the code will not be elaborated here.

[0122] This application embodiment also provides a computing device, including at least one processor, a memory, and a communication interface, wherein the processor is used to execute... Figure 5 The method described.

[0123] Figure 7 A schematic diagram of the structure of a computing device provided in an embodiment of this application.

[0124] like Figure 7As shown, the computing device 700 includes at least one processor 701, a memory 702, and a communication interface 703. The processor 701, memory 702, and communication interface 703 are communicatively connected, which can be achieved via a wired (e.g., bus) or wireless connection. The communication interface 703 is used to send and / or receive data from other devices. The memory 702 stores computer instructions, which the processor 701 executes to perform the methods described in the foregoing method embodiments, thereby achieving automated annotation of CSI data, effectively reducing the annotation cost of CSI data and improving the annotation efficiency and accuracy of CSI data.

[0125] It should be understood that, in the embodiments of this application, the processor 701 may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0126] The memory 702 may include read-only memory and random access memory, and provides instructions and data to the processor 701. The memory 702 may also include non-volatile random access memory. Optionally, the random access memory may be, for example, high bandwidth memory (HBM).

[0127] The memory 702 can be volatile memory or non-volatile memory, or it can include both. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM).

[0128] It should be understood that the computing device 700 according to the embodiments of this application can perform the implementation of the embodiments of this application. Figure 5 The method shown is described in detail above, and will not be repeated here for the sake of brevity.

[0129] Embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, wherein when the computer instructions are executed by a processor, the aforementioned method is implemented.

[0130] An embodiment of this application provides a chip including at least one processor and an interface, wherein the at least one processor determines program instructions or data through the interface; the at least one processor is used to execute the program instructions to implement the method mentioned above.

[0131] Embodiments of this application provide a computer program or computer program product that includes instructions that, when executed, cause a computer to perform the methods mentioned above.

[0132] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0133] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented using hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0134] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of this application. It should be understood that the above description is only a specific embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A method for labeling training data, characterized in that, The method comprises: obtaining channel state information data of a wireless signal of a target area, the target area being covered by a wireless communication network; obtaining visual perception data of the target area, the visual perception data being obtained based on an output result of a computer vision model after inference prediction based on video frame data collected from the target area; labeling the channel state information data as the visual perception data to obtain labeled training data, the labeled training data being used for training a perception model based on channel state information data.

2. The method of claim 1, wherein, The channel state information data has a first timestamp, and the first timestamp indicates a time when the wireless signal is received. The visual perception data has a second timestamp, and the second timestamp indicates a time when the video frame data corresponding to the visual perception data is collected. The labeling the channel state information data as the visual perception data to obtain labeled training data comprises: aligning the channel state information data and the visual perception data based on the first timestamp and the second timestamp to obtain a plurality of groups of data, each group of data comprising the channel state information data and the visual perception data with the same timestamp; labeling the channel state information data in the same group of data as the visual perception data in the same group of data.

3. The method according to claim 1 or 2, characterized in that, The channel state information data is obtained based on the wireless signal received by a wireless receiving device, and the video frame data is obtained based on image collection of the target area by a camera device. The method further comprises: performing a clock synchronization operation on the wireless receiving device and the camera device to make the time of the wireless receiving device and the camera device consistent.

4. The method of claim 3, wherein, The obtaining the visual perception data of the target area comprises: receiving the visual perception data sent by the camera device to obtain the visual perception data, the visual perception data being obtained based on an output result of the computer vision model deployed on the camera device after processing the video frame data.

5. The method of claim 3, wherein, The obtaining the visual perception data of the target area comprises: receiving the visual perception data sent by an execution device to obtain the visual perception data, the visual perception data being obtained based on an output result of the computer vision model deployed on the execution device after processing the video frame data received from the camera device, wherein the execution device is a computing device in communication connection with the camera device.

6. The method according to any one of claims 1 to 5, characterized in that, The visual perception data comprises one or more of presence detection results, crowd counting results, action recognition results, and fall detection results in the target area.

7. The method according to any one of claims 1 to 6, characterized in that, The wireless signal is a WIFI signal.

8. A device for labeling training data, characterized in that, The method comprises: a first obtaining module configured to obtain channel state information data of a wireless signal of a target area, the target area being covered by a wireless communication network; a second obtaining module configured to obtain visual perception data of the target area, the visual perception data being obtained based on an output result of a computer vision model after inference prediction based on video frame data collected from the target area; A labeling module is configured to label the channel state information data with the visual perception data as labels to obtain labeled training data, wherein the labeled training data is used to train a perception model based on channel state information data.

9. A computing device comprising a memory and a processor, wherein, The memory stores instructions which, when executed by the processor, cause the method of any one of claims 1-7 to be implemented.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program, when executed by the processor, causes the method of any one of claims 1-7 to be implemented.

Images