Data processing method, device, electronic device and storage medium for sweeping robot

Abstract

An embodiment of the present invention provides a data processing method, device, electronic device and storage medium for a sweeping robot, relating to the field of smart home technology. The method includes: in response to a control instruction for the sweeping robot, obtaining environmental data of the environment in which the sweeping robot is located and equipment data of the sweeping robot; inputting the environmental data and equipment data into a target cleaning model, obtaining cleaning instructions for the environment, and executing cleaning operations corresponding to the cleaning instructions. The target cleaning model is a local model trained by using federated learning between the sweeping robot and the smart home device. Therefore, through localized model training, not only privacy and security are improved, but also the sweeping robot can adapt to different cleaning environments, thereby improving the level of intelligent cleaning and ensuring the cleaning effect of the sweeping robot.

Description

Technical Field

[0001] The present invention relates to the field of smart home technology, and in particular to a data processing method for a sweeping robot, a data processing device for a sweeping robot, an electronic device, and a computer-readable storage medium. Background Art

[0002] With the development of smart home technology, more and more smart home devices are entering people's homes. Among them, sweeping robots, as household appliances that automatically clean floors through intelligent programming or remote control, use sensors, algorithms, and cleaning systems to autonomously plan paths, avoid obstacles, and complete tasks such as vacuuming and mopping. These robots bring great convenience to home cleaning and can effectively reduce the user's household chores. However, as sweeping robots increase their functionality, the amount of data they collect and process also increases, raising concerns about privacy and security among users. Summary of the Invention

[0003] The embodiments of the present invention provide a data processing method, device, electronic device, and computer-readable storage medium for a sweeping robot to solve or partially solve the problem of privacy data leakage during the operation of the sweeping robot.

[0004] An embodiment of the present invention discloses a data processing method for a sweeping robot, which is applied to the sweeping robot. The sweeping robot and several smart home devices are located in the same home local area network. The method includes:

[0005] In response to a control instruction for the cleaning robot, acquiring environmental data of an environment in which the cleaning robot is located and device data of the cleaning robot;

[0006] The environmental data and the device data are input into a target cleaning model to obtain cleaning instructions for the environment, and a cleaning operation corresponding to the cleaning instructions is executed. The target cleaning model is a local model trained by federated learning between the sweeping robot and the smart home device.

[0007] In some feasible implementations, each of the smart home devices is configured with a device model corresponding to its own device function, and the method further includes:

[0008] Obtaining functional data corresponding to each of the smart home devices;

[0009] Acquire an initial cleaning model, input each of the functional data into the initial cleaning model for model training, obtain a corresponding first cleaning model, and generate collaborative training instructions for the first cleaning model;

[0010] Sending the collaborative training instruction to each of the smart home devices, the collaborative training instruction including at least first model parameters of the first cleaning model, the collaborative training instruction being used to instruct the smart home device to extract corresponding second model parameters from a device model trained according to the functional data, and performing federated learning aggregation on the first model parameters and the second model parameters to obtain aggregated third model parameters;

[0011] receiving a third model parameter sent by each of the smart home devices;

[0012] The first cleaning model is updated using the third model parameters, and the step of generating collaborative training instructions with the first cleaning model is returned to until a preset iteration condition is met, and the iteration is stopped to obtain the target cleaning model.

[0013] In some feasible implementations, generating collaborative training instructions for the first cleaning model includes:

[0014] extracting first model parameters from the first cleaning model;

[0015] Get collaborative training request;

[0016] Homomorphically encrypt the collaborative training request and the first model parameters to generate a collaborative training instruction for the first clean model.

[0017] In some feasible implementations, performing homomorphic encryption on the collaborative training request and the first model parameters to generate a collaborative training instruction for the first clean model includes:

[0018] Obtain the device identification of the sweeping robot and the timestamp corresponding to the current moment;

[0019] Homomorphically encrypt the collaborative training request, the first model parameters, the device identifier, and the timestamp to generate a collaborative training instruction for the first clean model.

[0020] In some feasible implementations, performing federated learning aggregation on the first model parameters and the second model parameters to obtain aggregated third model parameters includes:

[0021] Obtaining a first weight value corresponding to the sweeping robot;

[0022] Obtaining a second weight value corresponding to the smart home device;

[0023] The first weight value and the first model parameter, the second weight value and the second model parameter are used to perform federated learning aggregation to obtain aggregated third model parameters.

[0024] In some feasible implementations, the control instruction includes a voice control instruction, and the obtaining of environmental data of an environment in which the cleaning robot is located and device data of the cleaning robot in response to the control instruction for the cleaning robot includes:

[0025] Obtaining voice control instructions for the sweeping robot;

[0026] Extracting voiceprint information corresponding to the voice control command; if the voiceprint information represents a voiceprint corresponding to an authorized target user, obtaining environmental data of the environment in which the sweeping robot is located and device data of the sweeping robot.

[0027] Some possible implementations also include:

[0028] During the cleaning operation, if the sweeping robot collects a face image and / or an image of a sensitive object, the face image and / or the image of the sensitive object is blurred.

[0029] An embodiment of the present invention further discloses a data processing device for a sweeping robot, which is applied to the sweeping robot. The sweeping robot and several smart home devices are located in the same home local area network. The device includes:

[0030] a data acquisition module, configured to acquire environmental data of an environment in which the cleaning robot is located and device data of the cleaning robot in response to a control instruction directed to the cleaning robot;

[0031] A cleaning execution module is used to input the environmental data and the device data into a target cleaning model, obtain cleaning instructions for the environment, and execute cleaning operations corresponding to the cleaning instructions. The target cleaning model is a local model trained by federated learning between the sweeping robot and the smart home device.

[0032] In some feasible implementations, each of the smart home devices is configured with a device model corresponding to its own device function, and the apparatus further includes:

[0033] A function data acquisition module, used to acquire function data corresponding to each of the smart home devices;

[0034] A model training module, configured to obtain an initial cleaning model, input each of the functional data into the initial cleaning model for model training, obtain a corresponding first cleaning model, and generate collaborative training instructions for the first cleaning model;

[0035] a collaboration module, configured to send the collaborative training instruction to each of the smart home devices, the collaborative training instruction including at least first model parameters of the first cleaning model, the collaborative training instruction instructing the smart home device to extract corresponding second model parameters from a device model trained according to the functional data, and perform federated learning aggregation on the first model parameters and the second model parameters to obtain aggregated third model parameters;

[0036] a parameter receiving module, configured to receive the third model parameter sent by each of the smart home devices;

[0037] An iteration module is used to update the first cleaning model using the third model parameters and return to the step of generating collaborative training instructions with the first cleaning model until a preset iteration condition is met, stop iteration, and obtain the target cleaning model.

[0038] In some feasible implementations, the model training module is specifically used to:

[0039] extracting first model parameters from the first cleaning model;

[0040] Get collaborative training request;

[0041] Homomorphically encrypt the collaborative training request and the first model parameters to generate a collaborative training instruction for the first clean model.

[0042] In some feasible implementations, the model training module is specifically used to:

[0043] Obtain the device identification of the sweeping robot and the timestamp corresponding to the current moment;

[0044] Homomorphically encrypt the collaborative training request, the first model parameters, the device identifier, and the timestamp to generate a collaborative training instruction for the first clean model.

[0045] In some feasible implementations, the collaboration module is specifically configured to:

[0046] Obtaining a first weight value corresponding to the sweeping robot;

[0047] Obtaining a second weight value corresponding to the smart home device;

[0048] The first weight value and the first model parameter, the second weight value and the second model parameter are used to perform federated learning aggregation to obtain aggregated third model parameters.

[0049] In some feasible implementations, the control instruction includes a voice control instruction, and the data acquisition module is specifically configured to:

[0050] Obtaining voice control instructions for the sweeping robot;

[0051] Extracting voiceprint information corresponding to the voice control command; if the voiceprint information represents a voiceprint corresponding to an authorized target user, obtaining environmental data of the environment in which the sweeping robot is located and device data of the sweeping robot.

[0052] Some possible implementations also include:

[0053] The image processing module is used to blur the face image and / or the sensitive object image if the sweeping robot collects the face image and / or the sensitive object image during the cleaning operation.

[0054] An embodiment of the present invention further discloses an electronic device, comprising a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other via the communication bus;

[0055] The memory is used to store computer programs;

[0056] The processor is configured to implement the method described in the embodiment of the present invention when executing the program stored in the memory.

[0057] An embodiment of the present invention further discloses a computer-readable storage medium having instructions stored thereon. When executed by one or more processors, the processors are enabled to execute the method according to the embodiment of the present invention.

[0058] The embodiments of the present invention include the following advantages:

[0059] In an embodiment of the present invention, for a sweeping robot, by deploying the sweeping robot and other smart home devices in the same home LAN, the sweeping robot can perform local processing with other smart home devices. When the user inputs a control instruction for the sweeping robot, the sweeping robot can obtain environmental data of the environment in which it is located and device data of the sweeping robot, and then input the environmental data and device data into a target cleaning model to obtain cleaning instructions for the environment and execute cleaning operations corresponding to the cleaning instructions. Among them, the target cleaning model is a local model trained by using federated learning between the sweeping robot and the smart home devices. Therefore, by deploying the sweeping robot in a local home LAN, local processing of data is achieved, and the risk of leakage in the process of uploading data to the cloud is reduced. At the same time, by performing federated learning training on the corresponding cleaning model with other smart home devices, through localized model training, not only privacy security is improved, but also the sweeping robot can adapt to different cleaning environments, the level of intelligent cleaning is improved, and the cleaning effect of the sweeping robot is ensured. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 This is a flowchart of the steps of a data processing method of a sweeping robot provided in an embodiment of the present invention;

[0061] Figure 2 is a schematic diagram of a scenario of localization processing provided in an embodiment of the present invention;

[0062] Figure 3 This is a structural block diagram of a data processing device of a sweeping robot provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0063] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0064] As an example, as the functions of sweeping robots increase, the amount of data they collect and process is also increasing, which has caused users to worry about privacy and security. For example, during operation, a sweeping robot collects data about its environment and uploads it to the cloud. The cloud makes decisions based on the received data and sends the results to the sweeping robot, which then performs the corresponding cleaning operation. During this process, the data uploaded by the sweeping robot may contain user privacy data, making data leakage a possibility during the upload process, resulting in the disclosure of user privacy information.

[0065] In this regard, in the present invention, for the sweeping robot, by deploying the sweeping robot and other smart home devices in the same home LAN, the sweeping robot can be locally processed with other smart home devices. When the user inputs a control instruction for the sweeping robot, the sweeping robot can obtain the environmental data of the environment and the device data of the sweeping robot, and then input the environmental data and the device data into the target cleaning model to obtain the cleaning instructions for the environment, and execute the cleaning operations corresponding to the cleaning instructions. Among them, the target cleaning model is a local model trained by the sweeping robot and the smart home devices using the federated learning method. Therefore, by deploying the sweeping robot in the local home LAN, localized data processing is achieved, and the risk of leakage in the process of uploading data to the cloud is reduced. At the same time, by training the corresponding cleaning model through federated learning with other smart home devices, through localized model training, not only privacy security is improved, but also the sweeping robot can adapt to different cleaning environments, improve the level of intelligent cleaning, and ensure the cleaning effect of the sweeping robot.

[0066] Reference Figure 1 , shows a flowchart of the steps of a data processing method for a sweeping robot provided in an embodiment of the present invention, which is applied to a sweeping robot. The sweeping robot and several smart home devices are located in the same home local area network. Specifically, the method may include the following steps:

[0067] Step 101: acquiring environmental data of an environment in which the cleaning robot is located and device data of the cleaning robot in response to a control instruction for the cleaning robot;

[0068] In an embodiment of the present invention, a home local area network can be deployed in a home environment, and all smart home devices located in the home environment can be connected to the home local area network. On the one hand, this enables the smart home devices to have the ability to connect to the Internet so as to obtain corresponding data from the cloud server. On the other hand, it enables different smart home devices to share data based on the home local area network so as to perform localized federated learning.

[0069] Optionally, a sweeping robot can be equipped with a high-performance encryption chip to provide corresponding data computing capabilities and encryption processing. At the same time, sensitive information involved in the working process of the sweeping robot, including cleaning maps, user habits, environmental images, etc., are all stored and processed locally and not synchronized with the cloud server, thereby avoiding data leakage problems in the sweeping robot.

[0070] During actual operation, the robot vacuum can respond to control commands directed to it and obtain environmental data of its environment and device data, so as to make appropriate cleaning decisions and perform corresponding cleaning operations based on the environmental data and device data. The environmental data can be real-time information reflecting the cleaning scene, and may include topographic data of the home environment (such as a floor plan), obstacle coordinates, obstacle size, ground condition information, stain information, ground slope, temperature and humidity, lighting information, and the real-time location of pedestrians or pets. The device data can be used to reflect the operating status of the robot vacuum itself, and may include hardware parameters, real-time location, path planning, network status, and power supply information, etc., which are not limited by the present invention.

[0071] Among them, users can input corresponding control commands in a variety of ways. For example, users can operate directly on the sweeping robot through physical buttons, such as pressing the start, pause, recharge and other buttons; they can also use the matching remote control for remote control to realize various functions of the sweeping robot; they can also use mobile terminal applications to send commands remotely on the mobile terminal. Whether in different rooms at home or when out and about, as long as the mobile terminal is connected to the Internet or establishes a corresponding near-field communication connection with the sweeping robot, commands can be issued to the sweeping robot. In addition, for some sweeping robots that support voice interaction, they can also recognize voice commands issued by users through voice devices such as smart speakers, such as "Start cleaning the living room" and "Return to the charging station".

[0072] In some feasible implementations, during the voice control process, when the user inputs a voice control command for the sweeping robot, the sweeping robot can obtain the voice control command for the sweeping robot, and then extract the voiceprint information corresponding to the voice control command. If the voiceprint information represents the voiceprint corresponding to the authorized target user, the environmental data of the environment in which the sweeping robot is located and the device data of the sweeping robot are obtained. By extracting the voiceprint information in the voice control command and comparing it with the voiceprint of the authorized target user, the sweeping robot can effectively identify whether the person issuing the command is an authorized user, avoid accidental touch or malicious control of the sweeping robot, effectively enhance security and privacy protection, and at the same time control the sweeping robot to work through convenient voice control, thereby improving the user experience.

[0073] Step 102: Input the environmental data and the device data into a target cleaning model, obtain cleaning instructions for the environment, and execute cleaning operations corresponding to the cleaning instructions. The target cleaning model is a local model trained by federated learning between the sweeping robot and the smart home device.

[0074] After the robot vacuum collects the corresponding environmental and device data, it can input the environmental and device data into the corresponding target cleaning model for prediction, determine the cleaning strategy for the environment, and then execute the cleaning operation corresponding to the cleaning strategy. The target cleaning model is a local model trained using federated learning between the robot vacuum and smart home devices. By deploying the robot vacuum in the local home LAN, data processing is localized, reducing the risk of data leakage during upload to the cloud. At the same time, by training the corresponding cleaning model through federated learning with other smart home devices, localized model training not only improves privacy and security, but also enables the robot vacuum to adapt to different cleaning environments, improving the level of intelligent cleaning and ensuring the cleaning effect of the robot vacuum.

[0075] In one example, it is assumed that the environmental data collected by the sweeping robot includes at least:

[0076] Visual data: RGB (Red, Green, Blue) cameras and depth sensors are used to capture floor images and identify stain types (liquid, solid), obstacles (furniture, wires), and floor materials (carpet, tile).

[0077] Spatial data: LiDAR or ToF (Time-of-Flight) sensors generate real-time 2D / 3D maps, marking obstacle coordinates and traversable areas.

[0078] Environmental parameters: Temperature and humidity sensors detect ground humidity (to avoid slipping when mopping with wet cloth), and PM2.5 sensors determine air quality (to determine whether increased vacuuming is needed).

[0079] Furthermore, the collected device data includes at least:

[0080] Hardware status: battery level (determines task priority), brush wear (affects cleaning efficiency), dust box capacity (triggering cleaning reminders).

[0081] Positioning data: The SLAM (Simultaneous Localization and Mapping) system provides real-time coordinates and coverage of the cleaned area.

[0082] User commands: Preset modes (such as "Quiet Mode" or "Deep Clean").

[0083] The robot vacuum can then fuse the collected data. Specifically, it can align the stain locations in the visual data with the SLAM map coordinates to generate a semantically labeled environmental map. It can also calculate the currently available cleaning intensity (e.g., disabling the power mode when the battery level is <30%) by combining the device's battery level and brush status. Next, it performs a structured mapping on the preprocessed data to obtain a corresponding structured vector, such as environmental features (e.g., stain type code, obstacle density), device status (battery percentage, brush life value), and task parameters (user-preset mode). This structured vector is then input into the target cleaning model for prediction, outputting the corresponding cleaning instructions, and then executing the cleaning operation corresponding to the cleaning instructions.

[0084] In some feasible implementations, a target cleaning model is configured in the sweeping robot, and the corresponding cleaning functions of the sweeping robot can be realized through the target cleaning model, such as path planning, real-time obstacle avoidance, user habit analysis, etc. Correspondingly, other smart home devices deployed in the same home local area network can also be configured with device models corresponding to their own device functions. For example, a voice recognition model can be configured in the smart speaker, and semantic recognition can be accurately performed through the voice recognition model; an image recognition model can be configured in the smart camera, and image recognition can be accurately performed through the image recognition model; a device status analysis model can be configured in the smart home appliance controller, and the user's usage habits of home appliances can be analyzed through the device status analysis model, and performance optimization can be performed based on the usage habits; a corresponding data acquisition model can be configured in the environmental sensor, and the data acquisition method can be optimized through the data acquisition model to reduce the energy consumption of the device while ensuring the normal operation of the acquisition function, etc. The present invention does not impose any restrictions on this.

[0085] For the target cleaning model, federated learning technology can be used to allow the sweeping robot to conduct model collaborative training locally with other smart home devices without uploading the original data to the cloud server. In this way, through distributed learning, not only can the security of user privacy data be guaranteed, but also the intelligence level of the sweeping robot can be improved. The sweeping robot can combine the data provided by other smart home devices to provide targeted cleaning strategies that match the current home environment, thereby improving the scene adaptability of the cleaning function.

[0086] Optionally, by obtaining the functional data corresponding to each smart home device and obtaining an initial cleaning model, each functional data is input into the initial cleaning model for model training, a corresponding first cleaning model is obtained, and a collaborative training instruction for the first cleaning model is generated, and then the collaborative training instruction is sent to each smart home device, the collaborative training instruction includes at least the first model parameter of the first cleaning model, and the collaborative training instruction is used to instruct the smart home device to extract the corresponding second model parameter from the device model trained according to the functional data, and perform federated learning aggregation on the first model parameter and the second model parameter to obtain the aggregated third model parameter. Then, the third model parameter sent by each smart home device can be received, and the first cleaning model can be updated using the third model parameter, and the step of generating the collaborative training instruction with the first cleaning model is returned until the preset iteration condition is met, the iteration is stopped, and the target cleaning model is obtained, thereby realizing model localization training through federated learning with other smart home devices, ensuring data privacy, integrating multi-device data, so that the sweeping robot can cover more scenarios and dynamically adapt to changes in the home environment. At the same time, based on diversified data, model generalization can be avoided and the robustness of the cleaning model can be improved.

[0087] Among them, functional data can be data corresponding to the functions of the smart home devices themselves. For example, smart speakers can provide user voice command data (such as "start cleaning" and "avoid the bedroom") to the sweeping robot to assist in optimizing voice interaction and task scheduling models; smart cameras can share environmental image features (such as furniture layout and obstacle types) with the sweeping robot to improve the visual recognition and path planning capabilities of the sweeping robot; smart home appliance controllers can provide home appliance status data (such as the location and working status of refrigerators and washing machines) to the sweeping robot to optimize the obstacle avoidance strategy for dynamic obstacles; environmental sensors (such as temperature and humidity, infrared sensors) can provide environmental parameters such as ground material and lighting conditions to the sweeping robot to assist in adjusting the cleaning mode (such as carpet booster vacuuming and obstacle avoidance sensitivity under strong light); other smart cleaning devices (such as window cleaning robots and vacuum cleaners) can share cleaning strategy data (such as repeated cleaning paths for stubborn stains) with the sweeping robot to improve overall cleaning efficiency.

[0088] It should be noted that whether it is a sweeping robot or other smart home devices, the device model related to its own function can be trained through the corresponding training data set. Then, based on the device model after preliminary training, the corresponding model parameters are extracted, and federated learning is performed with other smart home devices to achieve collaborative training of the model without uploading the original data to the cloud server. This distributed learning method can not only protect user privacy, but also improve the intelligence level of the sweeping robot, making it more adaptable to different home environments.

[0089] Optionally, when the sweeping robot completes the training of the cleaning model locally through multi-dimensional data and obtains the first cleaning model, it can extract the first model parameters from the first cleaning model, then obtain a collaborative training request, and then homomorphically encrypt the collaborative training request and the first model parameters to generate collaborative training instructions for the first cleaning model. In this way, in the process of collaborative model training with other smart home devices, the user privacy data is further guaranteed by encrypting the data.

[0090] Furthermore, during the encryption process, the sweeping robot can also obtain the device identification of the sweeping robot and the timestamp corresponding to the current moment, and then perform homomorphic encryption on the collaborative training request, the first model parameters, the device identification and the timestamp to generate collaborative training instructions for the first cleaning model. By adding the corresponding identification identification, the encryption of the data is improved, data attacks are effectively prevented, and the security of data transmission is ensured.

[0091] Correspondingly, when the corresponding smart home device receives the first model parameters sent by the sweeping robot, it can perform federated learning aggregation in combination with the second model parameters of its own device model to obtain the aggregated third model parameters, and then return the third model parameters to the sweeping robot, so that the sweeping robot can update the cleaning model according to the third model parameters. By obtaining diversified model parameters and combining the actual data of the sweeping robot itself, the cleaning model is updated using transfer learning technology, so that the sweeping robot can quickly learn the cleaning strategies and intelligent decision-making methods in the corresponding home environment, thereby improving the adaptability of the sweeping robot to various complex home environments.

[0092] In a specific implementation, the smart home device that receives the sweeping robot can first obtain the first weight value corresponding to the sweeping robot, then obtain the second weight value corresponding to the smart home device, and then use the first weight value and the first model parameter, the second weight value and the second model parameter to perform federated learning aggregation to obtain the aggregated third model parameter, thereby integrating the model parameters of other smart home devices and combining the actual data of the sweeping robot itself, and using transfer learning technology to update the cleaning model, so that the sweeping robot can quickly learn the cleaning strategies and intelligent decision-making methods in the corresponding home environment, thereby improving the adaptability of the sweeping robot to various complex home environments.

[0093] For example, taking federated learning between a sweeping robot and a smart camera as an example, through federated learning, the sweeping robot can learn the smart camera's more powerful image recognition model, so that it can more accurately identify facial images and images of sensitive objects in the home environment, etc., so as to provide corresponding privacy protection strategies based on the recognition results to prevent privacy issues caused by image information leakage during data use.

[0094] In one example, the model training process for federated learning may include:

[0095] First, all smart devices (robot vacuums, smart speakers, cameras, etc.) can be connected via the home LAN, using two-way certificate authentication (such as TLS 1.3 (Transport Layer Security Protocol Version 1.3)) to establish a secure communication channel. Each device is pre-installed with a unique digital certificate to ensure communication is safe from eavesdropping and spoofing. Next, a smart home device (such as a smart home gateway) is selected as the federated learning coordinator, or a decentralized architecture is adopted. All devices negotiate to generate a homomorphic encryption key pair. The public key is shared with all participating devices, while the private key is retained by each participant.

[0096] During local model training, robot vacuums use their own sensor data (lidar, collision sensors) to train their cleaning models, smart cameras fine-tune their object recognition models using real-time images, and smart speakers optimize their command understanding models through voice interaction. This allows each device to complete model training locally using lightweight machine learning frameworks (such as TensorFlow Lite), eliminating the need to upload raw data to cloud servers. After each smart home device completes model training, it extracts key model parameters (such as the neural network weight matrix) and normalizes them to ensure consistent scale. Then the Paillier homomorphic encryption algorithm can be used. For example, the sweeping robot converts the parameters of the cleaning model into numerical vectors, and uses the public key of other devices to encrypt each parameter to generate a ciphertext parameter package. At the same time, differential privacy noise (such as random values ​​within the range of ±0.05) is added to prevent parameter back-inference and improve data security. The encrypted parameters are then transmitted to the smart home device through a secure channel. After receiving the model parameters sent by the sweeping robot, the corresponding smart home device can perform weighted summation (assigning weights according to the amount of data of each device, etc.), and then perform mean calculation based on the weighted summation result (normalization of the total weight), and finally perform parameter clipping (limiting the impact of outliers). The entire calculation process is completed in an encrypted state to ensure data security.

[0097] After the corresponding smart home device completes the aggregation of model parameters, the aggregated ciphertext parameter package can be returned to the sweeping robot. The sweeping robot can decrypt the aggregated parameters with its own private key and use hash verification to ensure that the parameters have not been tampered with during transmission. The aggregated parameters are then mixed with the local original parameters in a ratio of 7:3 (to prevent sudden deviation from local features). The validation dataset is then run on the updated model to evaluate the improvement in the obstacle avoidance success rate. If the effect meets the requirements, the new model is retained; otherwise, the previous version is returned. Further, optimization can be performed through continuous iteration. For example, a round of collaborative training can be automatically initiated every 8 hours, and then the participating nodes can be dynamically adjusted based on the device power (less than 20% will not participate) and network quality (delay > 100ms will skip), etc. This is not limited by the present invention.

[0098] Through the above process, by deploying the sweeping robot in the local home LAN, local data processing is achieved, reducing the risk of data leakage during uploading to the cloud. At the same time, by conducting federated learning training with other smart home devices, the corresponding cleaning model is trained through localized model training, which not only improves privacy security, but also enables the sweeping robot to adapt to different cleaning environments, improves the level of intelligent cleaning, and ensures the cleaning effect of the sweeping robot.

[0099] Furthermore, during the cleaning process, if the robot vacuum captures facial images and / or images of sensitive items, the facial images and / or images of sensitive items are blurred. By blurring specific images, the leakage of private data can be effectively avoided, thereby improving data security. Privacy blurring is a protection method that directly affects user privacy data, complementing data localization and federated learning. In this embodiment of the present invention, data localization ensures the local storage and security of private data, providing processing targets for privacy blurring, while the model trained by federated learning helps improve the accuracy and efficiency of privacy blurring.

[0100] In some examples, image blurring is performed through federated learning, where image data from multiple smart devices (such as smart cameras) is jointly trained to build a more robust image recognition model. This model is continuously optimized through federated learning, improving the recognition accuracy of private areas such as faces, ID cards, and valuables. During privacy blurring, a trained high-precision image recognition model is used to analyze images captured by the robot vacuum's camera in real time to precisely locate private areas (such as the coordinates of a face or the outline of an ID card). Based on the characteristics of the private area, an adaptive blurring algorithm (such as Gaussian blur for faces to render facial features unrecognizable, or mosaic processing for ID cards to blur text and images) is used. The results are not stored; blurring is performed in real time only while the image is being displayed, improving the effectiveness and accuracy of the privacy blurring process.

[0101] It's important to note that in the aforementioned process, the robot vacuum's data localization provides a secure data foundation for federated learning and privacy obfuscation. By storing and encrypting the user's private data locally, it prevents data theft during transmission and provides a reliable data source for subsequent federated learning and privacy obfuscation operations. Furthermore, in federated learning, locally stored data is used only for local model training, eliminating the need to upload raw data to the cloud, reducing the risk of data leakage. In privacy obfuscation, locally stored image data is obfuscated to protect user privacy. Without data localization, the effectiveness of federated learning and privacy obfuscation is significantly reduced, as data is more vulnerable to attacks during cloud storage and transmission.

[0102] Accordingly, federated learning, in conjunction with data localization, can enhance the intelligence of robot vacuums while protecting privacy. Furthermore, through federated learning, robot vacuums can collaborate with other smart devices to train models locally, leveraging data from other devices to optimize their own models without sharing raw data. This not only protects user privacy but also enables robot vacuums to adapt to diverse home environments, improving cleaning efficiency and intelligent decision-making capabilities. Furthermore, the more accurate models trained by federated learning can support privacy obfuscation, for example, more accurately identifying private areas in images and enhancing the effectiveness of privacy obfuscation.

[0103] In some examples, during federated learning, robot vacuums in different households upload locally trained model parameters tailored to their respective home environments (e.g., path planning parameters for complex furniture layouts, obstacle avoidance parameters for different obstacles (carpets, steps, wires, etc.), and cleaning strategy parameters for different floor materials (ceramic tiles, wood floors, carpets)) in an encrypted format to a decentralized local server or share them on the home network. After obtaining these diverse model parameters, the robot vacuums use transfer learning techniques to fine-tune the local models based on the actual data from their own homes.

[0104] For example, when a sweeping robot enters a new home environment and detects a furniture layout that is different from the previous training data, it extracts the path planning model parameters under a similar layout from the shared model parameters, migrates them to the local model, and uses a small amount of data from the current home for fine-tuning, thereby quickly learning the cleaning strategies and intelligent decision-making methods in that home environment, enabling it to better adapt to various complex home environments.

[0105] It should be noted that the embodiments of the present invention include but are not limited to the above examples. It is understandable that those skilled in the art can also make settings according to actual needs under the guidance of the ideas of the embodiments of the present invention, and the present invention does not limit this.

[0106] In an embodiment of the present invention, for a sweeping robot, by deploying the sweeping robot and other smart home devices in the same home LAN, the sweeping robot can perform local processing with other smart home devices. When the user inputs a control instruction for the sweeping robot, the sweeping robot can obtain environmental data of the environment in which it is located and device data of the sweeping robot, and then input the environmental data and device data into a target cleaning model to obtain cleaning instructions for the environment and execute cleaning operations corresponding to the cleaning instructions. Among them, the target cleaning model is a local model trained by using federated learning between the sweeping robot and the smart home devices. Therefore, by deploying the sweeping robot in a local home LAN, local processing of data is achieved, and the risk of leakage in the process of uploading data to the cloud is reduced. At the same time, by performing federated learning training on the corresponding cleaning model with other smart home devices, through localized model training, not only privacy security is improved, but also the sweeping robot can adapt to different cleaning environments, the level of intelligent cleaning is improved, and the cleaning effect of the sweeping robot is ensured.

[0107] In order to enable those skilled in the art to better understand the technical solutions in the embodiments of the present invention, the following examples are provided for illustrative purposes:

[0108] As an example, see Figure 2, shows a schematic diagram of the scenario of local processing provided in an embodiment of the present invention. For a sweeping robot, it can ensure the privacy and security of users and improve the user experience through data localization processing and data encryption storage. At the same time, it can also conduct model collaborative training with other smart home devices locally without uploading original data to the cloud. The distributed learning method can not only protect user privacy, but also improve the intelligence level of the sweeping robot, making it more adaptable to different home environments.

[0109] To protect camera privacy, robot vacuums can be equipped with intelligent recognition systems that automatically blur images of faces and sensitive items during operation to prevent privacy leaks. Furthermore, robot vacuums can employ acoustic fingerprint recognition technology to extract the MFCC (Mel-Frequency Cepstral Coefficients) features of the user's voice and compare them using a deep neural network, with an error rate of ≤0.01%. Furthermore, the camera automatically de-codes private areas such as faces and ID cards while in operation, and the processing results are not stored to protect user privacy. Furthermore, robot vacuums can employ advanced acoustic fingerprint technology to authenticate users by recognizing their unique voice characteristics. Only authorized users can control the robot vacuum via voice commands, effectively preventing unauthorized access and data leaks.

[0110] In addition, the sweeping robot can also continuously update the system through system monitoring and updates, and continuously optimize the system based on user feedback and technical progress.

[0111] It should be noted that for the sake of simplicity, the method embodiments are described as a series of actions. However, those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in this specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0112] Reference Figure 3 , shows a structural block diagram of a data processing device for a sweeping robot provided in an embodiment of the present invention, which is applied to a sweeping robot. The sweeping robot and several smart home devices are located in the same home local area network, and specifically may include the following modules:

[0113] A data acquisition module 301 is configured to acquire environmental data of an environment in which the cleaning robot is located and device data of the cleaning robot in response to a control instruction for the cleaning robot;

[0114] The cleaning execution module 302 is used to input the environmental data and the device data into the target cleaning model, obtain cleaning instructions for the environment, and execute cleaning operations corresponding to the cleaning instructions. The target cleaning model is a local model trained by federated learning between the sweeping robot and the smart home device.

[0115] In some feasible implementations, each of the smart home devices is configured with a device model corresponding to its own device function, and the apparatus further includes:

[0116] A function data acquisition module, used to acquire function data corresponding to each of the smart home devices;

[0117] A model training module, configured to obtain an initial cleaning model, input each of the functional data into the initial cleaning model for model training, obtain a corresponding first cleaning model, and generate collaborative training instructions for the first cleaning model;

[0118] a collaboration module, configured to send the collaborative training instruction to each of the smart home devices, the collaborative training instruction including at least first model parameters of the first cleaning model, the collaborative training instruction instructing the smart home device to extract corresponding second model parameters from a device model trained according to the functional data, and perform federated learning aggregation on the first model parameters and the second model parameters to obtain aggregated third model parameters;

[0119] a parameter receiving module, configured to receive the third model parameter sent by each of the smart home devices;

[0120] An iteration module is used to update the first cleaning model using the third model parameters and return to the step of generating collaborative training instructions with the first cleaning model until a preset iteration condition is met, stop iteration, and obtain the target cleaning model.

[0121] In some feasible implementations, the model training module is specifically used to:

[0122] extracting first model parameters from the first cleaning model;

[0123] Get collaborative training request;

[0124] Homomorphically encrypt the collaborative training request and the first model parameters to generate a collaborative training instruction for the first clean model.

[0125] In some feasible implementations, the model training module is specifically used to:

[0126] Obtain the device identification of the sweeping robot and the timestamp corresponding to the current moment;

[0127] Homomorphically encrypt the collaborative training request, the first model parameters, the device identifier, and the timestamp to generate a collaborative training instruction for the first clean model.

[0128] In some feasible implementations, the collaboration module is specifically configured to:

[0129] Obtaining a first weight value corresponding to the sweeping robot;

[0130] Obtaining a second weight value corresponding to the smart home device;

[0131] The first weight value and the first model parameter, the second weight value and the second model parameter are used to perform federated learning aggregation to obtain aggregated third model parameters.

[0132] In some feasible implementations, the control instruction includes a voice control instruction, and the data acquisition module 301 is specifically configured to:

[0133] Obtaining voice control instructions for the sweeping robot;

[0134] Extracting voiceprint information corresponding to the voice control command; if the voiceprint information represents a voiceprint corresponding to an authorized target user, obtaining environmental data of the environment in which the sweeping robot is located and device data of the sweeping robot.

[0135] Some possible implementations also include:

[0136] The image processing module is used to blur the face image and / or the sensitive object image if the sweeping robot collects the face image and / or the sensitive object image during the cleaning operation.

[0137] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0138] In addition, an embodiment of the present invention also provides an electronic device, including: a processor, a memory, and a computer program stored in the memory and runnable on the processor. When the computer program is executed by the processor, the various processes of the above-mentioned data processing method embodiment of the sweeping robot are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0139] The present invention also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the various processes of the above-mentioned data processing method embodiment of the sweeping robot are implemented, and the same technical effects are achieved. To avoid repetition, the details are not described here. The computer-readable storage medium is, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0140] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0141] It should be understood by those skilled in the art that the embodiments of the present invention may be provided as methods, apparatuses, or computer program products. Therefore, the embodiments of the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the embodiments of the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, EEPROM, Flash, and eMMC, etc.) containing computer-usable program code.

[0142] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0143] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0144] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0145] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they become aware of the basic creative concepts. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0146] Finally, it should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or terminal device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or terminal device that includes the element.

[0147] The above is a detailed introduction to a data processing method for a sweeping robot and a data processing device for a sweeping robot provided by the present invention. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A data processing method for a sweeping robot, characterized in that: Applied to a sweeping robot, the sweeping robot and several smart home devices are located in the same home local area network, and the method includes: In response to a control instruction for the cleaning robot, acquiring environmental data of an environment in which the cleaning robot is located and device data of the cleaning robot; Inputting the environment data and the device data into a target cleaning model to obtain cleaning instructions for the environment and executing cleaning operations corresponding to the cleaning instructions, wherein the target cleaning model is a local model trained by federated learning between the robot vacuum cleaner and the smart home device; The smart home device is a smart home device other than a sweeping robot, and each smart home device is configured with a device model corresponding to its own device function. The method further includes: Obtaining functional data corresponding to each of the smart home devices; Acquire an initial cleaning model, input each of the functional data into the initial cleaning model for model training, obtain a corresponding first cleaning model, and generate collaborative training instructions for the first cleaning model; Sending the collaborative training instruction to each of the smart home devices, the collaborative training instruction including at least first model parameters of the first cleaning model, the collaborative training instruction being used to instruct the smart home device to extract corresponding second model parameters from a device model trained according to the functional data, and performing federated learning aggregation on the first model parameters and the second model parameters to obtain aggregated third model parameters; receiving a third model parameter sent by each of the smart home devices; The first cleaning model is updated using the third model parameters, and the step of generating collaborative training instructions with the first cleaning model is returned to until a preset iteration condition is met, and the iteration is stopped to obtain the target cleaning model.

2. The method according to claim 1, characterized in that The generating of the collaborative training instruction for the first cleaning model includes: extracting first model parameters from the first cleaning model; Get collaborative training request; Homomorphically encrypt the collaborative training request and the first model parameters to generate a collaborative training instruction for the first clean model.

3. The method according to claim 2, characterized in that The performing homomorphic encryption on the collaborative training request and the first model parameters to generate a collaborative training instruction for the first clean model includes: Obtain the device identification of the sweeping robot and the timestamp corresponding to the current moment; Homomorphically encrypt the collaborative training request, the first model parameters, the device identifier, and the timestamp to generate a collaborative training instruction for the first clean model.

4. The method according to claim 1, wherein The performing federated learning aggregation on the first model parameters and the second model parameters to obtain aggregated third model parameters includes: Obtaining a first weight value corresponding to the sweeping robot; Obtaining a second weight value corresponding to the smart home device; The first weight value and the first model parameter, the second weight value and the second model parameter are used to perform federated learning aggregation to obtain aggregated third model parameters.

5. The method according to claim 1, wherein The control instruction includes a voice control instruction, and the step of obtaining environmental data of an environment in which the cleaning robot is located and device data of the cleaning robot in response to the control instruction for the cleaning robot includes: Obtaining voice control instructions for the sweeping robot; Extracting voiceprint information corresponding to the voice control command; if the voiceprint information represents a voiceprint corresponding to an authorized target user, obtaining environmental data of the environment in which the sweeping robot is located and device data of the sweeping robot.

6. The method according to claim 1, wherein Also includes: During the cleaning operation, if the sweeping robot collects a face image and / or an image of a sensitive object, the face image and / or the image of the sensitive object is blurred.

7. A data processing device for a sweeping robot, characterized in that: Applied to a sweeping robot, the sweeping robot and several smart home devices are located in the same home local area network, and the device includes: a data acquisition module, configured to acquire environmental data of an environment in which the cleaning robot is located and device data of the cleaning robot in response to a control instruction directed to the cleaning robot; a cleaning execution module, configured to input the environment data and the device data into a target cleaning model, obtain cleaning instructions for the environment, and execute cleaning operations corresponding to the cleaning instructions, wherein the target cleaning model is a local model trained using federated learning between the robot vacuum cleaner and the smart home device; The smart home devices are other smart home devices except the sweeping robot, and each of the smart home devices is configured with a device model corresponding to its own device function. The apparatus further includes: A function data acquisition module, used to acquire function data corresponding to each of the smart home devices; A model training module, configured to obtain an initial cleaning model, input each of the functional data into the initial cleaning model for model training, obtain a corresponding first cleaning model, and generate collaborative training instructions for the first cleaning model; a collaboration module, configured to send the collaborative training instruction to each of the smart home devices, the collaborative training instruction including at least first model parameters of the first cleaning model, the collaborative training instruction instructing the smart home device to extract corresponding second model parameters from a device model trained according to the functional data, and perform federated learning aggregation on the first model parameters and the second model parameters to obtain aggregated third model parameters; a parameter receiving module, configured to receive the third model parameter sent by each of the smart home devices; An iteration module is used to update the first cleaning model using the third model parameters and return to the step of generating collaborative training instructions with the first cleaning model until a preset iteration condition is met, stop iteration, and obtain the target cleaning model.

8. An electronic device, characterized in that: comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; The memory is used to store computer programs; The processor is configured to implement the method according to any one of claims 1 to 7 when executing a program stored in the memory.

9. A computer-readable storage medium having instructions stored thereon, which, when executed by one or more processors, cause the processors to perform the method according to any one of claims 1 to 7.

Images