Unmanned cleaning robot cleaning control system based on deep learning

Abstract

The invention relates to the field of unmanned cleaning robots, and particularly discloses an unmanned cleaning robot cleaning control system based on deep learning, which comprises the following steps: firstly, acquiring an unmanned cleaning robot to-be-cleaned area image acquired by a camera and user-marked key cleaning area data acquired by a database, and then utilizing a deep learning technology to acquire a to-be-cleaned area image of an unmanned cleaning robot; and feature extraction and correlation analysis are performed on the two images, and finally the cleaning path of the unmanned cleaning robot is generated through the generator, so that the cleaning robot can adapt to different environments and requirements, the cleaning efficiency and quality of the unmanned cleaning robot are improved, and the user experience is improved.

Description

Technical Field

[0001] The present application relates to the field of unmanned cleaning robots, and more specifically, to a cleaning control system of an unmanned cleaning robot based on deep learning. Background Art

[0002] With the continuous advancement of AI technology, smart devices have penetrated into all aspects of people's lives, and cleaning robots are a typical example. Cleaning robots, also known as sweeping robots, can move autonomously in the space that needs to be cleaned, removing dust and other debris on the ground by vacuuming without manual operation. These robots have intelligent control systems that can clean the floor of a designated area without human intervention.

[0003] Most cleaning robots currently on the market use a full-area cleaning strategy, that is, after planning the entire cleaning area, they clean it one by one. However, there is a problem with this strategy: due to the different frequencies and natures of people's activities in different areas, the degree of dirtiness in each area is also different. If the cleaning robot simply cleans the entire area according to the established cleaning route, it may not be able to focus on cleaning particularly dirty areas, which not only reduces the cleaning efficiency, but may also cause the robot to repeatedly clean the same area, resulting in low cleaning efficiency of the cleaning robot.

[0004] Therefore, an unmanned cleaning robot cleaning control system based on deep learning is desired. Summary of the invention

[0005] In order to solve the above technical problems, this application is proposed. The embodiment of this application provides an unmanned cleaning robot cleaning control system based on deep learning, which first obtains the image of the unmanned cleaning robot to be cleaned collected by the camera and the user-marked key cleaning area data collected by the database, and then uses deep learning technology to perform feature extraction and correlation analysis on the two, and finally generates the cleaning path of the unmanned cleaning robot through a generator, so that the cleaning robot can adapt to different environments and needs, thereby improving the cleaning efficiency and quality of the unmanned cleaning robot and enhancing the user experience.

[0006] According to one aspect of the present application, a deep learning-based unmanned cleaning robot cleaning control system is provided, which includes:

[0007] The unmanned cleaning robot cleaning data acquisition module is used to obtain the image of the unmanned cleaning robot to be cleaned collected by the camera and the user-marked key cleaning area data collected by the database;

[0008] The unmanned cleaning robot cleaning data extraction module is used to extract the global fusion feature vector of the area to be cleaned and the global semantic feature vector of the key cleaning marked by the user from the image of the area to be cleaned by the unmanned cleaning robot collected by the camera and the key cleaning area data marked by the user collected by the database;

[0009] The cleaning path generation module is used to generate a cleaning path for the unmanned cleaning robot based on the global fusion feature vector of the area to be cleaned and the global semantic feature vector of the key cleaning marked by the user.

[0010] Compared with the prior art, the present application provides an unmanned cleaning robot cleaning control system based on deep learning, which first obtains the image of the unmanned cleaning robot to be cleaned collected by the camera and the user-marked key cleaning area data collected by the database, and then uses deep learning technology to perform feature extraction and correlation analysis on the two, and finally uses a generator to generate the cleaning path of the unmanned cleaning robot, so that the cleaning robot can adapt to different environments and needs, thereby improving the cleaning efficiency and quality of the unmanned cleaning robot and enhancing the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0012] Figure 1 This is a block diagram of an unmanned cleaning robot cleaning control system based on deep learning according to an embodiment of the present application.

[0013] Figure 2 This is a block diagram of an unmanned cleaning robot cleaning data extraction module in an unmanned cleaning robot cleaning control system based on deep learning according to an embodiment of the present application.

[0014] Figure 3 This is a block diagram of a feature extraction unit for the area to be cleaned in a deep learning-based unmanned cleaning robot cleaning control system according to an embodiment of the present application.

[0015] Figure 4 This is a block diagram of a cleaning path generation module in a deep learning-based unmanned cleaning robot cleaning control system according to an embodiment of the present application.

[0016] Figure 5 is a block diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0017] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0018] Figure 1 : is a block diagram of an unmanned cleaning robot cleaning control system based on deep learning according to an embodiment of the present application. Figure 1 As shown, according to the unmanned cleaning robot cleaning control system 100 based on deep learning according to the embodiment of the present application, it includes: an unmanned cleaning robot cleaning data acquisition module 110, which is used to obtain the image of the area to be cleaned by the unmanned cleaning robot collected by the camera and the key cleaning area data marked by the user collected by the database; an unmanned cleaning robot cleaning data extraction module 120, which is used to extract the global fusion feature vector of the area to be cleaned and the user-marked key cleaning global semantic feature vector from the image of the area to be cleaned by the unmanned cleaning robot collected by the camera and the user-marked key cleaning area data collected by the database; a cleaning path generation module 130, which is used to generate the cleaning path of the unmanned cleaning robot based on the global fusion feature vector of the area to be cleaned and the user-marked key cleaning global semantic feature vector.

[0019] In the above-mentioned unmanned cleaning robot cleaning control system 100 based on deep learning, the unmanned cleaning robot cleaning data acquisition module 110 is used to obtain the image of the unmanned cleaning robot to be cleaned area collected by the camera and the user-marked key cleaning area data collected by the database. It should be understood that with the continuous development of artificial intelligence technology, intelligent devices have been deeply integrated into our daily lives, of which cleaning robots are an obvious example. These robots, commonly known as sweeping robots, can automatically move in the space that needs to be cleaned, and use the vacuuming function to remove dust and debris on the ground without manual operation. They are equipped with an intelligent control system that can complete the floor cleaning of the designated area without human intervention. At present, most cleaning robots on the market adopt a cleaning mode that covers the entire area, that is, after planning the entire cleaning area, each part is cleaned in sequence. However, there is a problem with this method: due to the different frequencies and natures of use in different areas, the degree of dirtiness of each area varies greatly. If the robot only cleans the entire area according to a fixed route, it may not be able to focus on cleaning particularly dirty areas. This not only reduces the cleaning efficiency, but also may cause the robot to repeatedly clean the same area, thereby reducing the overall cleaning efficiency. Therefore, in the technical solution of the present application, by obtaining the images of the area to be cleaned by the unmanned cleaning robot captured by the camera and the data of key cleaning areas marked by users collected by the database, and combining deep learning technology, the cleaning path of the unmanned cleaning robot is formulated so that it can adapt to various environments and needs, thereby optimizing the cleaning effect, improving efficiency, and enhancing the user experience.

[0020] Specifically, the purpose of obtaining the image of the area to be cleaned by the unmanned cleaning robot collected by the camera and the key cleaning area data marked by the user collected by the database is to achieve efficient and accurate cleaning path planning. Among them, the images collected by the camera provide real-time environmental information, including the state of the ground, the location of obstacles, and the overall layout of the area. These image data help the system monitor and analyze the current situation of the cleaning area in real time to ensure that the robot can adapt to the dynamically changing environment. At the same time, the user-marked data collected by the database provides the user's personalized needs, such as key cleaning requirements in specific areas. These data may include the user's special attention to certain areas or specific cleaning instructions, which are crucial for the robot to plan the path and ensure that it meets the user's personalized cleaning requirements. Combining these two, the robot can develop a more intelligent and efficient cleaning path, which not only ensures comprehensive coverage of the area, but also gives priority to areas that users consider important, thereby improving cleaning quality and user satisfaction.

[0021] In a specific embodiment of the present application, first, a garbage detection algorithm model is developed using deep learning technology, and the model is deployed on the platform. Then, the camera regularly captures photos of the scene and calls the platform detection algorithm. If garbage is detected, the platform sends a location instruction to the unmanned cleaning robot. Finally, after receiving the instruction, the unmanned cleaning robot finds the optimal path to reach the designated location for cleaning operations.

[0022] In the above-mentioned unmanned cleaning robot cleaning control system 100 based on deep learning, the unmanned cleaning robot cleaning data extraction module 120 is used to extract the global fusion feature vector of the area to be cleaned and the global semantic feature vector of the key cleaning marked by the user from the image of the area to be cleaned by the unmanned cleaning robot collected by the camera and the key cleaning area data marked by the user collected by the database. In this way, the system can realize a comprehensive and accurate analysis of the area to be cleaned. Among them, when planning the cleaning path, the global fusion feature vector provides an overall view of the environment, and the user-marked feature vector ensures that the path planning meets the actual needs of the user.

[0023] Figure 2 FIG. 1 is a block diagram of an unmanned cleaning robot cleaning data extraction module in an unmanned cleaning robot cleaning control system based on deep learning according to an embodiment of the present application. Figure 2 As shown, in a specific embodiment of the present application, the unmanned cleaning robot cleaning data extraction module 120 includes: an attention feature encoding unit 121 for the area to be cleaned, which is used to encode the attention feature of the image of the area to be cleaned by the unmanned cleaning robot collected by the camera to obtain an attention feature vector of the area to be cleaned; a feature extraction unit 122 for the area to be cleaned, which is used to extract regional features from the image of the area to be cleaned by the unmanned cleaning robot collected by the camera to obtain a feature vector associated with the area to be cleaned by the unmanned cleaning robot; a global feature fusion unit 123 for the area to be cleaned, which is used to fuse the attention feature vector of the area to be cleaned and the feature vector associated with the area to be cleaned by the unmanned cleaning robot to obtain a global fusion feature vector of the area to be cleaned; and a user-marked cleaning feature extraction unit 124, which is used to extract features from the user-marked key cleaning area data collected by the database to obtain the user-marked key cleaning global semantic feature vector.

[0024] It should be understood that in the actual unmanned cleaning robot cleaning environment, the image data contains a lot of information, but not all areas are equally important. For example, dirt, obstacles, and various terrain features on the ground have different degrees of influence on the cleaning task. Attention feature coding technology can automatically identify and emphasize these key parts by weighting the important areas in the image, thereby improving the system's understanding and processing capabilities of the cleaning task. Specifically, attention feature coding identifies the most critical areas for the cleaning task by analyzing various features in the image. It can generate a weighted feature representation, namely the attention feature vector of the area to be cleaned, by calculating the relative importance of different areas in the image. This feature vector can highlight those areas that need special attention, such as dirt accumulation points or ground types that are difficult to clean, while ignoring those relatively simple or irrelevant areas. This processing method not only improves the efficiency of feature extraction, but also enables the system to formulate cleaning strategies in a more precise way. By applying attention feature coding to image data, the system can generate a more refined environmental model that emphasizes important areas and provides a more targeted basis for path planning. Ultimately, this approach helps ensure that the cleaning robot can prioritize key areas in actual operation, improve cleaning results, reduce redundant operations, and thus improve the efficiency and effectiveness of the entire cleaning process. Through attention feature encoding, the system can not only better adapt to complex environments, but also optimize the cleaning path while meeting the user's cleaning needs.

[0025] Furthermore, the regional feature extraction is performed on the image of the area to be cleaned by the unmanned cleaning robot collected by the camera to obtain the associated feature vector of the area to be cleaned by the unmanned cleaning robot, in order to capture the detailed features of the environment in the image data and convert these features into specific data that can be used for path planning. The image contains rich information, including the texture of the ground, the location of obstacles, and the cleaning requirements of different areas. This information usually exists in the form of raw pixels and needs to be converted into useful feature representations through the feature extraction process for effective analysis and processing. Among them, the main purpose of regional feature extraction is to identify and extract key information of different areas from the image. This process uses deep learning models such as convolutional neural networks (CNN) to process the image layer by layer to obtain regional features. The extracted associated feature vector can accurately describe the specific characteristics and relative position of the area to be cleaned, providing detailed environmental information for the path planning of the system.

[0026] Furthermore, the fusion of the attention feature vector of the area to be cleaned and the association feature vector of the area to be cleaned by the unmanned cleaning robot to obtain the global fusion feature vector of the area to be cleaned is to achieve a comprehensive understanding of the cleaning task and optimize the path planning. The attention feature vector provides the system with the key focus of the cleaning task by emphasizing important areas in the image (such as the location where dirt is concentrated or obstacles). These vectors highlight the areas that need special attention, allowing the cleaning robot to prioritize these high-priority areas, thereby improving the cleaning efficiency and effect. On the other hand, the association feature vector provides a global view of the environment through a detailed analysis of different areas in the image. This includes the geometry of the area, texture features, and spatial relationships between areas, which helps to understand the overall layout and structure of the environment. The association feature vector converts environmental details into structured data, allowing the system to accurately describe various characteristics of the area to be cleaned, including ground conditions and obstacle configuration. Fusion of these two feature vectors can combine the information of the key areas with the global environmental features to form a comprehensive and detailed global fusion feature vector of the area to be cleaned. With this approach, the system not only maintains accuracy when processing high-priority areas, but also takes into account the complexity of the entire environment when formulating cleaning paths, thereby avoiding omissions and redundant operations.

[0027] In particular, feature extraction is performed on the user-marked key cleaning area data collected from the database to obtain the user-marked key cleaning global semantic feature vector, in order to accurately integrate the user's specific needs and preferences, thereby optimizing the work efficiency and effect of the cleaning robot. User-marked data usually includes user priority indications or special cleaning requirements for certain areas. These marks not only reflect the user's personalized needs, but may also involve the particularity of certain areas, such as areas with higher cleaning frequency, areas that are easily dirty, or areas with special materials. The feature extraction process processes these user-marked data and converts them into a structured global semantic feature vector. This process includes analyzing the features of the user-marked area and extracting key information related to the cleaning task. For example, feature extraction can identify the spatial distribution, importance level, and related cleaning requirements of the user-marked area. This information is encoded into a global semantic feature vector to accurately describe the user's specific expectations and needs for the cleaning area.

[0028] In a specific embodiment of the present application, the attention feature encoding unit 121 of the area to be cleaned includes: passing the image of the area to be cleaned by the unmanned cleaning robot captured by the camera through a noise reduction module of the area to be cleaned by the unmanned cleaning robot to obtain a pixel enhanced image of the area to be cleaned by the unmanned cleaning robot; passing the pixel enhanced image of the area to be cleaned by the unmanned cleaning robot through an image spatial attention encoder of the area to be cleaned to obtain an attention feature vector of the area to be cleaned.

[0029] It should be understood that image data is often affected by a variety of interference factors, such as light changes, environmental noise, image blur, etc. These factors may cause unclear or incomplete image information, affecting the performance and efficiency of the cleaning robot. The main function of the noise reduction module is to suppress and remove the noise present in the image, thereby reducing unnecessary interference. By performing noise reduction on the original image, the module can effectively remove the noise caused by uneven light, sensor errors or other environmental factors, making the main features of the image more prominent. This process includes the use of various image processing techniques, such as filtering algorithms, denoising convolutional neural networks (DNNs), etc., to extract clear image details and features. After noise reduction, the generated pixel-enhanced images have higher image quality and clearer details. These enhanced images can more accurately reflect the actual state of the area to be cleaned, allowing the system to better identify important features in the image, such as dirt on the ground, the location of obstacles, and other cleaning-related details. Specifically, the image of the area to be cleaned by the unmanned cleaning robot captured by the camera is input into the encoder of the noise reduction module of the area to be cleaned by the unmanned cleaning robot, wherein the encoder uses a convolution layer to perform explicit spatial encoding on the image of the area to be cleaned by the unmanned cleaning robot captured by the camera to obtain image features; and the image features are input into the decoder of the noise reduction module of the area to be cleaned by the unmanned cleaning robot, wherein the decoder uses a deconvolution layer to perform deconvolution processing on the image features to obtain a pixel enhanced image of the area to be cleaned by the unmanned cleaning robot.

[0030] Furthermore, although the clarity of the enhanced image is improved, the image may still contain a lot of information, some of which are more critical to the cleaning task, such as places where dirt is concentrated, obstacles or special materials on the ground. These key information need special attention to ensure that the cleaning robot can complete the task efficiently. The role of the spatial attention encoder is to highlight and strengthen those areas that are most important for task execution by applying different levels of attention to different areas in the image. This encoder dynamically adjusts its attention to different areas by learning and identifying the relative importance of each spatial position in the image. Specifically, it can generate a weight map based on the content and context of the image, in which high-weight areas represent higher attention. This enables the cleaning system to focus on these key areas, thereby improving the efficiency and effectiveness of the task. In actual operation, the spatial attention encoder is usually trained using a deep learning model to learn how to evaluate the relevance and importance of each area in the image based on a large amount of annotated data. Through this process, the encoder can automatically identify which areas are the focus of cleaning, extract the features of these areas, and generate the corresponding attention feature vectors. By applying this attention feature vector to the cleaning task, the system can formulate a more accurate cleaning strategy based on specific cleaning requirements and environmental characteristics. Specifically, the convolutional coding part of the spatial attention encoder of the image of the area to be cleaned is used to perform deep convolutional coding on the pixel enhanced image of the area to be cleaned by the unmanned cleaning robot to obtain an initial convolutional feature map; the initial convolutional feature map is input into the spatial attention part of the spatial attention encoder of the image of the area to be cleaned to obtain a spatial attention map; the spatial attention map is activated by the Softmax function to obtain a spatial attention feature map; the spatial attention feature map and the initial convolutional feature map are multiplied by the position points to obtain the attention feature vector of the area to be cleaned. More specifically, the initial convolutional feature map is respectively average pooled and maximum pooled along the channel dimension to obtain an average feature matrix and a maximum feature matrix; the average feature matrix and the maximum feature matrix are cascaded and channel-adjusted to obtain a channel feature matrix; the channel feature matrix is ​​convolutionally encoded using the convolutional layer of the spatial attention feature map to obtain a spatial attention map.

[0031] Figure 3 FIG. 1 is a block diagram of a feature extraction unit for the area to be cleaned in a cleaning control system of an unmanned cleaning robot based on deep learning according to an embodiment of the present application. Figure 3As shown, in a specific embodiment of the present application, the feature extraction unit 122 of the area to be cleaned includes: a data preprocessing subunit 1221 of the area to be cleaned, which is used to perform data preprocessing on the image of the area to be cleaned by the unmanned cleaning robot acquired by the camera to obtain an image block input matrix of the area to be cleaned by the unmanned cleaning robot; and a convolutional encoding subunit 1222 of the area to be cleaned, which is used to pass the image block input matrix of the area to be cleaned by the unmanned cleaning robot through the image block convolution encoder of the area to be cleaned by the unmanned cleaning robot to obtain a feature vector associated with the area to be cleaned by the unmanned cleaning robot.

[0032] It should be understood that the original images collected by the camera usually have high resolution and rich information, but the data volume is large and the details are complex. Direct processing may lead to a waste of computing resources and extended processing time. Therefore, data preprocessing can significantly reduce the amount of image data processed each time, thereby helping to reduce computational complexity, reduce memory consumption, and make the system more efficient when processing.

[0033] Furthermore, the image block input matrix of the area to be cleaned by the unmanned cleaning robot is processed by the convolution encoder of the image block of the area to be cleaned by the unmanned cleaning robot, so as to perform feature analysis and decision making more effectively. The main role of the convolution encoder in this process is to perform in-depth feature extraction of image data through convolution operation, compress and transform the spatial information and features in the original image block, and generate a highly correlated feature vector. The convolution encoder takes advantage of the convolutional neural network (CNN) and extracts local features in the image layer by layer through the local-aware convolution layer. Each convolution layer can capture different patterns and features in the image block, such as edges, textures, and shapes, which are crucial for recognizing and understanding the content of the image. The convolution operation can automatically learn and extract important visual features by sliding the convolution kernel on the image block, thereby reducing redundant information and strengthening meaningful patterns. By stacking multiple convolutional layers and pooling layers, the convolution encoder can gradually compress the high-dimensional data of the image block into a low-dimensional feature representation. After these processing steps, the image block is encoded into a feature vector with high expressive power. These feature vectors can not only accurately describe the local features in the image block, but also capture the spatial relationship and interaction between different areas. Specifically, each layer of the unmanned cleaning robot's image block convolution encoder performs convolution processing, mean pooling processing based on the local feature matrix and nonlinear activation processing on the input data in the forward transfer of the layer, so that the last layer of the unmanned cleaning robot's image block convolution encoder outputs the unmanned cleaning robot's image block to be cleaned area-related feature vector, wherein the input of the unmanned cleaning robot's image block to be cleaned area convolution encoder is the unmanned cleaning robot's image block input matrix to be cleaned area.

[0034] In a specific embodiment of the present application, the data preprocessing subunit 1221 of the area to be cleaned includes: dividing the image of the unmanned cleaning robot's area to be cleaned captured by the camera into image blocks to obtain a sequence of image blocks of the unmanned cleaning robot's area to be cleaned; linearly embedding the sequence of image blocks of the unmanned cleaning robot's area to be cleaned through the sequence of image blocks of the area to be cleaned to obtain a plurality of image block embedding vectors of the unmanned cleaning robot's area to be cleaned; and arranging the plurality of image block embedding vectors of the unmanned cleaning robot's area to be cleaned in two dimensions to obtain an input matrix of the image blocks of the unmanned cleaning robot's area to be cleaned.

[0035] It should be understood that the original image usually has high resolution and complex information, which makes direct processing of the entire image computationally intensive and inefficient. By dividing the image into smaller image blocks, the processing speed and system responsiveness can be significantly improved. The first step of image block partitioning is to decompose the large image into multiple small blocks of fixed size. Each image block represents a local area of ​​the original image. This blocking strategy allows the system to focus on a specific area in the image. Each image block contains local detailed information, so that the system can analyze and process these areas independently, thereby improving the ability to capture details in each area. This local processing method reduces the amount of computation because the system no longer needs to process the entire large image at once, but instead processes smaller, more manageable blocks.

[0036] Furthermore, the image block sequence of the area to be cleaned by the unmanned cleaning robot is processed by linear embedding coding of the image block sequence of the area to be cleaned, in order to map the high-dimensional data in the image block sequence to a low-dimensional space, thereby simplifying data representation and improving processing efficiency. Considering that the image block sequence is usually composed of multiple image blocks, each block contains local image information, which may be too complex and redundant in the original high-dimensional space. Directly processing this data may lead to excessive computational burden and low processing efficiency. By mapping each image block to a lower-dimensional feature space through a linear transformation. This embedded coding method uses linear transformations (such as matrix multiplication) to compress high-dimensional data into low-dimensional data while retaining the important features and structure of the original data as much as possible. In this way, each image block can be represented as an embedded vector, which captures the core features of the image block and reduces the dimension and complexity of the data.

[0037] Furthermore, the two-dimensional arrangement of the embedding vectors of the image blocks of the areas to be cleaned by multiple unmanned cleaning robots is to organize the high-dimensional feature data into a structure suitable for deep learning model processing, thereby improving the data processing efficiency and the learning ability of the model. In image processing, the original image data usually exists in the form of a two-dimensional grid. The two-dimensionally arranged embedding vector matrix can maintain the characteristics of this spatial structure, so that the deep learning model can effectively capture and learn the local and global features of the image data. This arrangement ensures the continuity and structure of the spatial information, so that the model can better understand the spatial relationships and patterns in the image. Arranging the embedding vectors into a two-dimensional matrix form helps to improve the efficiency of data processing. In deep learning models, especially convolutional neural networks (CNNs), input data is usually organized in a matrix form to facilitate convolution operations, pooling operations and other matrix operations. By arranging the image block embedding vectors into a two-dimensional matrix, the existing deep learning framework and computing tools can be directly used for efficient matrix calculations, which not only improves the processing speed but also reduces the complexity of calculation and programming.

[0038] In a specific embodiment of the present application, the feature extraction unit 123 of the area to be cleaned includes: passing the user-marked key cleaning area data collected from the database through a user-marked key cleaning semantic understanding context encoder to obtain a plurality of user-marked key cleaning semantic feature vectors; and splicing the plurality of user-marked key cleaning semantic feature vectors to obtain the user-marked key cleaning global semantic feature vector.

[0039] It should be understood that user-tagged data often exists in natural language or unstructured form, which makes it difficult to directly extract useful information from it. The role of the semantic understanding context encoder is to convert these tagged data into structured semantic feature vectors, which in turn represent the user's specific instructions and requirements for key areas. Through the context encoder, the system can analyze the semantic content of the user's tags and identify key features related to the cleaning area, such as the specific location of the area, the importance of cleaning, and potential cleaning needs. This encoding process can convert complex natural language information into vector representations that are easy to process and analyze, thereby enhancing the system's ability to understand user instructions. The context encoder improves the accuracy of feature extraction by capturing the semantic contextual relationships in the data. In user-tagged data, there may be complex relationships and dependencies between different tags. The context encoder uses the attention mechanism and context modeling technology in the deep learning model to fully understand and integrate these relationships, thereby generating a feature vector with rich semantic information. This feature vector can not only accurately reflect the user's tagging intention, but also capture the intrinsic characteristics and structure of the tagged area, improving the overall understanding of the cleaning area. Specifically, the user-marked key cleaning area data collected by the database is segmented to obtain a user-marked word sequence; the embedding layer of the user-marked key cleaning semantic understanding context encoder is used to map each user-marked word in the user-marked word sequence to a word embedding vector to obtain a sequence of user-marked word embedding vectors; the converter-based Bert model of the user-marked key cleaning semantic understanding context encoder is used to perform global context semantic encoding on the sequence of user-marked word embedding vectors to obtain multiple user-marked key cleaning semantic feature vectors.

[0040] Furthermore, the purpose of concatenating multiple user-marked key cleaning semantic feature vectors to obtain a user-marked key cleaning global semantic feature vector is to comprehensively integrate the semantic information from different user marks to form a comprehensive and unified feature representation to more accurately reflect the user's overall needs and intentions for the cleaning area. This method is of great significance in enhancing the system's understanding of user needs, optimizing cleaning strategies, and improving the intelligence of the system. The semantic feature vector marked by a single user can only reflect the user's preference or demand for a specific cleaning area, while different users may have different concerns and priorities for marking the same area. By concatenating the semantic feature vectors marked by multiple users, the information of these individual marks can be integrated into a comprehensive global semantic feature vector. This integration not only covers the views and needs of different users, but also captures the comprehensive performance of the importance and characteristics of the area, thereby providing a more comprehensive understanding and analysis basis for the system.

[0041] In the above-mentioned unmanned cleaning robot cleaning control system 100 based on deep learning, the cleaning path generation module 130 is used to generate the cleaning path of the unmanned cleaning robot based on the global fusion feature vector of the area to be cleaned and the global semantic feature vector of the user-marked key cleaning. It should be understood that the global fusion feature vector of the area to be cleaned provides a comprehensive view of the area to be cleaned, including the actual conditions of each area such as area, obstacles, dirt distribution and other information. This information helps the robot avoid obstacles when planning the path, optimize the cleaning efficiency, and ensure full coverage of the cleaning. The global semantic feature vector of the user-marked key cleaning reflects the user's special requirements for certain areas, such as high-frequency use areas, particularly dirty areas or other priority areas. These marks help the robot understand which areas are particularly important to the user, so as to give these areas more attention in path planning to ensure that the user's needs are met. By combining these two feature vectors, the robot can generate a cleaning path that meets both the actual conditions of the area and the needs of the user, thereby formulating a cleaning plan that is both efficient and meets the user's expectations, thereby improving the cleaning effect and user satisfaction.

[0042] Figure 4 : is a block diagram of a cleaning path generation module in a cleaning control system of an unmanned cleaning robot based on deep learning according to an embodiment of the present application. Figure 4 As shown, in a specific embodiment of the present application, the cleaning path generation module 130 includes: a feature association unit 131 of the area to be cleaned, which is used to associate the global fusion feature vector of the area to be cleaned and the global semantic feature vector of the user-marked key cleaning to obtain a cleaning path generation feature vector; a feature optimization unit 132 of the area to be cleaned, which is used to perform fine-grained feature-category label coherence interference correction on the cleaning path generation feature vector to obtain an optimized cleaning path generation feature vector; and an unmanned cleaning robot cleaning path generation unit 133, which is used to pass the optimized cleaning path generation feature vector through a generator to generate a cleaning path for an unmanned cleaning robot.

[0043] It should be understood that by associating the global fusion feature vector of the area to be cleaned and the global semantic feature vector of the user-marked key cleaning, a cleaning path generation feature vector can be generated, which integrates the actual characteristics of the area and the personalized needs of the user. The cleaning path generation feature vector allows the system to consider the actual cleaning difficulty of the area and the user's priority requirements when formulating the cleaning path, thereby formulating the optimal cleaning path. For example, in path planning, the system can give priority to cleaning key areas marked by the user, and optimize the path based on processing the global fusion features to avoid complex areas or obstacles, improve cleaning efficiency and comprehensive coverage.

[0044] In particular, in the technical solution of the present application, the cleaning path generation feature vector is obtained by feature processing the pixel enhanced image features of the cleaning area, the spatial and convolutional features of the image blocks, and the semantic features of the user marks. These features are highly abstract and contain various spatial and semantic information. In the generator, if the weights fail to accurately match these complex features, the generated result may be inconsistent with the expected path. In this way, there may be problems of overfitting certain features or ignoring other features, resulting in quasi-coherent interference in the generated cleaning path of the unmanned cleaning robot. Therefore, in the technical solution of the present application, the cleaning path generation feature vector is subjected to fine-grained feature-category label coherent interference correction to obtain an optimized cleaning path generation feature vector.

[0045] Among them, the sweeping path generation feature vector is subjected to fine-grained feature-category label coherent interference correction to obtain an optimized sweeping path generation feature vector, including: calculating the product between the sweeping path generation feature vector and its transposed vector to obtain a sweeping path feature autocorrelation expression matrix; calculating the coherent interference phase between the decoding weight matrix of the generator and the sweeping path feature autocorrelation expression matrix to obtain a sweeping path feature-weight fine-grained coupling representation matrix; performing probabilistic processing on the sweeping path feature-weight fine-grained coupling representation matrix based on the Sigmoid activation function to obtain a sweeping path feature-weight fine-grained coupling probabilistic representation matrix; performing back-propagation expression compensation on the sweeping path generation feature vector based on the sweeping path feature-weight fine-grained coupling probabilistic representation matrix to obtain a correlation compensation representation vector; and calculating the position-weighted sum between the correlation compensation representation vector and the sweeping path generation feature vector to obtain the optimized sweeping path generation feature vector.

[0046] The coherent interference phase between the decoding weight matrix of the generator and the sweeping path feature autocorrelation expression matrix is ​​calculated using the following phase calculation formula to obtain a sweeping path feature-weight fine-grained coupling representation matrix;

[0047] Wherein, the phase state calculation formula is:

[0048]

[0049] Among them, V 1i represents the i-th row vector of the class decoding weight matrix of the generator, V 2j represents the j-th row vector of the sweeping path feature-weight fine-grained coupling representation matrix, ∑ represents the covariance matrix of the dataset to which the i-th row vector of the class decoding weight matrix of the generator and the j-th row vector of the sweeping path feature-weight fine-grained coupling representation matrix belong, ∑ -1 represents the inverse matrix of the covariance matrix, A(V 1i,V 2j ) represents the eigenvalue of the (i, j)th position of the sweeping path feature-weight fine-grained coupling probabilistic representation matrix.

[0050] The following relevant compensation formula is used to perform back-propagation expression compensation on the feature vector generated by the cleaning path based on the feature-weight fine-grained coupling probabilistic representation matrix of the cleaning path to obtain a relevant compensation representation vector;

[0051] Wherein, the relevant compensation formula is:

[0052]

[0053] Among them, V r represents the relevant compensation representation vector, A represents the sweeping path feature-weight fine-grained coupling probabilistic representation matrix, V c represents the feature vector generated by the cleaning path, Represents matrix multiplication.

[0054] In the technical solution of the present application, when the sweeping path generating feature vector is generated by a generator, since the weight of the generator also needs to adapt to the sweeping path generating feature vector, category coherent interference with the sweeping path generating feature vector may occur. Based on this, in the technical solution of the present application, the sweeping path generating feature vector is subjected to fine-grained feature-category label coherent interference correction, which first represents the autocorrelation interference feature spectrum with the autocorrelation matrix of the sweeping path generating feature vector, and then simulates the fine-grained feature-category label coherent interference phase between the sweeping path feature and the weight matrix of the generator by calculating the correlation measure between the autocorrelation matrix of the sweeping path generating feature vector and the decoding weight matrix of the generator, and then performs back-propagation expression compensation on the sweeping path generating feature vector with the sweeping path feature-weight fine-grained coupling representation matrix, and restores the equivalent probability intensity representation of the sweeping path generating feature vector in the absence of interference by reverse compensation, and obtains the optimized sweeping path generating feature vector to improve the accuracy of the generation result.

[0055] Furthermore, the optimized cleaning path generates a feature vector that integrates the actual conditions of the area to be cleaned and the key requirements marked by the user, providing a comprehensive view and priority information of the cleaning area. This vector contains actual information such as terrain characteristics, obstacle distribution, dirt concentration in the area, as well as the user's key concerns on certain areas. This comprehensive feature provides detailed guidance for path planning, so that the generated path can cover all necessary areas and give priority to the key areas marked by the user. The cleaning path generated by the generator ensures that the cleaning path can efficiently cover the entire area to be cleaned, while giving priority to the key areas marked by the user. Based on the information in the feature vector, the generator automatically plans the best path, avoiding subjectivity and errors in manual intervention.

[0056] In summary, the embodiment of the present application first obtains the image of the area to be cleaned by the unmanned cleaning robot collected by the camera and the key cleaning area data marked by the user collected by the database, and then uses deep learning technology to perform feature extraction and correlation analysis on the two. Finally, a generator is used to generate the cleaning path of the unmanned cleaning robot, so that the cleaning robot can adapt to different environments and needs, thereby improving the cleaning efficiency and quality of the unmanned cleaning robot and enhancing the user experience.

[0057] As described above, the unmanned cleaning robot cleaning control system 100 based on deep learning according to the embodiment of the present application can be implemented in various terminal devices. In one example, the unmanned cleaning robot cleaning control system 100 based on deep learning can be integrated into the terminal device as a software module and / or hardware module. For example, the unmanned cleaning robot cleaning control system 100 based on deep learning can be a software module in the operating system of the terminal device, or can be an application developed for the terminal device; of course, the unmanned cleaning robot cleaning control system 100 based on deep learning can also be one of the many hardware modules of the terminal device.

[0058] Alternatively, in another example, the deep learning based unmanned cleaning robot cleaning control system 100 and the terminal device may also be separate devices, and the deep learning based unmanned cleaning robot cleaning control system 100 may be connected to the terminal device via a wired and / or wireless network, and transmit interactive information in accordance with an agreed data format.

[0059] Below, reference Figure 5 To describe an electronic device according to an embodiment of the present application.

[0060] See also Figure 5 , Figure 5 The hardware structure of an electronic device of another embodiment is illustrated, and the electronic device includes:

[0061] The processor 11 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solution provided in the embodiment of the present application; the memory 12 can be implemented in the form of a read-only memory (Read Only Memory, ROM), a static storage device, a dynamic storage device or a random access memory (Random Access Memory, RAM), etc.

[0062] The memory 12 can store an operating system and other application programs. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 12, and the processor 11 is called to execute the unmanned cleaning robot cleaning control system based on deep learning in the embodiment of this application;

[0063] Input / output interface 13, used to implement information input and output;

[0064] The communication interface 14 is used to realize the communication interaction between the device and other devices. The communication can be realized through a wired manner (such as USB, network cable, etc.) or a wireless manner (such as mobile network, WIFI, Bluetooth, etc.);

[0065] A bus 15 that transmits information between the various components of the device (e.g., the processor 11, the memory 12, the input / output interface 13, and the communication interface 14);

[0066] The processor 11 , the memory 12 , the input / output interface 13 and the communication interface 14 are connected to each other in communication within the device via a bus 15 .

[0067] An embodiment of the present application also provides a storage medium, which is a computer-readable storage medium. The storage medium stores a computer program, and when the computer program is executed by a processor, it implements the above-mentioned unmanned cleaning robot cleaning control system based on deep learning.

[0068] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0069] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0070] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0071] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0072] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, products or devices.

[0073] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0074] In the several embodiments provided in the present application, it should be understood that the disclosed devices and systems can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0075] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0076] In addition, each functional unit in each embodiment of the present application can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional unit. If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the system of each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-On lyMemory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs. The preferred embodiment of the embodiment of the present application is described above with reference to the accompanying drawings, and the scope of rights of the embodiment of the present application is not limited thereto. Any modifications, equivalent substitutions and improvements made by those skilled in the art without departing from the scope and essence of the embodiments of the present application shall be within the scope of rights of the embodiments of the present application.

Claims

1. A cleaning control system for an unmanned cleaning robot based on deep learning, characterized in that: include: The unmanned cleaning robot cleaning data acquisition module is used to obtain the image of the unmanned cleaning robot to be cleaned collected by the camera and the user-marked key cleaning area data collected by the database; The unmanned cleaning robot cleaning data extraction module is used to extract the global fusion feature vector of the area to be cleaned and the global semantic feature vector of the key cleaning marked by the user from the image of the area to be cleaned by the unmanned cleaning robot collected by the camera and the key cleaning area data marked by the user collected by the database; The cleaning path generation module is used to generate a cleaning path for the unmanned cleaning robot based on the global fusion feature vector of the area to be cleaned and the global semantic feature vector of the key cleaning marked by the user.

2. The unmanned cleaning robot cleaning control system based on deep learning according to claim 1 is characterized in that: The unmanned cleaning robot cleaning data extraction module comprises: The attention feature encoding unit of the area to be cleaned is used to perform attention feature encoding on the image of the area to be cleaned by the unmanned cleaning robot collected by the camera to obtain an attention feature vector of the area to be cleaned; A feature extraction unit for the area to be cleaned, used for performing regional feature extraction on the image of the area to be cleaned by the unmanned cleaning robot acquired by the camera to obtain a feature vector associated with the area to be cleaned by the unmanned cleaning robot; A global feature fusion unit for the area to be cleaned, used to fuse the attention feature vector of the area to be cleaned and the associated feature vector of the area to be cleaned by the unmanned cleaning robot to obtain a global fusion feature vector of the area to be cleaned; The user-marked cleaning feature extraction unit is used to extract features from the user-marked key cleaning area data collected from the database to obtain the user-marked key cleaning global semantic feature vector.

3. The unmanned cleaning robot cleaning control system based on deep learning according to claim 2 is characterized in that: The attention feature encoding unit of the area to be cleaned includes: The image of the area to be cleaned by the unmanned cleaning robot collected by the camera is passed through a noise reduction module of the area to be cleaned by the unmanned cleaning robot to obtain a pixel enhanced image of the area to be cleaned by the unmanned cleaning robot; The pixel enhanced image of the area to be cleaned by the unmanned cleaning robot is passed through the image space attention encoder of the area to be cleaned to obtain the attention feature vector of the area to be cleaned.

4. The unmanned cleaning robot cleaning control system based on deep learning according to claim 3 is characterized in that: The feature extraction unit of the area to be cleaned includes: The to-be-cleaned area data preprocessing subunit is used to perform data preprocessing on the image of the to-be-cleaned area by the unmanned cleaning robot acquired by the camera to obtain an input matrix of image blocks of the to-be-cleaned area by the unmanned cleaning robot; The convolutional encoding subunit of the area to be cleaned is used to input the image block matrix of the area to be cleaned by the unmanned cleaning robot through the convolutional encoder of the image block of the area to be cleaned by the unmanned cleaning robot to obtain the associated feature vector of the area to be cleaned by the unmanned cleaning robot.

5. The unmanned cleaning robot cleaning control system based on deep learning according to claim 4 is characterized in that: The data preprocessing subunit of the area to be cleaned includes: Dividing the image of the area to be cleaned by the unmanned cleaning robot captured by the camera into image blocks to obtain an image block sequence of the area to be cleaned by the unmanned cleaning robot; The image block sequence of the area to be cleaned by the unmanned cleaning robot is linearly embedded in the image block sequence of the area to be cleaned to obtain a plurality of image block embedding vectors of the area to be cleaned by the unmanned cleaning robot; The embedding vectors of the multiple image blocks of the area to be cleaned by the unmanned cleaning robot are arranged in two dimensions to obtain an input matrix of the image blocks of the area to be cleaned by the unmanned cleaning robot.

6. The unmanned cleaning robot cleaning control system based on deep learning according to claim 5 is characterized in that: The feature extraction unit of the area to be cleaned includes: The user-marked key cleaning area data collected from the database is passed through a user-marked key cleaning semantic understanding context encoder to obtain a plurality of user-marked key cleaning semantic feature vectors; The multiple user-marked key-sweep semantic feature vectors are concatenated to obtain the user-marked key-sweep global semantic feature vector.

7. The deep learning-based unmanned cleaning robot cleaning control system according to claim 6, characterized in that: The cleaning path generation module comprises: A feature associating unit for the area to be cleaned is used to associate the global fusion feature vector of the area to be cleaned with the global semantic feature vector of the user-marked key cleaning to obtain a cleaning path generation feature vector; A feature optimization unit for the area to be cleaned, used for performing fine-grained feature-category label coherence interference correction on the feature vector generated by the cleaning path to obtain an optimized feature vector generated by the cleaning path; The unmanned cleaning robot cleaning path generation unit is used to generate the unmanned cleaning robot cleaning path by passing the optimized cleaning path generation feature vector through a generator.

8. The deep learning-based unmanned cleaning robot cleaning control system according to claim 7, characterized in that: The feature optimization unit of the area to be cleaned includes: Calculating the product between the sweeping path generated feature vector and its transposed vector to obtain a sweeping path feature autocorrelation expression matrix; Calculating the coherent interference phase between the decoding weight matrix of the generator and the sweeping path feature autocorrelation expression matrix to obtain a sweeping path feature-weight fine-grained coupling representation matrix; The cleaning path feature-weight fine-grained coupling representation matrix is ​​subjected to probabilistic processing based on a Sigmoid activation function to obtain a cleaning path feature-weight fine-grained coupling probabilistic representation matrix; Based on the sweeping path feature-weight fine-grained coupling probabilistic representation matrix, back-propagating expression compensation is performed on the sweeping path generated feature vector to obtain a related compensation representation vector; A position-weighted sum is calculated between the relevant compensation representation vector and the cleaning path generation feature vector to obtain the optimized cleaning path generation feature vector.

9. The deep learning-based unmanned cleaning robot cleaning control system according to claim 8, characterized in that: Calculating the coherent interference phase between the decoding weight matrix of the generator and the sweeping path feature autocorrelation expression matrix to obtain a sweeping path feature-weight fine-grained coupling representation matrix, including: The coherent interference phase between the decoding weight matrix of the generator and the sweeping path feature autocorrelation expression matrix is ​​calculated using the following phase calculation formula to obtain a sweeping path feature-weight fine-grained coupling representation matrix; Wherein, the phase state calculation formula is: Among them, V 1i represents the i-th row vector of the class decoding weight matrix of the generator, V 2j represents the j-th row vector of the sweeping path feature-weight fine-grained coupling representation matrix, ∑ represents the covariance matrix of the dataset to which the i-th row vector of the class decoding weight matrix of the generator and the j-th row vector of the sweeping path feature-weight fine-grained coupling representation matrix belong, ∑ -1 represents the inverse matrix of the covariance matrix, A(V 1i ,V 2j ) represents the eigenvalue of the (i, j)th position of the sweeping path feature-weight fine-grained coupling probabilistic representation matrix.

10. The deep learning-based unmanned cleaning robot cleaning control system according to claim 9, characterized in that: The back-propagation expression compensation is performed on the sweeping path generated feature vector based on the sweeping path feature-weight fine-grained coupling probabilistic representation matrix to obtain a related compensation representation vector, including: The following relevant compensation formula is used to perform back-propagation expression compensation on the feature vector generated by the cleaning path based on the feature-weight fine-grained coupling probabilistic representation matrix of the cleaning path to obtain a relevant compensation representation vector; Wherein, the relevant compensation formula is: Among them, V r represents the relevant compensation representation vector, A represents the sweeping path feature-weight fine-grained coupling probabilistic representation matrix, V c represents the feature vector generated by the cleaning path, Represents matrix multiplication.

Images