A method and apparatus for semantic definition of two-dimensional maps based on RGB and entity contours

Abstract

This invention discloses a method and apparatus for semantic definition of two-dimensional maps based on RGB and entity contours. It addresses the problems of current contour-based methods for object type recognition in two-dimensional map semantic definition, which are heavily reliant on contour quality. Insufficient contour clarity or noise-induced contour breaks affect the accuracy of feature analysis and pattern matching. Furthermore, contour extraction and analysis involve complex computational processes, leading to significant resource consumption. The method includes: scanning an indoor space to acquire a geographic image of the space and various entity objects; determining the pixel values ​​of each entity object in a two-dimensional map; drawing a two-dimensional semantic map based on the pixel values ​​of each entity object; identifying the contours of all entity objects and extracting contour semantic information; and updating the two-dimensional semantic map by combining the pixel values ​​and contour semantic information.

Description

Technical Field

[0001] This invention belongs to the field of semantic definition technology of two-dimensional maps, specifically relating to a two-dimensional map semantic definition method based on RGB and entity contours. Background Technology

[0002] In the fields of interior decoration and construction assistance, mobile robots are being used more and more widely. In indoor environments, especially in unstructured indoor scenarios, the localization and path planning of mobile robots are key technical issues. Semantic maps can help robots identify and understand the layout and functional areas of indoor spaces, thereby better assisting mobile robots in localization and path planning. Currently, the semantic definition of two-dimensional maps has significant research value in the fields of interior decoration and construction assistance. By constructing indoor two-dimensional semantic maps, robots can autonomously perceive and understand their environment, helping them to perform effective path planning and navigation in complex indoor environments.

[0003] Most current semantic definitions of 2D maps adopt contour-based object type recognition methods, which typically involve the following steps:

[0004] 1. Edge detection: Detecting edges in an image using image processing techniques such as the Canny algorithm or the Sobel operator;

[0005] 2. Contour Extraction: Contour extraction algorithms are used to determine and separate the contours of individual objects in an image;

[0006] 3. Feature Analysis: Analyze the extracted contours and calculate their shape descriptors, such as perimeter, area, and moment invariants, to facilitate the differentiation of features of different objects;

[0007] 4. Pattern matching: The calculated features are matched with a pre-established object feature database to identify the type of object.

[0008] However, the current two-dimensional map semantic definition method based on contour for object type recognition has the following drawbacks: (1) It is highly dependent on the quality of the contour. If the contour in the image is not clear enough or the contour is broken due to noise, it will affect the accuracy of subsequent feature analysis and pattern matching; (2) Contour extraction and analysis involve complex calculation processes, especially when processing high-resolution images or real-time video streams, which will lead to a large consumption of computing resources. Summary of the Invention

[0009] In view of the shortcomings of related technologies, the purpose of this invention is to provide a two-dimensional map semantic definition method and apparatus based on RGB and entity contours, aiming to solve the technical problems of existing two-dimensional map semantic definition methods being highly dependent on contour quality and consuming a large amount of computing resources for contour extraction and analysis.

[0010] To solve the above-mentioned technical problems, the present invention is implemented as follows.

[0011] According to one aspect of the present invention, a method for semantic definition of a two-dimensional map based on RGB and entity contours is provided, comprising the following steps:

[0012] S1. Scan the indoor space to obtain a geographic image of the indoor space and various entity objects;

[0013] The robot collects indoor environmental data through its built-in sensors, processes and analyzes the data, and extracts geographic images.

[0014] Feature extraction and classification are performed on the collected geographic images to identify different types of entities.

[0015] The types of entities are walls, columns in a room, doors, windows, and beams, and beams are further divided into high beams and low beams.

[0016] S2. Determine the pixel values ​​of each entity object in the two-dimensional map;

[0017] Specifically, the pixel values ​​of each entity are determined based on its type;

[0018] S3. Draw a two-dimensional semantic map based on the pixel values ​​of each entity object;

[0019] Specifically, the pixel values ​​of each entity object are combined with the geographic image to create a two-dimensional semantic map. The specific implementation steps are as follows:

[0020] S31. Based on geographic images, a two-dimensional map of the indoor environment is created by combining laser SLAM algorithm and deep learning method;

[0021] S32. Input the geographic image and the pixel values ​​of each entity object into the improved Deeplabv3+ semantic segmentation model to obtain the semantic labels of each entity object. The semantic labels are also a kind of two-dimensional map, with the shape of the entity object as the shape and the pixel value as the color.

[0022] S33. Extract the semantic information of each entity pixel from the semantic tags of each entity object, and synchronously map it to the two-dimensional map based on the coordinates of each pixel in the two-dimensional map of the indoor environment.

[0023] S4. Identify the outlines of all entity objects and extract semantic information of the outlines;

[0024] The geographic image in step S1 is used to identify the contours of each entity through contour detection method, and the distance from the contour to the interior of the object is detected. Based on this distance, the semantic information of the layer between the outer edge and the inner edge of the contour is extracted and defined as contour semantic information.

[0025] S5. Combine pixel values ​​with contour semantic information to update the two-dimensional semantic map. The specific implementation steps are as follows:

[0026] S51. Fuse the pixel values ​​of each entity object with the semantic information of the contour layer to obtain pixel contour semantic information;

[0027] S52. Replace the semantic information of the entity pixels with the semantic information of the pixel outline, and update the two-dimensional map synchronously based on the coordinates of each pixel in the two-dimensional map of the indoor environment.

[0028] In step S51, the pixel values ​​of each entity object are fused with the semantic information of the contour layer. The specific implementation steps are as follows:

[0029] S511. Represent the semantic information of the contour layer as a triple SC.<D,V,N> , where D represents the entity object type, V represents the set of coordinate information of the outer edge of the contour, and N represents the set of coordinate information of each pixel in the contour layer;

[0030] S512. Merge the pixel values ​​of each entity object with the semantic information of the contour layer to form a quadruple SCR.<D,V,N,R> Where R is the pixel value;

[0031] S513, quaternion SCR<D,V,N,R> Input the reused DeepLabv3+ model to obtain pixel contour semantic information.

[0032] According to another aspect of the present invention, a two-dimensional map semantic definition device based on RGB and entity contours is provided, comprising: an image and entity acquisition module, a pixel value determination module, a semantic map drawing module, a contour extraction module, and a map update module, each module being installed inside a robot and transmitting data between them via a wireless communication protocol.

[0033] Image and entity acquisition module: configured to scan indoor space and acquire geographic images and various entity objects of the indoor space;

[0034] The robot collects indoor environmental data through its built-in sensors, processes and analyzes the data, and extracts geographic images.

[0035] Feature extraction and classification are performed on the collected geographic images to identify different types of entities.

[0036] The types of entities are walls, columns inside a room, doors, windows, and beams, with beams further divided into high beams and low beams.

[0037] Pixel value determination module: Configured to determine the pixel value of each entity object in the 2D map;

[0038] Specifically, the pixel values ​​of each entity are determined based on its type;

[0039] Semantic map drawing module: configured to draw a two-dimensional semantic map based on the pixel values ​​of each entity object;

[0040] Specifically, the pixel values ​​of each entity object are combined with the geographic image to create a two-dimensional semantic map, which is achieved through the following steps:

[0041] S31. Based on geographic images, a two-dimensional map of the indoor environment is created by combining laser SLAM algorithm and deep learning method;

[0042] S32. Input the geographic image and the pixel values ​​of each entity object into the improved Deeplabv3+ semantic segmentation model to obtain the semantic labels of each entity object. The semantic labels are also a kind of two-dimensional map, with the shape of the entity object as the shape and the pixel value as the color.

[0043] S33. Extract the semantic information of each entity pixel from the semantic tags of each entity object, and synchronously map it to the two-dimensional map based on the coordinates of each pixel in the two-dimensional map of the indoor environment.

[0044] Contour extraction module: Configured to recognize the contours of all entity objects and extract contour semantic information;

[0045] The geographic image acquired by the image and entity acquisition module is used to identify the contours of each entity through contour detection method, and the distance from the contour to the object is detected. Based on this distance, the semantic information of the layer between the outer edge and the inner edge of the contour is extracted and defined as contour semantic information.

[0046] Map update module: Configured to update the 2D map of the indoor environment by combining pixel values ​​and contour semantic information, it achieves this through the following steps:

[0047] S51. Fuse the pixel values ​​of each entity object with the semantic information of the contour layer to obtain pixel contour semantic information;

[0048] S52. Replace the semantic information of the entity pixels with the semantic information of the pixel outline, and update the two-dimensional map synchronously based on the coordinates of each pixel in the two-dimensional map of the indoor environment.

[0049] In step S51, the pixel values ​​of each entity object are fused with the semantic information of the contour layer, including the following steps:

[0050] S511. Represent the semantic information of the contour layer as a triple SC.<D,V,N> , where D represents the entity object type, V represents the set of coordinate information of the outer edge of the contour, and N represents the set of coordinate information of each pixel in the contour layer;

[0051] S512. Merge the pixel values ​​of each entity object with the semantic information of the contour layer to form a quadruple SCR.<D,V,N,R> Where R is the pixel value;

[0052] S513, quaternion SCR<D,V,N,R> Input the reused DeepLabv3+ model to obtain pixel contour semantic information.

[0053] Beneficial effects:

[0054] (1) When defining the semantics of a two-dimensional map, the RGB values ​​of entities are fused with the semantic information of the contours, so that the recognition of the semantics of a two-dimensional map depends less on the quality of the contours. Even if the contours in the image are not clear enough or the contours are broken due to noise, the semantics of the entity contours can be accurately recognized without affecting the accuracy of subsequent feature analysis and pattern matching.

[0055] (2) It does not require the use of complex feature analysis and pattern matching methods, thus avoiding a large consumption of computing resources;

[0056] (3) A Transformer-based few-sample fine-grained image classification model is used to obtain entities of different types, which overcomes the problem that few-sample learning has poor classification effect in fine-grained image classification due to the small number of samples. Attached Figure Description

[0057] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0058] Figure 1 This is a flowchart illustrating a two-dimensional map semantic definition method based on RGB and entity contours according to the present invention.

[0059] Figure 2 This is an example of a two-dimensional semantic map drawn based on the pixel values ​​of each entity object according to the present invention;

[0060] Figure 3 This is a schematic diagram of the modules of a two-dimensional map semantic definition device based on RGB and entity contours according to the present invention; Detailed Implementation

[0061] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, and not all of them. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present application. It should be noted that, unless otherwise specified, the embodiments and features in the embodiments of the present application can be combined with each other.

[0062] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0063] Example 1

[0064] like Figure 1 As shown, this invention proposes a two-dimensional map semantic definition method based on RGB and entity contours, including the following steps:

[0065] S1: Scan the indoor space to obtain a geographic image of the indoor space and various entity objects;

[0066] The robot collects indoor environmental data through its built-in sensors, processes and analyzes the data, and extracts geographic images.

[0067] Furthermore, this geographic image is an image of the space where the robot is located;

[0068] Furthermore, the sensor is a binocular camera or a lidar, which can rotate 360 ​​degrees to collect indoor environmental data from all directions;

[0069] Feature extraction and classification are performed on the collected geographic images to identify different types of entities.

[0070] Furthermore, the collected geographic images are used as input to a Transformer-based few-sample fine-grained image classification model to obtain entities of various types;

[0071] Furthermore, this Transformer-based small-sample fine-grained image classification model combines CNN and Transformer to obtain a sample set and a test set. The sample set and test set contain collected indoor environment images. The sample set has 500 images, and the test set has 100 images, which is much smaller than the sample size of other machine learning models.

[0072] In training a Transformer-based few-sample fine-grained image classification model, a CNN is used to extract image features. After feature extraction, a bilinear pooling layer is used to further extract features. The features are then input into a Transformer encoder, which uses a multi-head self-attention mechanism and positional encoding to capture global and contextual information. The features processed by the Transformer encoder are then input into a Softmax classifier to obtain the type of each entity in the image.

[0073] Furthermore, the types of entities are walls, columns in a room, doors, windows, and beams, with beams further divided into high beams and low beams.

[0074] This Transformer-based few-shot fine-grained image classification model overcomes the problem that few-shot learning results in poor classification performance in fine-grained image classification due to the small number of samples.

[0075] S2. Determine the pixel values ​​of each entity object in the two-dimensional map;

[0076] Specifically, the pixel values ​​of each entity are determined based on its type;

[0077] Furthermore, the background is black in the 2D map with pixel values ​​of RGB(0,0,0), the walls and pillars inside the room are green in the 2D map with pixel values ​​of RGB(0,255,0), the door is yellow with pixel values ​​of RGB(255,255,0), the window is sky blue with pixel values ​​of RGB(0,255,255), the high beam is light gray with pixel values ​​of RGB(127,127,127), and the low beam is dark gray with pixel values ​​of RGB(195,195,195). Structurally, the low beam cuts off the high beam.

[0078] S3. Draw a two-dimensional semantic map based on the pixel values ​​of each entity object, such as... Figure 2 As shown;

[0079] Specifically, the pixel values ​​of each entity object are combined with the geographic image to create a two-dimensional semantic map. The specific implementation steps are as follows:

[0080] S31. Based on geographic images, a two-dimensional map of the indoor environment is created by combining laser SLAM algorithm and deep learning method;

[0081] S32. Input the geographic image and the pixel values ​​of each entity object into the improved Deeplabv3+ semantic segmentation model to obtain the semantic labels of each entity object. The semantic labels are also a kind of two-dimensional map, with the shape of the entity object as the shape and the pixel value as the color.

[0082] S33. Extract the semantic information of each entity pixel from the semantic tags of each entity object, and synchronously map it to the two-dimensional map based on the coordinates of each pixel in the two-dimensional map of the indoor environment.

[0083] Furthermore, the improved DeepLabv3+ adopts an encoder-decoder architecture, which extracts features from the geographic image through the encoder to obtain the features of each entity object, obtains the pixel value of each entity object from step S32, and fuses the pixel value with the feature of the entity object; the fused feature is then restored into a segmentation map of the same size as the original image through the decoder to obtain the semantic label of each entity.

[0084] S4. Identify the outlines of all entity objects and extract semantic information of the outlines;

[0085] The geographic image in step S1 is used to identify the contours of each entity through contour detection method, and the distance from the contour to the interior of the object is detected. Based on this distance, the semantic information of the layer between the outer edge and the inner edge of the contour is extracted and defined as contour semantic information.

[0086] Furthermore, the contour detection method employs a target contour detection method that combines visual attention mechanism with the Canny operator.

[0087] S5. Combine pixel values ​​with contour semantic information to update the two-dimensional semantic map. The specific implementation steps are as follows:

[0088] S51. Fuse the pixel values ​​of each entity object with the semantic information of the contour layer to obtain pixel contour semantic information;

[0089] S52. Replace the semantic information of the entity pixels with the semantic information of the pixel outline, and update the two-dimensional map synchronously based on the coordinates of each pixel in the two-dimensional map of the indoor environment.

[0090] In step S51, the pixel values ​​of each entity object are fused with the semantic information of the contour layer. The specific implementation steps are as follows:

[0091] S511. Represent the semantic information of the contour layer as a triple SC.<D,V,N> , where D represents the entity object type, V represents the set of coordinate information of the outer edge of the contour, and N represents the set of coordinate information of each pixel in the contour layer;

[0092] S512. Merge the pixel values ​​of each entity object with the semantic information of the contour layer to form a quadruple SCR.<D,V,N,R> Where R is the pixel value;

[0093] S513, quaternion SCR<D,V,N,R> Input the reused DeepLabv3+ model to obtain pixel contour semantic information;

[0094] Furthermore, the reused DeepLabv3+ model has different input and output data from the DeepLabv3+ model in step S32. It takes quadruple information as input and semantic information as output, and is trained using historical data.

[0095] Example 2

[0096] like Figure 2 As shown, this invention proposes a two-dimensional map semantic definition device based on RGB and entity contours, including: an image and entity acquisition module, a pixel value determination module, a semantic map drawing module, a contour extraction module, and a map update module. Each module is installed inside the robot and transmits data between them through a wireless communication protocol.

[0097] Image and entity acquisition module: configured to scan indoor space and acquire geographic images and various entity objects of the indoor space;

[0098] The robot collects indoor environmental data through its built-in sensors, processes and analyzes the data, and extracts geographic images.

[0099] Furthermore, this geographic image is an image of the space where the robot is located;

[0100] Furthermore, the sensor is a binocular camera or a lidar, which can rotate 360 ​​degrees to collect indoor environmental data from all directions;

[0101] Feature extraction and classification are performed on the collected geographic images to identify different types of entities.

[0102] Furthermore, the collected geographic images are used as input to a Transformer-based few-sample fine-grained image classification model to obtain entities of various types;

[0103] Furthermore, this Transformer-based small-sample fine-grained image classification model combines CNN and Transformer to obtain a sample set and a test set. The sample set and test set contain collected indoor environment images. The sample set has 500 images, and the test set has 100 images, which is much smaller than the sample size of other machine learning models.

[0104] In training a Transformer-based few-sample fine-grained image classification model, a CNN is used to extract image features. After feature extraction, a bilinear pooling layer is used to further extract features. The features are then input into a Transformer encoder, which uses a multi-head self-attention mechanism and positional encoding to capture global and contextual information. The features processed by the Transformer encoder are then input into a Softmax classifier to obtain the type of each entity in the image.

[0105] Furthermore, the types of entities are walls, columns in a room, doors, windows, and beams, with beams further divided into high beams and low beams.

[0106] This Transformer-based few-shot fine-grained image classification model overcomes the problem that few-shot learning results in poor classification performance in fine-grained image classification due to the small number of samples.

[0107] Pixel value determination module: Configured to determine the pixel value of each entity object in the 2D map;

[0108] Specifically, the pixel values ​​of each entity are determined based on its type;

[0109] Furthermore, the background is black in the 2D map with pixel values ​​of RGB(0,0,0), the walls and pillars inside the room are green in the 2D map with pixel values ​​of RGB(0,255,0), the door is yellow with pixel values ​​of RGB(255,255,0), the window is sky blue with pixel values ​​of RGB(0,255,255), the high beam is light gray with pixel values ​​of RGB(127,127,127), and the low beam is dark gray with pixel values ​​of RGB(195,195,195). Structurally, the low beam cuts off the high beam.

[0110] Semantic map drawing module: configured to draw a two-dimensional semantic map based on the pixel values ​​of each entity object;

[0111] Specifically, the pixel values ​​of each entity object are combined with the geographic image to create a two-dimensional semantic map. The steps to achieve this are as follows:

[0112] S31. Based on geographic images, a two-dimensional map of the indoor environment is created by combining laser SLAM algorithm and deep learning method;

[0113] S32. Input the geographic image and the pixel values ​​of each entity object into the improved Deeplabv3+ semantic segmentation model to obtain the semantic labels of each entity object. The semantic labels are also a kind of two-dimensional map, with the shape of the entity object as the shape and the pixel value as the color.

[0114] S33. Extract the semantic information of each entity pixel from the semantic tags of each entity object, and synchronously map it to the two-dimensional map based on the coordinates of each pixel in the two-dimensional map of the indoor environment.

[0115] Furthermore, the improved DeepLabv3+ adopts an encoder-decoder architecture, which extracts features from the geographic image through the encoder to obtain the features of each entity object, obtains the pixel value of each entity object from step S32, and fuses the pixel value with the feature of the entity object; the fused feature is then restored into a segmentation map of the same size as the original image through the decoder to obtain the semantic label of each entity.

[0116] Contour extraction module: Configured to recognize the contours of all entity objects and extract contour semantic information;

[0117] Specifically, the geographic images acquired by the image and entity acquisition module are used to identify the contours of each entity through contour detection methods, and the distance from the contour to the object is detected. Based on this distance, the semantic information of the layer between the outer edge and the inner edge of the contour is extracted and defined as contour semantic information.

[0118] Furthermore, the contour detection method employs a target contour detection method that combines visual attention mechanism with the Canny operator.

[0119] Map update module: Configured to update the 2D semantic map by combining pixel values ​​and contour semantic information, it implements the following steps:

[0120] S51. Fuse the pixel values ​​of each entity object with the semantic information of the contour layer to obtain the contour semantic information;

[0121] S52. Replace the semantic information of entity pixels with the semantic information of the outline, and update the two-dimensional map synchronously based on the coordinates of each pixel in the two-dimensional map of the indoor environment.

[0122] In step S51, the pixel values ​​of each entity object are fused with the semantic information of the contour layer, including the following steps:

[0123] S511. Represent the semantic information of the contour layer as a triple SC.<D,V,N> , where D represents the entity object type, V represents the set of coordinate information of the outer edge of the contour, and N represents the set of coordinate information of each pixel in the contour layer;

[0124] S512. Merge the pixel values ​​of each entity object with the semantic information of the contour layer to form a quadruple SCR.<D,V,N,R> Where R is the pixel value;

[0125] S513, quaternion SCR<D,V,N,R> Input the reused DeepLabv3+ model to obtain pixel contour semantic information;

[0126] Furthermore, the reused DeepLabv3+ model has different input and output data from the DeepLabv3+ model in step S32. It takes quadruple information as input and semantic information as output, and is trained using historical data.

[0127] Example 3

[0128] This application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor runs the computer program to enable the computer device to perform the automatic polishing method for a mobile robot based on a compensation value machine learning model as described in Embodiment 1.

[0129] This application also provides a computer-readable storage medium storing a computer program. When the computer program is read and run by a processor, it executes the automatic polishing method for a mobile robot based on a compensation value machine learning model described in Embodiment 1.

[0130] Optionally, specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementations, and will not be repeated here.

[0131] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0132] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0133] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0134] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0135] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0136] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0137] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method for semantic definition of two-dimensional maps based on RGB and entity contours, characterized in that, Includes the following steps: S1. Scan the indoor space to obtain a geographic image of the indoor space and various entity objects; The robot collects indoor environmental data through its built-in sensors, processes and analyzes the data, and extracts geographic images. Feature extraction and classification are performed on the collected geographic images to identify different types of entities; S2. Determine the pixel values ​​of each entity object in the two-dimensional map; Specifically, the pixel values ​​of each entity object are determined based on the type of each entity. S3. Draw a two-dimensional semantic map based on the pixel values ​​of each entity object; Specifically, the pixel values ​​of each entity object are combined with the geographic image to create a two-dimensional semantic map; The pixel values ​​of each entity object are combined with the geographic image to create a two-dimensional semantic map. The specific implementation steps are as follows: S31. Based on geographic images, a two-dimensional map of the indoor environment is created by combining laser SLAM algorithm and deep learning method; S32. Input the geographic image and the pixel values ​​of each entity object into the improved Deeplabv3+ semantic segmentation model to obtain the semantic labels of each entity object. The semantic labels are also a kind of two-dimensional map, with the shape of the entity object as the shape and the pixel value as the color. S33. Extract the semantic information of each entity pixel from the semantic tags of each entity object, and synchronously map it to the two-dimensional map based on the coordinates of each pixel in the two-dimensional map of the indoor environment. S4. Identify the outlines of all entity objects and extract semantic information of the outlines; The geographic image in step S1 is used to identify the contours of each entity through contour detection method, and the distance from the contour to the interior of the object is detected. Based on this distance, the semantic information of the layer between the outer edge and the inner edge of the contour is extracted and defined as contour semantic information. The contour detection method described above is a target contour detection method that combines visual attention mechanism with Canny operator; S5. Combine pixel values ​​with contour semantic information to update the two-dimensional semantic map; The two-dimensional semantic map is updated by combining pixel values ​​and contour semantic information, specifically including the following steps: S51. Fuse the pixel values ​​of each entity object with the semantic information of the contour layer to obtain pixel contour semantic information; S52. Replace the semantic information of each pixel in the entity with the semantic information of the pixel outline, and update the two-dimensional semantic map synchronously according to the coordinates of each pixel in the two-dimensional semantic map.

2. The method as described in claim 1, characterized in that, In step S51, the pixel values ​​of each entity object are fused with the semantic information of the contour layer to obtain pixel contour semantic information, which specifically includes the following steps: S511. Represent the semantic information of the contour layer as a triple SC.<D,V,N> , where D represents the entity object type, V represents the set of coordinate information of the outer edge of the contour, and N represents the set of coordinate information of each pixel in the contour layer; S512. Merge the pixel values ​​of each entity object with the semantic information of the contour layer to form a quadruple SCR.<D,V,N,R> Where R is the pixel value; S513, quaternion SCR<D,V,N,R> Input the reused DeepLabv3+ model to obtain pixel contour semantic information.

3. The method as described in claim 1, characterized in that, The entities are classified as walls, room pillars, doors, windows, and beams, with beams further divided into high beams and low beams. The specific pixel values ​​of these entities are as follows: the background is black in the 2D map with a pixel value of RGB(0,0,0); walls and room pillars are green in the 2D map with a pixel value of RGB(0,255,0); doors are yellow with a pixel value of RGB(255,255,0); windows are sky blue with a pixel value of RGB(0,255,255); high beams are light gray with a pixel value of RGB(127,127,127); and low beams are dark gray with a pixel value of RGB(195,195,195). Structurally, low beams interrupt high beams.

4. The method as described in claim 1, characterized in that, The improved DeepLabv3+ semantic segmentation model in step S32 adopts an encoder-decoder architecture. The encoder extracts features from the geographic image to obtain the features of each entity object. The pixel values ​​of each entity object are obtained from step S32, and the pixel values ​​are fused with the features of the entity object. The fused features are then restored into a segmentation map of the same size as the original image by the decoder to obtain the semantic labels of each entity.

5. The method as described in claim 4, characterized in that, Step S1 involves feature extraction and classification of the acquired geographic images to identify different types of entities. Specifically, this includes the following steps: The collected geographic images are used as input to a Transformer-based few-sample fine-grained image classification model to obtain entities of various types. This Transformer-based few-shot fine-grained image classification model combines CNN and Transformer. During the training of the Transformer-based few-shot fine-grained image classification model, CNN is used to extract image features. After feature extraction, bilinear pooling layers are further used to extract features. The features are then input into the Transformer encoder, which uses multi-head self-attention mechanism and positional encoding to capture global and contextual information. The features processed by the Transformer encoder are then input into the Softmax classifier to obtain the type of each entity in the image.

6. A two-dimensional map semantic definition device based on RGB and entity contours, comprising: The image and entity acquisition module, pixel value determination module, semantic map drawing module, contour extraction module, and map update module are all installed inside the robot and transmit data between each other via a wireless communication protocol. Image and entity acquisition module: configured to scan indoor space and acquire geographic images and various entity objects of the indoor space; The robot collects indoor environmental data through its built-in sensors, processes and analyzes the data, and extracts geographic images. Feature extraction and classification are performed on the collected geographic images to identify different types of entities; Pixel value determination module: Configured to determine the pixel value of each entity object in the 2D map; Specifically, the pixel values ​​of each entity are determined based on its type; Semantic map drawing module: configured to draw a two-dimensional semantic map based on the pixel values ​​of each entity object; Specifically, the pixel values ​​of each entity object are combined with the geographic image to create a two-dimensional semantic map. The specific implementation steps are as follows: S31. Based on the geographic image, a two-dimensional map of the indoor environment is created using a combination of laser SLAM algorithm and deep learning method; S32. The geographic image and the pixel values ​​of each entity object are input into the improved Deeplabv3+ semantic segmentation model to obtain the semantic labels of each entity object. The semantic labels are also a type of two-dimensional map, with the shape of the entity object as the shape and the pixel value as the color; S33. The semantic information of each entity pixel is extracted from the semantic labels of each entity object, and synchronously mapped to the two-dimensional map according to the coordinates of each pixel in the two-dimensional map of the indoor environment. Contour extraction module: Configured to recognize the contours of all entity objects and extract contour semantic information; The geographic image acquired by the image and entity acquisition module is used to identify the contours of each entity through contour detection method, and the distance from the contour to the object is detected. Based on this distance, the semantic information of the layer between the outer edge and the inner edge of the contour is extracted and defined as contour semantic information. Map update module: configured to update the two-dimensional semantic map by combining pixel values ​​and contour semantic information; the specific steps of updating the two-dimensional semantic map by combining pixel values ​​and contour semantic information are as follows: S51, fuse the pixel values ​​of each entity object with the contour layer semantic information to obtain pixel contour semantic information; S52, replace the semantic information of each pixel in the entity with the pixel contour semantic information, and update the two-dimensional semantic map synchronously according to the coordinates of each pixel in the two-dimensional semantic map.

7. An electronic device, comprising a memory and a processor, the memory for storing a computer program, the processor for running the computer program to cause the computer device to perform the two-dimensional map semantic definition method based on RGB and entity contours as described in any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, which, when read and executed by a processor, performs the two-dimensional map semantic definition method based on RGB and entity contours as described in any one of claims 1 to 5.

Images