Deep learning-based crowdsourcing acquisition high-precision map desensitization method and system

Abstract

The invention discloses a crowdsourcing acquisition map data desensitization method and system based on deep learning. According to the method, a'high-precision map desensitization rule and target sensitivity level table 'is constructed for grading and classifying a to-be-desensitized target, and a desensitization method is specified and comprises a shape type, a privacy type and a character type. For shape type and privacy type sensitive targets in a high-precision map, the positions of the sensitive targets are detected by using a sensitive target detection algorithm based on Faster RCNN, and different desensitization methods are used according to the sensitive levels of the sensitive targets. For character sensitive targets in a high-precision map, all character areas in the map are detected by using a natural scene text detection model I3CL, the text areas are recognized by using a text recognition model SVTR to obtain text contents of the character areas, sensitive characters in the character areas are matched through a sensitive word matching module, and the sensitive characters in the text areas are matched through a sensitive word matching module; and a sensitive area removing model based on LaMa is used for the sensitive character area, and sensitive characters are removed from the graph.

Description

Technical Field

[0001] The present invention belongs to the field of information security, and specifically relates to a method and system for desensitizing high-precision map data collected by crowdsourcing based on deep learning, which is mainly used for desensitizing high-precision map images. Background Art

[0002] High Definition Map (HD Map) is an important component of the autonomous driving system and plays an important role in autonomous driving technology. At present, HD Map is in a critical period of technology research and development. However, in the process of HD Map collection and production, the desensitization of HD Map mainly relies on manual desensitization, which has a huge repetitive cost and affects the production efficiency of HD Map. Compared with manual desensitization, the use of deep learning to automatically desensitize HD Map can not only improve the production efficiency, but also greatly reduce the production cost of HD Map.

[0003] There is a large amount of sensitive information in high-precision maps, such as the height limit, width limit, clearance, load capacity and slope of important bridges; and private information such as faces and license plates that involve public privacy. If these sensitive information are used directly without processing, it is easy to cause sensitive information leakage.

[0004] HD map images are complex and contain many types of sensitive targets. Existing HD map desensitization work relies on manual desensitization. Related research focuses on desensitizing privacy information such as faces and license plates. However, sensitive units and facilities and sensitive attribute information in HD maps contain more sensitive information and also need to be desensitized. These sensitive targets have different characteristics. Faces, license plates, sensitive units and facilities have obvious appearance characteristics. Sensitive text in sensitive attribute information describes sensitive attribute information. Therefore, it is difficult to detect and identify sensitive targets in HD maps using a single deep learning algorithm. It is necessary to combine the characteristics of sensitive targets and use multiple algorithms for collaborative processing. How to use deep learning technology to efficiently and accurately detect and identify sensitive information in HD maps, desensitize this sensitive information, and realize automatic desensitization of HD maps has become a problem to be solved. Summary of the invention

[0005] The present invention aims to propose a crowdsourcing high-precision map desensitization method based on deep learning. By constructing a "High-precision Map Desensitization Rules and Target Sensitivity Level Table" to classify the desensitized targets, the sensitive information in the high-precision map is detected and identified through the collaborative processing of multiple deep learning algorithms. Different processing methods are adopted for sensitive targets of different levels to achieve automatic desensitization of high-precision maps.

[0006] The technical solution adopted by the present invention is: constructing the "High-precision Map Desensitization Rules and Target Sensitivity Level Table" to classify the desensitized targets and stipulate the desensitization method, specifically dividing sensitive targets into high sensitivity level and low sensitivity level, and dividing sensitive targets into three categories, namely shape class, privacy class and text class, among which the privacy class belongs to the low sensitivity level, and the shape class and text class belong to the high sensitivity level. For the shape class and privacy class sensitive targets in the high-precision map, the location of the sensitive target is detected using the sensitive target detection algorithm based on FasterRCNN, and different desensitization methods are used according to the sensitivity level of the sensitive target. For the text-type sensitive targets in the high-precision map, the natural scene text detection model I3CL is used to detect all the text areas in the map, and the text recognition model SVTR is used to identify the text area to obtain the text content of the text area. The sensitive text in the text area is matched by the sensitive word matching module, and the sensitive area removal model based on LaMa is used for the sensitive text area, and the sensitive text is erased from the map.

[0007] The present invention provides a method for desensitizing high-precision maps collected by crowdsourcing based on deep learning, and the specific implementation steps are as follows:

[0008] (1) Construct a high-precision map desensitization rule and target sensitivity level table, and set the category, sensitivity level, and desensitization rule of sensitive targets;

[0009] (2) Use the natural scene text detection model to locate the text area in the HD map. The detection model will output the coordinates of the minimum enclosing curve of the text area in the HD map;

[0010] (3) Use the text recognition network to identify the text region and obtain the text content in each text instance;

[0011] (4) Collect sensitive words that reflect sensitive attribute information to build a sensitive word library, store it in a tree structure, use AC automaton to match sensitive words, detect whether the identified text area contains sensitive information, and record the coordinates of the text area containing sensitive words;

[0012] (5) Use the image completion model to perform desensitization processing on the detected sensitive text areas;

[0013] (6) Use the FasterRCNN-based sensitive target detection algorithm to determine the category and location of sensitive targets in high-precision maps;

[0014] (7) According to the category of the detected sensitive target, match the sensitivity level corresponding to the target in the high-precision map desensitization rules and target sensitivity level table;

[0015] (8) Use different desensitization methods according to different sensitivity levels.

[0016] Furthermore, in step (1), sensitive targets are divided into privacy, shape and text categories; the sensitivity levels are divided into high sensitivity and low sensitivity, privacy targets belong to the low sensitivity level, and shape and text targets belong to the high sensitivity level; the desensitization rule for the low sensitivity level is to directly encrypt the sensitive area, and the desensitization rule for the high sensitivity level is to erase the significant shapes and logos and keep them consistent with the surrounding environment.

[0017] Furthermore, in step (2), the natural scene text detection model I3CL is used to locate the text area position in the high-precision map, specifically including:

[0018] The high-precision map image is input into the convolutional neural network combined with the feature pyramid network FPN to extract multi-scale image features; then the feature maps of each scale are refined through the intra-text collaborative learning module to enhance the information exchange between the gap areas between text instances and the text; then the text candidate box is selected through the region candidate network RPN for text bounding box regression and text instance segmentation; the bounding box regression branch performs the text box boundary regression task and predicts the position of the text box; the inter-text collaborative learning module is introduced in the text instance segmentation branch to learn the interdependence between text instances and generate fine text contours.

[0019] Furthermore, the processing process of the in-text collaborative learning module is as follows: first, the input is divided into three branches, each branch includes three layers, each layer contains three parallel convolution layers, namely 1×N horizontal convolution, N×1 vertical convolution and an ordinary convolution, the features extracted in each layer are fused and then input into the next layer, and finally the features of the three branches are fused and then passed through an ordinary convolution layer, and then fused with the original input features and passed through a ReLU activation function;

[0020] The text-to-text collaborative learning module adopts the TransFormer structure. The features are first input into a convolutional layer to reduce the channel dimension. Then, the feature dimension is reduced through adaptive maximum pooling. Next, each feature is flattened into a vector. The obtained feature sequence is input into a TransFormer encoder, which has three regular encoder layers and four self-attention layers.

[0021] Furthermore, in step (3), the text region is identified using a text recognition network SVTR, specifically including:

[0022] Step 3.1, input a text image of size H×W×3, where H and W are the height and width of the image, and 3 represents the three color channels of RGB. The text image is decomposed into blocks of size H / 4×W / 4 with a depth of D0 through the patch embedding module;

[0023] Step 3.2, three stages of feature extraction are performed at different scales, which are finally simplified to a feature representation of 1×W / 4×D3. Each stage consists of a global mixing block, a local mixing block, and a merge or combination operation. D3 is the depth value.

[0024] In step 3.3, the final feature representation is linearly predicted to generate a character sequence, where components of the same character are transcribed as repeated characters and non-text components are transcribed as blank symbols.

[0025] Furthermore, the specific implementation of step (4) includes:

[0026] Step 4.1: Build a Trie tree. Build all sensitive words into a Trie tree. Each node represents a character. The path from the root node to the leaf node represents a sensitive word. At the same time, each node stores a flag bit to indicate whether the string ending with the node is a sensitive word.

[0027] Step 4.2, build AC automaton, convert the Trie tree into AC automaton, add a failure pointer to each node in the Trie tree, pointing to another node on the Trie tree, indicating which node should be jumped to to continue matching when the match fails;

[0028] Step 4.3, match the text. For each recognized text, start traversing from the root node of the AC automaton; if the current character matches successfully, traverse down to the next node; if the match fails, jump to another node according to the failure pointer to continue matching; when the flag bit of a certain node is true, it means that a sensitive word has been found, and the position of this text area is recorded.

[0029] Furthermore, the detected sensitive text areas are desensitized using the image completion model LaMa, which specifically includes:

[0030] Step 5.1, generate a mask using the detected sensitive area, and input the original image and the mask into the image completion model;

[0031] Step 5.2, the completion model first performs downsampling for feature extraction;

[0032] In step 5.3, the input features are divided into two branches. The local branch uses the traditional convolution layer to extract local information, and the global branch uses fast Fourier convolution to extract features in both the spatial domain and the frequency domain to achieve feature extraction of the global receptive field. Finally, the local information and the global information are cross-fused.

[0033] In step 5.4, the fused features are upsampled and the desensitized image is output.

[0034] Furthermore, in step (6), the specific implementation of using the FasterRCNN-based sensitive target detection algorithm to locate the category and position of sensitive targets in the high-precision map includes: inputting the image to be desensitized into the convolutional neural network backbone combined with the FPN network structure to extract a multi-scale feature pyramid for subsequent sensitive target detection and classification; the regional candidate network RPN combines the multi-scale feature pyramid prediction to generate a large number of rectangular sensitive target candidate boxes; performing target box border regression and classification to determine the category and position of the sensitive target.

[0035] Furthermore, in step (8), different desensitization methods are used according to different sensitivity levels, specifically including: for privacy-related targets with low sensitivity levels, scrambling encryption is used for the privacy area according to the coordinates corresponding to the privacy target; for shape-related targets with high sensitivity levels, image completion model LaMa is used to perform desensitization processing on the sensitive area according to the coordinates corresponding to the shape-related targets;

[0036] The scrambling encryption for low-sensitivity privacy targets specifically includes: direct encryption of the privacy area according to the coordinates corresponding to the privacy target, which disrupts the original image structure by changing the position and order of pixels in the image and hides the image content in the privacy area;

[0037] The LaMa desensitization processing of the image completion model for shape-type targets with high sensitivity levels specifically includes: generating a mask image based on the coordinates corresponding to the shape-type targets and the original image; inputting the mask image and the original image into the LaMa algorithm, and regenerating sensitive units and facilities with the help of image features in non-sensitive areas, and the original sensitive areas are replaced with an environment similar to the surrounding area.

[0038] The present invention also provides a crowdsourcing collection high-precision map desensitization system based on deep learning, including:

[0039] one or more processors;

[0040] A storage device is used to store one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in the above solution.

[0041] Compared with the prior art, the advantages and beneficial effects of the present invention are as follows:

[0042] The present invention provides a method for desensitizing crowdsourced map data based on deep learning, which is mainly used for desensitizing real-life images. In view of the fact that there are many types of sensitive targets and complex scenes in crowdsourced data, there is currently no corresponding specification for desensitizing crowdsourced data, and the desensitization detection has a low degree of automation and low detection accuracy. A "High-precision Map Desensitization Rules and Target Sensitivity Level Table" is constructed to specify the categories and sensitivity levels of targets to be desensitized, and to divide the sensitive information collected by crowdsourcing in image data into privacy, shape, and text categories. The privacy category contains privacy information involving public privacy, the shape category contains sensitive units and facilities with obvious appearance characteristics, and the text category contains sensitive attribute information. A method for crowdsourcing data desensitization based on deep learning is proposed. A text detection and recognition model based on I3CL and SVTR is designed to address the problems of complex background and variable scale of text-based sensitive targets. The I3CL text detection algorithm is used to detect text in real-life images, and the SVTR text recognition algorithm is used to recognize text in real-life images. The AC automaton is used to build an index tree for the sensitive word library, and the identified sensitive words are matched with text, and the sensitive text area is located. The Lama algorithm is used to erase the sensitive text area to keep it consistent with the surrounding environment, thereby realizing intelligent desensitization. A sensitive target detection model based on FasterRCNN and a sensitive area removal model based on LaMa are designed for privacy-related and shape-related sensitive targets. For privacy-related targets, an encryption algorithm is used to encrypt the privacy area, and for shape-related targets, LaMa is used to erase the sensitive area, thereby improving the accuracy and efficiency of crowdsourcing data desensitization. Therefore, the present invention can realize automatic detection and non-sensitive desensitization of sensitive targets. In addition, the present invention can adopt different desensitization methods for different sensitive targets to achieve a balance between performance and security. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 1 is a general schematic diagram of high-precision map desensitization according to an embodiment of the present invention;

[0044] Figure 2 is a diagram of the I3CL structure in an embodiment of the present invention;

[0045] Figure 3 It is a structural diagram of the collaborative learning module in the text in the embodiment of the present invention;

[0046] Figure 4 4 is a structural diagram of SVTR in an embodiment of the present invention.

[0047] Figure 5 is a structural diagram of an AC automaton in an embodiment of the present invention;

[0048] Figure 6 is a LaMa structure diagram in an embodiment of the present invention.

[0049] Figure 7 1 is the desensitization effect of a privacy-sensitive target in an embodiment of the present invention, wherein (a) and (b) are desensitization effects of different privacy-sensitive targets respectively.

[0050] Figure 8 This is the desensitization effect of shape-sensitive targets in an embodiment of the present invention, where (a)-(g) are desensitization effects of different shape-sensitive targets respectively.

[0051] Fig. 9 This is the desensitization effect of text-based sensitive targets in the embodiment of the present invention. DETAILED DESCRIPTION

[0052] The specific implementation process of the present invention is described in detail below with reference to the accompanying drawings.

[0053] like Figure 1 As shown, an embodiment of the present invention provides a method for desensitizing a high-precision map collected by crowdsourcing based on deep learning, comprising the following steps:

[0054] Step 1: Construct the "High-precision Map Desensitization Rules and Target Sensitivity Level Table", which specifies the categories, sensitivity levels, and desensitization rules of sensitive targets. Sensitive targets are divided into privacy, shape, and text categories. Privacy targets include faces and license plates involving public privacy; shape targets include units and facilities related to public safety, and civil facilities that have a significant impact on production and life; text targets include relevant feature identification areas involved in the side or distant views of various units and facilities, and the identification areas have sensitive text logos. The sensitivity level is divided into high sensitivity and low sensitivity. Privacy targets belong to the low sensitivity level, and shape and text targets belong to the high sensitivity level; the desensitization rule for the low sensitivity level is to use direct encryption for sensitive areas, and the desensitization rule for the high sensitivity level is to erase significant shapes and logos and keep them consistent with the surrounding environment.

[0055] Step 2: For the text area in the crowdsourced high-precision map, use the natural scene text detection model I3CL to locate the text area in the image. The overall framework of the I3CL model is as follows: Figure 2 As shown in the figure. First, the image is input into the backbone network with FPN architecture to generate a multi-scale feature pyramid. Secondly, the inter-text collaborative learning module is used to refine the features at each scale of the feature pyramid. Next, the region proposal network is used to generate text proposals for subsequent processes. Then, bounding box regression and mask prediction are performed in two parallel branches. The frame branch further refines and classifies the text proposals. In the mask branch, the intra-text collaborative learning module is used for collaborative learning between all positive text instances. The detection model will output the coordinates of the minimum enclosing curve of the text area in the high-precision map.

[0056] Step 2.1: The high-precision map image is input into the convolutional neural network combined with the feature pyramid network (FPN) to extract multi-scale image features.

[0057] Step 2.2, the structure of the in-text collaborative learning module is as follows Figure 3 As shown in the figure, the intra-text collaborative learning module refines the feature maps of each feature map. The processing process is as follows: first, the input is divided into three branches, each branch includes three layers, each layer contains three parallel convolution layers, namely 1×N horizontal convolution, N×1 vertical convolution and an ordinary convolution. The features extracted in each layer are fused and then input into the next layer. Finally, the features of the last three branches are fused and then passed through an ordinary convolution layer, and then fused with the original input features and passed through a ReLU activation function. This method contains a set of paths with multiple receptive fields, which enhances the information exchange between the gap areas between text instances and the text.

[0058] In step 2.3, the region proposal network (RPN) selects text candidate boxes for text bounding box regression and text instance segmentation.

[0059] In step 2.4, the bounding box regression branch performs the text box bounding regression task to predict the location of the text box.

[0060] In step 2.5, the inter-text collaborative learning module is introduced in the text instance segmentation branch. The inter-text collaborative learning module adopts the TransFormer structure. The features are first input into a convolutional layer to reduce the channel dimension. Then, the feature dimension is reduced by adaptive maximum pooling. Next, each feature is flattened into a vector. The resulting feature sequence is input into a TransFormer encoder, which has three regular encoder layers and four self-attention layers. Through the self-attention layer, the long-range dependencies between different text instances in the image can be captured by adaptively focusing on specific text instances with similar background contexts or font appearances. Through this collaborative learning method, the representation ability of the learned features can be improved. Learn the interdependencies between text instances and generate fine text contours.

[0061] Step 3: The text recognition network SVTR identifies the text area. The structure of SVTR is as follows: Figure 4 As shown in the figure, the network converts the input image text into character components through progressive overlapping patch embedding, and each character component is associated with a part of the text character in the image. Then it passes through three hybrid blocks for merging or combining operations and extracting features at different scales. Finally, a linear predictor is used to obtain the text content in each text instance.

[0062] In step 3.1, the input text image is downsampled through the convolution module to obtain a larger receptive field and improve the ability to express local information.

[0063] In step 3.2, three stages of feature extraction are performed at different scales, each of which consists of a global mixing block, a local mixing block, and a merging or combining operation. The global mixing block can establish long-term dependencies between components from different characters and weaken the influence of non-text components while increasing the importance of text components. The local mixing block uses a window to constrain character components.

[0064] Step 3.3, the input of stage 1 is an image of size H×W×3, where H and W are the height and width of the image, and 3 represents the three RGB color channels. In the patch embedding module, the image is decomposed into small character components, which may contain only part of a character. At the beginning of each stage, the image is converted into blocks of size H / 4×W / 4 with a depth of D0. In the fusion module, a series of mixing blocks capture patterns within and between characters. This includes local and global mixing, where the local mixing block is responsible for extracting local patterns similar to strokes, and the global mixing block is responsible for capturing long-distance dependencies between different characters. The final fusion operation further processes the features to help integrate feature information at different levels.

[0065] Step 3.4, the input of stage 2 is the feature map of H / 4×W / 4×D0 output by the previous stage, which also includes fusion modules and fusion operations. In this stage, the feature map is further refined to H / 8×W / 4×D1, where D1 is usually deeper than D0 to capture more complex features.

[0066] Step 3.5, the input of stage 3 is H / 8×W / 4×D1, and the feature map of the final fusion module and fusion operation is further compressed to H / 16×W / 4×D2, and then simplified to a feature representation of 1×W / 4×D3. The final feature representation is linearly predicted to generate a character sequence, where components of the same character are transcribed as repeated characters and non-text components are transcribed as blank symbols.

[0067] Step 4: collect sensitive words that reflect sensitive attribute information to build a sensitive word library, store it in a tree structure, and use AC automaton to match sensitive words. The structure of AC automaton is as follows: Figure 5 As shown in the figure, AC automaton is a multi-pattern string matching algorithm based on Trie tree. AC automaton mainly includes two processes: construction and search. In the construction phase, it first creates a Trie tree to store all pattern strings, and then converts the Trie tree into a finite state automaton by building a failure pointer (also called a mismatch pointer). In the search phase, it searches for pattern strings in the text by reading the text and transferring states character by character. This method can quickly detect whether the identified text area contains sensitive information and record the coordinates of the text area containing sensitive words.

[0068] Step 4.1, construct a Trie tree. All sensitive words are constructed into a Trie tree. Each node represents a character, and the path from the root node to the leaf node represents a sensitive word. At the same time, each node can store a flag bit to indicate whether the string ending with the node is a sensitive word.

[0069] Step 4.2, build an AC automaton, convert the Trie tree into an AC automaton, add a failure pointer to each node in the Trie tree, pointing to another node on the Trie tree, indicating which node should be jumped to continue matching when the match fails.

[0070] Step 4.3, match the text. For each recognized text, start traversing from the root node of the AC automaton. If the current character matches successfully, traverse down to the next node; if the match fails, jump to another node according to the failure pointer to continue matching. When the flag bit of a node is true, it means that a sensitive word has been found, and the position of this text area is recorded.

[0071] Step 5: Use the image completion model LaMa for the detected sensitive text area. The LaMa network input includes an original RGB color image (3 channels) and a binary mask image (1 channel). The mask image marks the area to be repaired, usually in white, and the rest in black. The mask is multiplied by the original image, a step called "masking" or "covering", to obtain an image with missing areas. The mask and the masked image are stacked together to form a 4-channel input tensor, and the network can use both image information and mask position information. The LaMa network introduces a fast Fourier convolution module, which has a global receptive field and can capture long-range dependencies in the image, which helps to repair large missing areas. The input first passes through a downsampling layer to reduce the spatial dimension for subsequent processing. Then, the data stream is split into two branches, a local branch and a global branch. The local branch uses a traditional convolution layer to extract local features. The global branch uses a fast Fourier convolution module to extract global context information. In this branch, the image data is converted to the frequency domain, a convolution operation is performed, and then inversely transformed back to the spatial domain. Finally, the information of the local and global branches is fused and upsampled to gradually restore the resolution of the original image, and the restored image is output. The detected sensitive area is used to generate a mask, and the original image and the mask are input into the image completion model. The sensitive area is regenerated using the non-sensitive area in the image as known information to replace the original sensitive area.

[0072] Step 5.1, use the detected sensitive area to generate a mask, and input the original image and mask into the image completion model.

[0073] In step 5.2, the network first performs downsampling for feature extraction.

[0074] In step 5.3, the input features are divided into two branches. The local branch is responsible for extracting local information, and the global branch uses fast Fourier convolution to extract features in both the spatial domain (image domain) and the frequency domain to achieve feature extraction of the global receptive field. Finally, the local information and the global information are cross-fused.

[0075] In step 5.4, the fused features are upsampled and the desensitized image is output.

[0076] Step 6: For sensitive units and facilities in the crowdsourced high-precision map, use the target detection algorithm FasterRCNN to locate the position of sensitive targets in the image, and the detection model will output the coordinates of sensitive units and facilities.

[0077] Step 6.1: Input the image to be desensitized into the convolutional neural network backbone combined with the FPN network structure to extract the multi-scale feature pyramid for subsequent sensitive target detection and classification. The multi-scale feature pyramid is a combination of convolutional feature maps of different scales, which can fuse the semantic features of different stages of the convolutional neural network and generate convolutional feature maps of different scales to adapt to sensitive targets of different sizes on the image;

[0078] Step 6.2, in the first stage, the region proposal network RPN combines multi-scale feature pyramid prediction to generate a large number of rectangular text candidate boxes.

[0079] Step 6.3, the second stage, performs target box bounding box regression and classification to determine the category and location of sensitive targets.

[0080] Step 7: According to the category of the detected sensitive target, match the sensitivity level corresponding to the target in the "High-precision Map Desensitization Rules and Target Sensitivity Level Table" and use different desensitization methods according to different sensitivity levels.

[0081] Step 8: For privacy targets, which are of low sensitivity, scrambling encryption is used on the privacy area according to the coordinates corresponding to the privacy target. Its function is to disrupt the original image structure by changing the position and order of pixels in the image and hide the image content in the privacy area.

[0082] Step 9: For shape-type targets, which belong to the high sensitivity level, the LaMa algorithm is used to complete the sensitive area according to the coordinates corresponding to the shape-type targets.

[0083] Step 9.1, generate a mask image based on the coordinates corresponding to the shape class target and the original image.

[0084] In step 9.2, the mask image and the original image are input into the LaMa algorithm. With the help of the image features of the non-sensitive area, the sensitive units and facilities are regenerated, and the original sensitive area is replaced with an environment similar to the surrounding area.

[0085] Figure 7 , Figure 8 and Fig. 9 The desensitization effects of privacy-related, shape-related, and text-related sensitive targets are shown respectively. The original image is shown on the left side of the system, and the desensitized image is shown on the right side of the system. The privacy-related targets show the desensitization effects of faces and license plates, the shape-related targets show the desensitization effects of bridge height limit signs, weight limit signs, chimneys, condensation towers, high-voltage power towers, meteorological towers, etc.; the text-related sensitive targets show the desensitization effects of important facilities containing sensitive texts, such as archives.

[0086] The embodiment of the present invention also provides a crowdsourcing collection high-precision map desensitization system based on deep learning, including:

[0087] one or more processors;

[0088] A storage device is used to store one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in the above solution.

[0089] The above contents are further detailed descriptions of the present invention in combination with the best implementation scheme, and it cannot be determined that the specific implementation of the present invention is limited to these descriptions. It should be understood by those skilled in the art that various modifications can be made to the details without departing from the scope of the appended claims, and all should be deemed to belong to the protection scope of the present invention.

Claims

1. A crowdsourcing high-precision map desensitization method based on deep learning, characterized in that: The following steps are involved: Step (1), constructing a high-precision map desensitization rule and target sensitivity level table, setting the category, sensitivity level, and desensitization rule of the sensitive target; Step (2), using a natural scene text detection model to locate the text area in the high-precision map. The detection model will output the coordinates of the minimum enclosing curve of the text area in the high-precision map; Step (3), using a text recognition network to identify the text region and obtain the text content in each text instance; Step (4), collect sensitive words reflecting sensitive attribute information to build a sensitive word library, store it in a tree structure, use AC automaton to match sensitive words, detect whether the identified text area contains sensitive information, and record the coordinates of the text area containing sensitive words; Step (5), using the image completion model to perform desensitization processing on the detected sensitive text area; Step (6), using the sensitive target detection algorithm based on FasterRCNN to determine the category and location of sensitive targets in the high-precision map; Step (7), according to the category of the detected sensitive target, match the sensitivity level corresponding to the target in the high-precision map desensitization rules and target sensitivity level table; Step (8), different desensitization methods are used according to different sensitivity levels.

2. The method for desensitizing high-precision maps collected by crowdsourcing based on deep learning according to claim 1 is characterized in that: In step (1), sensitive targets are divided into privacy, shape and text categories; the sensitivity levels are divided into high sensitivity and low sensitivity, privacy targets belong to the low sensitivity level, and shape and text targets belong to the high sensitivity level; the desensitization rule for the low sensitivity level is to directly encrypt the sensitive area, and the desensitization rule for the high sensitivity level is to erase the significant shapes and logos and keep them consistent with the surrounding environment.

3. The method for desensitizing high-precision maps collected by crowdsourcing based on deep learning according to claim 1 is characterized in that: In step (2), the natural scene text detection model I3CL is used to locate the text area in the high-precision map, which specifically includes: The high-precision map image is input into the convolutional neural network combined with the feature pyramid network FPN to extract multi-scale image features; then the feature maps of each scale are refined through the intra-text collaborative learning module to enhance the information exchange between the gap areas between text instances and the text; then the text candidate box is selected through the region candidate network RPN for text bounding box regression and text instance segmentation; the bounding box regression branch performs the text box boundary regression task and predicts the position of the text box; the inter-text collaborative learning module is introduced in the text instance segmentation branch to learn the interdependence between text instances and generate fine text contours.

4. The method for desensitizing high-precision maps collected by crowdsourcing based on deep learning according to claim 1 is characterized in that: The processing process of the in-text collaborative learning module is as follows: first, the input is divided into three branches, each branch includes three layers, each layer contains three parallel convolution layers, namely 1×N horizontal convolution, N×1 vertical convolution and an ordinary convolution, the features extracted in each layer are fused and then input into the next layer, and finally the features of the three branches are fused and then passed through an ordinary convolution layer, and then fused with the original input features and passed through a ReLU activation function; The text-to-text collaborative learning module adopts the TransFormer structure. The features are first input into a convolutional layer to reduce the channel dimension. Then, the feature dimension is reduced through adaptive maximum pooling. Next, each feature is flattened into a vector. The obtained feature sequence is input into a TransFormer encoder, which has three regular encoder layers and four self-attention layers.

5. The method for desensitizing high-precision maps collected by crowdsourcing based on deep learning according to claim 1 is characterized in that: In step (3), the text region is identified using the text recognition network SVTR, which specifically includes: Step 3.1, input a text image of size H×W×3, where H and W are the height and width of the image, and 3 represents the three color channels of RGB. The text image is decomposed into blocks of size H / 4×W / 4 with a depth of D0 through the patch embedding module; Step 3.2, three stages of feature extraction are performed at different scales, which are finally simplified to a feature representation of 1×W / 4×D3. Each stage consists of a global mixing block, a local mixing block, and a merging or combining operation. i is the depth value; In step 3.3, the final feature representation is linearly predicted to generate a character sequence, where components of the same character are transcribed as repeated characters and non-text components are transcribed as blank symbols.

6. The method for desensitizing high-precision maps collected by crowdsourcing based on deep learning according to claim 1 is characterized in that: The specific implementation of step (4) includes: Step 4.1: Build a Trie tree. Build all sensitive words into a Trie tree. Each node represents a character. The path from the root node to the leaf node represents a sensitive word. At the same time, each node stores a flag bit to indicate whether the string ending with the node is a sensitive word. Step 4.2, build AC automaton, convert the Trie tree into AC automaton, add a failure pointer to each node in the Trie tree, pointing to another node on the Trie tree, indicating which node should be jumped to to continue matching when the match fails; Step 4.3, match the text. For each recognized text, start traversing from the root node of the AC automaton; if the current character matches successfully, traverse down to the next node; if the match fails, jump to another node according to the failure pointer to continue matching; when the flag bit of a certain node is true, it means that a sensitive word has been found, and the position of this text area is recorded.

7. The method for desensitizing high-precision maps collected by crowdsourcing based on deep learning according to claim 1 is characterized in that: The detected sensitive text areas are desensitized using the image completion model LaMa, including: Step 5.1, generate a mask using the detected sensitive area, and input the original image and the mask into the image completion model; Step 5.2, the completion model first performs downsampling for feature extraction; In step 5.3, the input features are divided into two branches. The local branch uses the traditional convolution layer to extract local information, and the global branch uses fast Fourier convolution to extract features in both the spatial domain and the frequency domain to achieve feature extraction of the global receptive field. Finally, the local information and the global information are cross-fused. In step 5.4, the fused features are upsampled and the desensitized image is output.

8. The method for desensitizing high-precision maps collected by crowdsourcing based on deep learning according to claim 1 is characterized in that: In step (6), the specific implementation of using the FasterRCNN-based sensitive target detection algorithm to locate the category and position of sensitive targets in the high-precision map includes: inputting the image to be desensitized into the convolutional neural network backbone combined with the FPN network structure to extract a multi-scale feature pyramid for subsequent sensitive target detection and classification; the regional candidate network RPN combines the multi-scale feature pyramid prediction to generate a large number of rectangular sensitive target candidate boxes; performing target box border regression and classification to determine the category and position of the sensitive target.

9. The method for desensitizing high-precision maps collected by crowdsourcing based on deep learning according to claim 1 is characterized in that: In step (8), different desensitization methods are used according to different sensitivity levels, including: for privacy-related targets with low sensitivity levels, scrambling encryption is used for the privacy area according to the coordinates corresponding to the privacy target; for shape-related targets with high sensitivity levels, image completion model LaMa is used to desensitize the sensitive area according to the coordinates corresponding to the shape-related targets; The scrambling encryption for low-sensitivity privacy targets specifically includes: direct encryption of the privacy area according to the coordinates corresponding to the privacy target, which disrupts the original image structure by changing the position and order of pixels in the image and hides the image content in the privacy area; The LaMa desensitization processing of the image completion model for shape-type targets with high sensitivity levels specifically includes: generating a mask image based on the coordinates corresponding to the shape-type targets and the original image; inputting the mask image and the original image into the LaMa algorithm, and regenerating sensitive units and facilities with the help of image features in non-sensitive areas, and the original sensitive areas are replaced with an environment similar to the surrounding area.

10. A crowdsourcing high-precision map desensitization system based on deep learning, characterized by: include: one or more processors; A storage device for storing one or more programs, which, when executed by the one or more processors, enables the one or more processors to implement the method according to any one of claims 1 to 9.

Images