Three-dimensional model sensitive character desensitization method, device and system and storage medium

Abstract

The invention discloses a three-dimensional model sensitive character desensitization method, device and system and a storage medium, and the method comprises the steps: S1, carrying out the texture image mapping of a three-dimensional model, and constructing a sensitive word library; s2, taking a texture image generated by mapping as input, and performing sensitive character automatic detection through a PP-OCRv3 network; s3, carrying out sensitive character carrier segmentation according to a character detection result of the texture image and a sensitive word library; and S4, inputting the bounding box of the non-carrier sensitive character and the carrier segmentation result into the LaMa texture regeneration network, reconstructing a texture region, and obtaining a desensitized texture image. By adopting the technical scheme of the invention, the text can be desensitized.

Description

Technical Field

[0001] The present invention belongs to the technical field of data protection, and particularly relates to a method and device, a system, and a storage medium for desensitizing sensitive text in a 3D model. Background Art

[0002] The acquisition of 3D models in urban scenes involves a large amount of detailed textures, which may contain sensitive information. Before public application, desensitization processing is required to ensure privacy and data security. In recent years, some scholars have introduced deep learning to automate the desensitization process. For example, through an automated 3D model desensitization framework, a deep convolutional neural network is used to segment sensitive patterns, and combined with GAN to generate real textures, avoiding privacy leakage in 3D reconstruction. Similarly, based on YOLOv5s for sensitive pattern target detection, and then through the Patch Match texture traversal algorithm, the texture of the sensitive area is automatically replaced for desensitization. Although the introduction of deep learning has improved the automation degree of the algorithm and the target processing effect, existing algorithms mostly focus on pattern processing and do not design effective algorithms for sensitive text target segmentation and desensitization problems. In summary, existing algorithms have the problem of being unable to desensitize text and are difficult to meet the application requirements in urban scenes. Summary of the Invention

[0003] The technical problem to be solved by the present invention is to provide a method and device, a system, and a storage medium for desensitizing sensitive text in a 3D model.

[0004] To achieve the above object, the present invention adopts the following technical solutions: A method for desensitizing sensitive text in a 3D model, comprising: Step S1, performing texture image mapping on the 3D model and constructing a sensitive word library; Step S2, taking the texture image generated by the mapping as the input, and performing automated detection of sensitive text through the PP-OCRv3 network; Step S3, performing segmentation of sensitive text carriers according to the text detection result of the texture image and the sensitive word library; Step S4, inputting the bounding box of the carrier-free sensitive text and the carrier segmentation result into the LaMa texture regeneration network to reconstruct the texture area and obtain the desensitized texture image.

[0005] Preferably, in step S3, the Canny algorithm is used to perform edge detection on the texture image containing the sensitive area; at the same time, the GrabCut algorithm is used to perform segmentation of sensitive text carriers The present invention also provides a device for desensitizing sensitive text in a 3D model, comprising: A first processing module, configured to perform texture image mapping on the 3D model and construct a sensitive word library; The second processing module is used to take the texture image generated by mapping as input and perform automated detection of sensitive text through the PP-OCRv3 network; The third processing module is used to perform segmentation of sensitive text carriers based on the text detection results of the texture image and the sensitive word library; The fourth processing module is used to input the bounding box of the carrier-free sensitive text and the carrier segmentation result into the LaMa texture regeneration network, reconstruct the texture area, and obtain the desensitized texture image.

[0006] Preferably, the third processing module uses the Canny algorithm to perform edge detection on the texture image containing the sensitive area; at the same time, the GrabCut algorithm is used to perform segmentation of the sensitive text carrier.

[0007] The present invention also provides a three-dimensional model sensitive text desensitization system, including: a memory and a processor, where a computer program run by the processor is stored on the memory, and the computer program executes the three-dimensional model sensitive text desensitization method when run by the processor.

[0008] The present invention also provides a storage medium, where a computer program is stored on the storage medium, and the computer program executes the three-dimensional model sensitive text desensitization method when running.

[0009] The present invention introduces classical computer vision methods to expand OCR, realizing the segmentation and texture regeneration of sensitive text carriers in three-dimensional data; among them, based on the lightweight OCR network PP-OCRv3, edge detection is performed through the Canny algorithm to obtain potential carriers of sensitive text. At the same time, the GrabCut image segmentation algorithm is used to obtain smoother and more accurate carrier edges, realizing the complete output of sensitive areas including text carriers; applying the LaMa texture regeneration network to perform high-quality reconstruction of texture sensitive areas and complete the texture desensitization of three-dimensional data. Description of the Drawings

[0010] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only the embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained according to the provided drawings without creative efforts.

[0011] Figure 1 It is a flowchart of the three-dimensional model sensitive text desensitization method according to the embodiment of the present invention.

[0012] Figure 2Schematic diagram of texture image processing results; among them, (a) is the original texture image, (b) is the OCR recognition mask, (c) is the edge detection result, (d) is the connected component analysis result, (e) is the potential text carrier, (f) is the mask morphological expansion, (g) is the GrabCut segmentation result, and (h) is the regenerated image. Specific implementation mode

[0013] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0014] To make the above objects, features, and advantages of the present invention more obvious and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific implementation modes.

[0015] Embodiment 1: As Figure 1 shown, an embodiment of the present invention provides a three-dimensional model sensitive text desensitization method, including: Step S1: Perform texture image mapping on the three-dimensional model and construct a sensitive word library

[0016] Step 11: Data traversal. Load the three-dimensional data of the three-dimensional model, traverse all nodes, and record the index value for each node with texture information , where is the total number of nodes with texture information.

[0017] Step 12: Texture image output. According to the index value access all triangular faces of the node, obtain the texture coordinates of each pixel point in the triangular face, and calculate the coordinates of this point in the output texture image according to the following formula.

[0018] ; where and are the length and width of the image respectively.

[0019] After the coordinate conversion is completed, map all pixel point values in the triangular face, as shown in the following formula: ; where is the th texture image output by the mapping, is the texture information of the th node. After traversing nodes with texture information, output two-dimensional texture images to be processed.

[0020] Step 13: Construction of the text rule library. Construct a sensitive word library according to application requirements, and standardize the two libraries into dictionary format.

[0021] Step S2: Use the texture images generated by mapping as input, and perform automated detection of sensitive text through the PP-OCRv3 network Step 21: Text detection. Input the texture image, extract features through the backbone network in the form of a pyramid, upsample all pyramid feature maps to the same size, and output the feature map : ; Generate a probability map and a threshold map through convolutional layer image feature fusion. Among them, and are the length and width of the feature map respectively. Subsequently, perform a variant on the basis of standard binarization, and perform differential binarization calculation according to the following formula to obtain a binary map , and the binary map of the segmentation result is as shown in Figure 2 (b) of

[0022] ; Among them, is a binarized map containing only 0 and 1, 1 represents a valid text area, and 0 represents the background area. and are the probability map and threshold map learned in the network, is the enhancement factor of the DB network, set to 50.

[0023] Step 22: Orientation classification. Obtain the set of all text areas in the detection stage, where are the abscissa, ordinate, length, and width of the detected text area in the image respectively. To determine the orientation of the text area and ensure correct recognition of horizontally and vertically arranged text, normalize the text area and output the orientation category: ; Among them, is the orientation category, It is a lightweight network.

[0024] Subsequently, the direction prediction result is classified through the softmax function to output the direction correction result of the text region , where is the regional rotation angle.

[0025] Step 23: Text detection. Based on the text region after direction correction , pass through the SVTR_LCNet text recognition network: ; Among them, are the extracted local features and global features.

[0026] Then, the extracted features pass through the Transformer encoder to generate the prediction result, and the self-attention mechanism improves the recognition and classification accuracy: ; Among them, is the output feature after passing through the encoder.

[0027] After the classification head generates the probability distribution of each category, connect the connectionist temporal classification CTC to decode the final text string: ; Among them, is the probability distribution of each character category.

[0028] Finally, output the text detection and segmentation results of the texture image , where and are the image text bounding box and text classification result respectively, is the number of texture images, is the number of texture image text segmentation boxes.

[0029] Step S3: Sensitive text carrier segmentation Take the height limit of important bridges and road signs as the sensitive word library Content, illustrate the process of text carrier segmentation of the desensitization algorithm in the embodiments of the present invention, and the specific steps are as follows: Step 31: Sensitive text matching. Regular expression match the text detection result of each texture image with the sensitive word library. Through the match, screen out the text regions containing sensitive words in the texture image to form a set of sensitive regions to be processed. Provide a preliminary positioning of the target region for subsequent edge detection and connected component analysis: ; Among them, represents the string of the text detection result, Indicates regular matching, is the sensitive word library, represents the number of sensitive strings in the word library.

[0030] Step 32: Potential carrier detection. For the texture image containing sensitive regions, the Canny algorithm is used for edge detection, and the detection result is as shown in Figure 2 (c) of. On this basis, connected component analysis is performed on the edge image to identify independent connected regions and fill them to form a closed region , and the filling result is as shown in Figure 2 (d) of.

[0031] For the bounding box of sensitive text in it is said that the number of pixels within the bounding box is , and it is compared with each connected component . If the pixels exceeding a certain threshold in coincide with , that is the number of pixels with white color within the connected component region within the bounding box range, are the upper-left coordinates and the length and width of the bounding box respectively. The qualified are retained to form the segmentation result of potential text carriers, as shown in Figure 2 (e) of.

[0032] Step 33: Carrier segmentation. Since the Canny edge detection may have certain texture edge missing in complex backgrounds, such as Figure 2 the white edge of the SITONG Bridge sign in the upper-right corner of (a) in Figure 2 is ignored in (e) of. To increase the detection tolerance and improve the carrier segmentation effect, the outer bounding box of the potential carrier is expanded and calculated using the following formula, and the expansion result is as shown in Figure 2 (f) of.

[0033] ; where, , , and are the upper-left coordinates and the length and width of the outer bounding box of the original segmentation result respectively, is the expansion pixel value.

[0034] Subsequently, using the expanded mask and the original image data as input, GrabCut image segmentation is performed, and at this time, an accurate carrier segmentation result is obtained. The segmented mask is as shown in Figure 2As shown in (g). The segmented text carrier and the bounding box of sensitive text where no carrier is detected are used as inputs for the next masking operation.

[0035] Step S4: Input the bounding box of sensitive text without a carrier and the carrier segmentation result into the LaMa texture regeneration network to reconstruct the texture region and obtain the desensitized texture image: Step 41: Mask dilation. Dilate the bounding box of sensitive text or the bounding box of the text carrier to a certain extent to expand the texture repair range and reduce the influence caused by object shadows, etc.

[0036] Step 42: Text region regeneration. Normalize the size of the original texture image and pass the context information to the decoder through the LaMa network to restore the missing texture and achieve seamless texture connection: ; After the pixels in the missing area are regenerated, update the original texture image, as Figure 2 shown in (h).

[0037] Step 43: Texture inverse mapping. Map the desensitized texture image back to the original 3D data, and the texture image where no sensitive content is detected is not mapped; finally, output the desensitized 3D data.

[0038] Embodiment 2: The embodiment of the present invention also provides a three-dimensional model sensitive text desensitization device, including: Step S1: Perform texture image mapping on the three-dimensional model and construct a sensitive word library; Step S2: Use the texture image generated by mapping as an input and perform automated detection of sensitive text through the PP-OCRv3 network; Step S3: Perform segmentation of the sensitive text carrier according to the text detection result of the texture image and the sensitive word library; Step S4: Input the bounding box of sensitive text without a carrier and the carrier segmentation result into the LaMa texture regeneration network to reconstruct the texture region and obtain the desensitized texture image.

[0039] As an implementation manner of the embodiment of the present invention, the third processing module uses the Canny algorithm to perform edge detection on the texture image containing the sensitive area; at the same time, uses the GrabCut algorithm to perform segmentation of the sensitive text carrier.

[0040] Embodiment 3: The embodiment of the present invention also provides a three-dimensional model sensitive text desensitization system, including: a memory and a processor, where a computer program is stored on the memory and run by the processor, and the computer program, when run by the processor, executes the three-dimensional model sensitive text desensitization method.

[0041] Example 4: An embodiment of the present invention further provides a storage medium, on which a computer program is stored, and the computer program executes the three-dimensional model sensitive text desensitization method when running.

[0042] The above embodiments are only descriptions of the preferred embodiments of the present invention, and do not limit the scope of the present invention. Without departing from the design spirit of the present invention, various deformations and improvements made by those of ordinary skill in the art to the technical solutions of the present invention should fall within the protection scope determined by the claims of the present invention.

Claims

1. A method for desensitizing sensitive text in a three-dimensional model, characterized in that: include: Step S1, performing texture image mapping on the three-dimensional model and building a sensitive word library; Step S2: Using the texture image generated by mapping as input, the PP-OCRv3 network is used to automatically detect sensitive text; Step S3, segmenting sensitive text carriers according to the text detection results of the texture image and the sensitive word library; Step S4: input the bounding box of the non-carrier sensitive text and the carrier segmentation result into the LaMa texture reconstruction network, reconstruct the texture area, and obtain the desensitized texture image.

2. The method for desensitizing sensitive text in a three-dimensional model as claimed in claim 1, characterized in that: In step S3, the Canny algorithm is used to perform edge detection on the texture image containing the sensitive area; and the GrabCut algorithm is used to segment the sensitive text carrier.

3. A 3D model sensitive text desensitization device, characterized in that: include: The first processing module is used to perform texture image mapping on the three-dimensional model and build a sensitive word library; The second processing module is used to use the texture image generated by the mapping as input and automatically detect sensitive text through the PP-OCRv3 network; The third processing module is used to segment the sensitive text carrier according to the text detection result of the texture image and the sensitive word library; The fourth processing module is used to input the bounding box of the non-carrier sensitive text and the carrier segmentation result into the LaMa texture reconstruction network, reconstruct the texture area, and obtain the desensitized texture image.

4. The three-dimensional model sensitive text desensitization device as claimed in claim 3, characterized in that: The third processing module uses the Canny algorithm to perform edge detection on the texture image containing sensitive areas; at the same time, it uses the GrabCut algorithm to segment sensitive text carriers.

5. A three-dimensional model sensitive text removal system, characterized in that: include: A memory and a processor, wherein the memory stores a computer program executed by the processor, and when the computer program is executed by the processor, the method for desensitizing sensitive text in a three-dimensional model as described in any one of claims 1 to 2 is executed.

6. A storage medium, characterized in that: The storage medium stores a computer program, which, when running, executes the method for removing sensitive text from a three-dimensional model as described in any one of claims 1 to 2.

Images