Book inside page stain detection and repair suggestion system, method and device and storage medium

Abstract

The invention relates to the technical field of image processing and computer vision, and discloses a book inside page stain detection and restoration suggestion system, method and device and a storage medium, and the system comprises an image preprocessing module, a damage detection module and a restoration suggestion generation module. The image preprocessing module converts an original image into a CIELAB color space, only performs brightness equalization operation on a brightness channel to overcome the influence of uneven illumination, and retains contamination information in a chrominance channel at the same time. The damage detection module loads a pre-trained deep convolutional neural network model, and performs pixel-level semantic segmentation on the standardized to-be-detected image; the model is trained by adopting a combined loss function of multi-class cross entropy loss and multi-class Dess loss so as to solve the problem of class imbalance of a damaged area and a background. The repair suggestion generation module analyzes the damage mask and generates a repair suggestion according to a rule-based decision tree. According to the invention, automatic detection and suggestion are realized, and the detection precision and efficiency are improved.

Description

Technical Field

[0001] This invention relates to the fields of image processing and computer vision technology, specifically to a system, method, apparatus, and storage medium for detecting and repairing damage to book pages. Background Technology

[0002] Paper documents such as books and archives are susceptible to damage from environmental factors, temperature, and humidity during long-term preservation, resulting in various forms of stains such as mold, water stains, and yellowing. Timely and accurate detection and assessment of these stains are crucial prerequisites for carrying out document conservation and restoration work.

[0003] Currently, the detection of soiled pages in books mainly relies on manual visual inspection. This method is not only time-consuming and labor-intensive, resulting in low efficiency, but also highly dependent on the professional experience of the restorers. The lack of objective and unified standards leads to a high degree of subjectivity and inconsistency in the assessment results.

[0004] To improve efficiency, some technical solutions have attempted to use computer vision technology for automated inspection. However, during the acquisition of images of book pages, uneven lighting often occurs due to the curved surface of the pages or the influence of ambient light. Existing processing methods typically analyze the image directly without effectively separating brightness interference from the chromaticity characteristics of the stain itself. This leads to shadow areas caused by uneven lighting being easily misjudged as stains, while true stain information (such as mold) is also masked by brightness differences, resulting in poor detection accuracy.

[0005] Furthermore, when using deep learning models for automated segmentation, the damaged areas (foreground) in book page images are typically much smaller than the intact background areas, resulting in a significant pixel class imbalance. Without intervention, the model will be biased towards the background class during training, leading to insufficient ability to identify small or irregularly edged damaged areas, and the segmentation accuracy will be insufficient to meet the needs of practical applications. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this invention provides a system, method, apparatus, and storage medium for detecting and repairing damage to book inner pages. It solves the problems in existing technologies, such as uneven illumination during image acquisition interfering with detection accuracy, poor segmentation accuracy due to imbalance between damaged areas and background pixel categories, and the high subjectivity and low efficiency of traditional manual inspection.

[0007] The first aspect of this invention provides a book page soiling detection and repair suggestion system, comprising: The image preprocessing module is used to perform color space conversion and brightness equalization operations on the acquired original images of the book's inner pages to output a standardized image to be detected. The damage detection module, connected to the image preprocessing module, is used to load a pre-trained deep convolutional neural network model to perform pixel-level semantic segmentation on the standardized image to be detected, so as to output a damage mask that identifies the damage category and location; the repair suggestion generation module, connected to the damage detection module, is used to analyze the damage mask to determine the type, degree and location features of the damage region, and generate repair suggestions that match the features based on an internal rule-based decision tree.

[0008] In one specific implementation, the image preprocessing module is used to convert the original image from the RGB color space to the CIELAB color space to separate the luminance channel and the chrominance channel. This conversion helps to decouple the luminance and chrominance information of the image.

[0009] In a further embodiment, the image preprocessing module is also used to perform luminance equalization only on the luminance channel, and then re-merge the equalized luminance channel with the chrominance channel and convert it back to the RGB color space. This processing method aims to eliminate the luminance differences in the original image caused by uneven lighting, while retaining the chrominance information of the chrominance channel (e.g., caused by water stains or mold), thereby improving the accuracy of subsequent detection.

[0010] In one specific implementation, the deep convolutional neural network model in the damage detection module is trained using a combined loss function. This combined loss function can be a weighted combination of multi-class cross-entropy loss and multi-class Dessian loss.

[0011] By combining multi-class cross-entropy loss and multi-class Dessian loss, the pixel class imbalance problem between damaged areas (foreground) and intact areas (background) in book inner page images can be effectively addressed, improving the model's segmentation accuracy for small targets or edge region damage.

[0012] In one specific implementation, the repair suggestion generation module is specifically used to: perform post-processing (e.g., connected component analysis) on the damage mask to identify one or more independent damage regions; and extract a set of damage feature vectors for each independent damage region, wherein the damage feature vectors may include damage type, damage degree and damage location information.

[0013] In a further embodiment, the repair suggestion generation module is also used to use the damage feature vector as input to the rule-based decision tree to match and output the repair suggestion.

[0014] A second aspect of this invention provides a method for detecting and repairing soiled pages in books, comprising the following steps: S201, Image preprocessing step: Perform color space conversion and brightness equalization operations on the acquired original images of the book's inner pages to output a standardized image to be detected; S202, Damage detection step: Load a pre-trained deep convolutional neural network model, perform pixel-level semantic segmentation on the standardized image to be detected, and output a damage mask that identifies the damage category and location. S203, Damage Feature Analysis Step, used to analyze the damage mask to determine the type, extent, and location characteristics of the damaged area; S204, Repair Suggestion Generation Step, used to generate matching repair suggestions based on the features determined in the Damage Feature Analysis Step.

[0015] A third aspect of the present invention provides a device for detecting and suggesting repairs for soiled pages of books, comprising: A processor, a memory, and a computer program stored in the memory, which, when executed by the processor, is used to implement the book page soiling detection and repair suggestion method as described in the second aspect of the present invention.

[0016] A fourth aspect of the present invention provides a storage medium for suggesting repairs to damaged book pages, wherein a computer program is stored thereon, and when the computer program is executed by a processor, it is used to implement the book page damage detection and repair suggestion method as described in the second aspect of the present invention.

[0017] In one specific implementation, the computer program may include: Image preprocessing instructions are used to perform the image preprocessing steps; Damage detection instructions are used to execute the damage detection steps; Damage feature analysis command, used to execute the damage feature analysis steps; Repair suggestion generation instructions are used to execute the repair suggestion generation step. This invention provides a system, method, apparatus, and storage medium for detecting and repairing soiled pages in books. It offers the following advantages: 1. This invention achieves the separation of luminance and chromaticity information by employing the CIELAB color space in image preprocessing. The solution performs equalization only on the luminance channel, effectively overcoming the interference of uneven illumination during shooting on image brightness, while fully preserving the chromaticity features reflecting water stains, mold, and other damage in the chromaticity channel, thus improving the accuracy of subsequent damage detection.

[0018] 2. In the training of the deep convolutional neural network model, this invention employs a weighted combination of multi-class cross-entropy loss and multi-class Dessian loss. This combined loss function can effectively solve the significant pixel class imbalance problem between damaged areas (foreground) and intact areas (background) in book page images, improving the model's segmentation accuracy for irregular edges or small-area damaged areas.

[0019] 3. This invention provides a complete automated process that not only achieves automatic damage detection and location, but also analyzes damage characteristics through a repair suggestion generation module and automatically generates repair suggestions using rule-based decision trees. This solution replaces the traditional method that relies on manual visual inspection and experience-based judgment, significantly improving the efficiency of book page soiling treatment and ensuring the objectivity and consistency of repair suggestions. Attached Figure Description

[0020] Figure 1 This is a block diagram of the functional modules of the system of the present invention; Figure 2 This is a flowchart of the method of the present invention.

[0021] The components are as follows: 110, Image Acquisition Interface; 120, Image Preprocessing Module; 130, Damage Detection Module; 140, Repair Suggestion Generation Module; 150, Data Storage Unit; and 160, Result Display Interface. Detailed Implementation

[0022] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] See attached document Figure 1 This invention provides a book page soiling detection and suggestion system, the system comprising: Image acquisition interface 110 is used to connect to an external image input device to acquire the original image I of the inner page of the book to be inspected. in The image acquisition interface 110 acquires the raw image I. in It is then transmitted to the image preprocessing module 120 for subsequent standardization processing.

[0024] Image preprocessing module 120 is used to process the acquired original images of book inner pages I in Perform color space conversion and brightness equalization operations to reduce the impact of ambient lighting on image quality, and output a standardized image I to be detected. pre .

[0025] The damage detection module 130 is connected to the image preprocessing module 120. The damage detection module 130 contains a pre-trained deep convolutional neural network model for receiving I... pre It performs deep feature extraction and pixel-level classification, and finally outputs a damage mask that identifies the type of damage or stain and its spatial location.

[0026] The repair suggestion generation module 140 is connected to the damage detection module 130. The repair suggestion generation module 140 is used to receive damage masks. The system analyzes the type, extent, and location of the damaged area, and generates repair suggestions R that match the damage characteristics based on an internal rule-based decision tree. sug .

[0027] Data storage unit 150 is used to store program instructions and various data required for the operation of this system. Data storage unit 150 stores computer programs used to implement the functions of image preprocessing module 120, damage detection module 130, and repair suggestion generation module 140. Data storage unit 150 is also used to store various types of data used or generated during system operation, including but not limited to: acquired raw image I. in Preprocessed image I pre The sample library used to train the damage detection model, the trained deep convolutional neural network model file, the rule decision tree used by the repair suggestion generation module 140, and the final generated repair suggestion R sug .

[0028] In a preferred embodiment, to support precise feature analysis in step S203, the data storage unit 150 is also used to store: Layout and severity determination parameters: These parameters are necessary for the repair suggestion generation module 140 to extract feature vectors in stage S203, including but not limited to: parameters used for D S The area thresholds T1 and T2, and the morphological threshold T are used for judgment. skel ,T in ; and for D L The coordinate template or physical dimensions (such as 2cm margins) used to determine and describe the standard page layout.

[0029] Intermediate scale calibration data: During system operation, the scale calibration data is generated by the image preprocessing module 120 (S201) and passed to the repair suggestion generation module 140 (S203). Specifically, it includes the effective DPI value of the current image and the page pixel height H. page and page pixel width W page .

[0030] The result display interface 160 is connected to the repair suggestion generation module 140 and is used to receive and output the final processing results of the system. The result display interface 160 is used to present the detection results to the user on a user interface. In one embodiment, the presentation includes: a damage mask output by the damage detection module 130. Perform visualization processing (e.g., highlight it and overlay it on the original image I). in (above), to indicate the spatial location of the damage; in a preferred embodiment, in order to achieve this 'highlight overlay', the processor 310 (when executing the functions of the result display interface 160) must solve the resolution mismatch between the damage mask and the original image.

[0031] Specifically, the damage mask Y output by the damage detection module 130 (or the category index map Y generated by S203 with a resolution of, for example, 512×512) index Its spatial dimension is much smaller than that of the original image I. in (or an ROI detected in S201 with a resolution of, for example, 4000×6000) page ).

[0032] Therefore, before overlaying, the processor 310 must first examine the low-resolution category index map Y. index Perform an upsampling operation to enlarge its spatial dimensions to match the ROI. page (or I) in The pixel sizes are completely identical.

[0033] To prevent illegal class values ​​or blurred boundaries from being generated during the upsampling process due to interpolation (e.g., creating a class 1.5 artifact between class 1 and class 2), the upsampling operation preferably uses the nearest neighbor interpolation algorithm.

[0034] Upsampling yields a high-resolution mask Y of the same size as the original image. highres Then, the processor 310 will then transfer the Y... highres Convert to a color-coded visualization (e.g., map category 1 (water stains) to blue and category 4 (tears) to red), and correlate it with the ROI. page Image (or I) in Alpha transparency blending is performed to generate the final visual overlay image.

[0035] Finally, the result display interface 160 overlays the visualization image, and the repair suggestion generation module 140 outputs text repair suggestions R. sug .

[0036] See attached document Figure 2 , Figure 2This is a flowchart of a method for detecting soiled pages in a book according to an embodiment of the present invention. The method is described in detail below. Figure 1 The workflow of the system shown may include the following steps: Step S201: Image Acquisition and Preprocessing. The system acquires the original images I of the book's inner pages. in Subsequently, the image preprocessing module 120 processes I... in Processing is performed. In one specific embodiment, the preprocessing operation includes: transferring I... in The image is converted from RGB color space to CIELAB color space to separate luminance and chrominance. Then, histogram equalization is performed only on the luminance channel to correct for uneven illumination, resulting in the preprocessed image I. pre .

[0037] Step S202: Damage Detection and Localization. The damage detection module 130 loads a pre-trained deep convolutional neural network model and processes the image I... pre As input, the model performs forward inference on I. pre Perform pixel-by-pixel semantic segmentation and output a pixel-level probability map, i.e., a damage mask. The mask It assigns the probability that each pixel in the image belongs to a different category, such as background, stain, or damage.

[0038] Step S203: Damage Feature Analysis. The repair suggestion generation module 140 receives the damage mask output in step S202. This module first... Post-processing is performed, such as binarization by setting a probability threshold and performing connected component analysis, to identify and separate one or more independent damage regions R. j .

[0039] Next, the module analyzes each damaged area R j Extract its structured feature vector. This feature vector should include at least: damage type D. T (Determined by the classification results of the DCNN model), damage level D S (e.g., based on R) j pixel area A j (Calculation) and damage location D L (e.g., based on R) j The coordinates of the centroid (c x ,c y (This is determined by comparing it with a predefined page layout (such as the text area and blank areas).

[0040] Step S204: Repair suggestion generation and output. The repair suggestion generation module 140 will generate and output the feature vector (D) extracted in S203. T DS D L The feature vector is used as input to query the internally stored rule-based decision tree g(·). Based on a pre-defined expert knowledge base, the decision tree matches the repair logic corresponding to the feature vector, ultimately determining and outputting a repair suggestion R for the damage. sug .

[0041] Image preprocessing module 120 is used to perform Figure 2 The step S201 shown is described. The main function of this module is to receive the original image I of the book's inner pages. in The image is then standardized to reduce the interference of complex background textures and uneven lighting in the original image on subsequent damage detection steps.

[0042] In a preferred embodiment, the above RGB to CIELAB color space conversion is performed based on the D65 standard white point and the sRGB standard at a 2° field of view. The equalized L... eq After merging the original a channel, b channel, and alpha channel into a CIELAB image, and then converting it back to RGB space, we get the I channel. pre At the same time, the same D65 white point and sRGB standards are used to ensure color consistency.

[0043] In one specific implementation, the image preprocessing module 120 first processes the input raw image I, which uses the RGB color space. in Perform a color space conversion. This conversion operation will convert the color space to I... in Transform to CIELAB color space to obtain I lab The transformation relationship can be expressed as: I in (RGB)→I lab (L,a,b); The purpose of this operation is to separate the image's luminance information (L channel) from its chrominance information (a channel and b channel). Subsequent luminance equalization will then be performed only on the L channel, while preserving the original chrominance information of the a and b channels, thus preventing color distortion during illumination correction.

[0044] Next, the image preprocessing module 120 processes I... lab The L channel in the image undergoes a brightness equalization operation to correct overly bright or dark areas in the image caused by uneven lighting during shooting. In one embodiment, this operation is implemented using histogram equalization.

[0045] Specifically, let H(l) be the pixel frequency (i.e., histogram) of the brightness value l in the L channel, where the value of l ranges from [0, L]. max ], L max This is the maximum brightness level (e.g., 255).

[0046] First, calculate the normalized cumulative distribution function C(l) of channel L, which is calculated using the following formula: Where, N p ∑ is the total number of pixels in the image; · . • This is the summation symbol.

[0047] Subsequently, the cumulative distribution function C(l) is applied as the transformation function T(l) to map the original brightness value L(x,y) of each pixel in the L channel to a new equalized brightness value L. eq (x, y). The mapping relationship is as follows: L eq (x,y)=T(L(x,y))=round(C(L(x,y))·L max ); Where (x,y) are pixel coordinates, and round(·) is the rounding function.

[0048] Finally, the image preprocessing module 120 will equalize the new L... eq The channels are then merged with the unprocessed original a and b channels to form a new CIELAB image. The image preprocessing module 120 then converts this CIELAB image back to the RGB color space to obtain the I... rgbeq .

[0049] In a preferred embodiment, before outputting the image to the damage detection module 130, the image preprocessing module 120 also performs page detection and scale calibration operations to ensure that the feature extraction based on physical size and relative size (D) in S203 is performed correctly. S and D L (Judgment) can be executed accurately.

[0050] This operation includes: Physical scale acquisition: Image preprocessing module 120 acquires the original image I from image acquisition interface 110. in The metadata, particularly its scan resolution (DPI) value, is stored as a system-global variable used to convert all subsequent physical units (such as cm or inch) to pixels. For example, a 2cm page margin would be converted to (2 / 2.54) × DPI pixels.

[0051] Page area detection: To prevent the scanned background from interfering with subsequent analysis (such as page height calculation), the image preprocessing module 120 performs I... rgbeqPerform layout analysis or edge detection algorithms (e.g., using Canny edge detection and Hough transform to find the largest rectangular outline) to automatically locate the main area of ​​the book's inner pages, i.e., the Region of Interest (ROI). page .

[0052] Image cropping and normalization: Image preprocessing module 120 based on ROI page to I rgbeq Cropping is performed to remove irrelevant background areas, resulting in the final preprocessed image I. pre This module also stores the ROI. page pixel height H page and pixel width W page .

[0053] The final I pre The image (cropped) is output to the damage detection module 130. The acquired DPI value and H... page and W page (The scale calculations used in S203) are then transmitted to the repair suggestion generation module 140.

[0054] Damage detection module 130 is connected to image preprocessing module 120 for performing... Figure 2 Step S202 is shown below. The function of the damage detection module 130 is to receive a standardized image I. pre Load a pre-trained deep convolutional neural network (DCNN) model and apply it to I. pre Perform pixel-level semantic segmentation, and finally output a damage mask that identifies the damage type and location.

[0055] In one specific implementation, the damage detection module 130 employs a deep convolutional neural network model that is an encoder-decoder architecture. This architecture is particularly suitable for image semantic segmentation tasks.

[0056] In a preferred embodiment of the present invention, the DCNN model adopts a U-Net network architecture. Its encoder part uses a ResNet-50 model as the backbone network for feature extraction to take advantage of its powerful pre-trained feature representation capabilities; the decoder part includes four symmetrical upsampling operations, and resolution restoration is achieved through transposed convolution.

[0057] The encoder portion of this architecture consists of a series of stacked convolutional and pooling layers. The encoder receives I... pre After the image is processed, image features are extracted through layer-by-layer convolution operations, and the spatial resolution of the feature map is gradually reduced through pooling operations, thereby capturing the deep abstract semantic information of the image.

[0058] The decoder portion of this architecture consists of a series of upsampling layers (e.g., transposed convolutions) and convolutional layers. The decoder receives abstract semantic features from the encoder output and gradually recovers the spatial resolution of the feature map through upsampling operations to achieve pixel-level localization.

[0059] To combine the shallow, high-resolution features (containing precise location and texture information) in the encoder with the deep semantic features (containing category information) in the decoder, this architecture also includes skip connections. Skip connections concatenate or add feature maps from different stages in the encoder with feature maps of corresponding spatial resolutions in the decoder, enabling the decoder to utilize multi-scale features simultaneously when recovering image details, thereby improving segmentation accuracy.

[0060] The model deployed in the damage detection module 130 needs to be trained offline. The training dataset consists of several (e.g., 5000) images of book pages with different degrees and types of damage. These images are pixel-level annotated by professional restorers, clearly delineating background areas, different types of stains (such as water stains, mold, and oil stains), and damaged areas (such as tears, insect damage, and missing parts). To improve the model's generalization ability, data augmentation operations, including random rotation, flipping, scaling, and color jitter, are performed on the dataset before training.

[0061] The goal of training is to minimize the difference between the damage mask predicted by the model and the real mask that is finely annotated by humans.

[0062] Assume this invention defines K categories (e.g., K=4, corresponding to 0: background, 1: stain, 2: damage, 3: defect). For an input image, the model outputs a prediction mask Y, which is an H×W×K probability map, where y i,k Represents the probability that pixel i belongs to category k; the true label mask. It is an H×W×K one-hot encoded tensor, where

[0063] In a more specific embodiment, to distinguish damage with different repair needs, K is actually set to 7, specifically including: 0: background, 1: water stains, 2: mold, 3: ink stains, 4: tears, 5: insect damage / perforation, 6: defects. The model training aims to directly output pixel-level probabilities for these 7 categories, but in the repair suggestion generation module 140, types 1, 2, and 3 are all categorized as D. T =Stains, types 4, 5, and 6 are all classified as D. T = Make decisions based on damage.

[0064] The final activation function of the model should be the Softmax activation function to ensure that for each pixel i, the sum of its probabilities across all K classes is 1.

[0065] Given that various damaged regions in book page images typically occupy only a small portion of the total image area, a significant class imbalance problem exists. To address this issue, this embodiment of the invention employs a combined loss function during model training. This function is derived from multi-class cross-entropy loss. and multi-category Des losses It is composed of weighted combinations.

[0066] During training, all input I pre Images are first uniformly cropped or scaled to 512×512 pixels. Training uses the Adam optimizer with an initial learning rate set to 1e-4, fine-tuned using cosine annealing. The loss function... In this context, the hyperparameters λ1 and λ2 used to balance the loss contribution are preferably set to λ1 = 0.4 and λ2 = 0.6.

[0067] Multi-class cross-entropy loss The formula used to calculate the difference between the predicted probability and the true label pixel by pixel is as follows: Where, N p K represents the total number of pixels in the image; K represents the total number of categories. is the summation symbol; log(·) is the logarithmic function.

[0068] Multi-category Des losses This loss function directly optimizes the overlap between the predicted and ground truth regions, exhibiting good robustness to imbalanced classes. Its calculation formula (based on the average Dice of K classes) is as follows: Here, ∈ is a very small smoothing constant (e.g., 1e-6) used to prevent the denominator from being zero.

[0069] Total loss function The weighted sum of the two types of losses mentioned above: Here, λ1 and λ2 are hyperparameters used to balance the contributions of the two types of loss.

[0070] During step S202, the damage detection module 130 loads the multi-classification model trained as described above. The module receives image I. pre Forward inference is then performed, and the model's Softmax layer generates a pixel-level probability map Y of size H×W×K. This Y is then passed to the repair suggestion generation module 140.

[0071] Repair suggestion generation module 140 is connected to damage detection module 130 for execution. Figure 2 Steps S203 and S204 are shown in the diagram. The function of the repair suggestion generation module 140 is to receive the damage mask output by the damage detection module 130. Perform feature analysis on the mask and generate corresponding repair suggestions R based on the analysis results. sug .

[0072] In one specific implementation, when step S203 is executed, the repair suggestion generation module 140 first performs a post-processing operation on the H×W×K probability map (damage mask Y) output by the damage detection module 130. This is not a simple binarization, but includes the following explicit steps: Class determination: First, perform an Argmax operation along the K-dimensional (class channel) path of the probability map Y to determine the index k of the class with the highest probability for each pixel (x,y). max This will generate an H×W category index map Y. index .

[0073] Generate damage mask: Next, based on Y index Generate a binary damage mask Y binary The rule for generating this mask is: if Y index If (x,y)>0 (i.e., the pixel is classified as any damage category other than the background), then Y binary (x,y)=1(foreground / damage); if Y index (x,y)=0 (i.e., background category), then Y binary (x,y)=0.

[0074] Connected component analysis: Subsequently, this module only analyzes Y. binary Perform a connected component analysis algorithm (e.g., 8-connectivity algorithm) to identify and separate one or more independent damaged regions (i.e., connected components with a pixel value of 1) R = {R1, R2, ..., R...} m}

[0075] Next, the repair suggestion generation module 140 performs repair suggestions on each identified damaged area R. j Extract a set of structured damage feature vectors (D) T D S D L ).

[0076] Damage type D T The determination of an independent damage region R j The repair suggestion is to generate module 140 to count all pixels within it in the Y region. index The mode of the corresponding category index in the figure (i.e., the category index that appears most frequently). The damage category corresponding to this mode index (e.g., water stains or tears) is determined as the final damage type D for that area.T If there are multiple modes, the category with the highest risk level or repair priority is preferentially selected as D T .

[0077] Degree of damage D S Quantification of, and this quantification logic is category-based. The repair recommendation generation module 140 first calculates the pixel area A j of the identified damaged area R j . The system presets area thresholds T1 (e.g., 1% of the total pixel area of the page) and T2 (e.g., 5%).

[0078] Rule A (for stain types): If the damage type D T of this area is determined to be a stain (or water stain, mildew stain, ink stain, etc. defined in the previous round of supplementation), then the determination of D S is only based on the area: If A j < T1, then D S is defined as minor; If T1 ≤ A j < T2, then D S is defined as moderate; If A j ≥ T2, then D S is defined as severe.

[0079] Rule B (for breakage types): If D T is determined to be a breakage (or tear, insect damage, defect, etc.), then the determination of D S adopts a morphological rule with a higher priority to solve the problem that the severity of slender tears or large holes is underestimated.

[0080] At this time, the repair recommendation generation module 140 must also calculate two other morphological features of R j : the skeleton length L skel (calculated by the skeleton extraction algorithm) and the diameter D in of the largest inscribed circle.

[0081] The system also needs to preset morphological thresholds, for example: the skeleton length threshold T skel (e.g., 20% of the page height) and the inscribed circle threshold T in (e.g., 5% of the page width).

[0082] The determination logic of D S (for breakage types) is as follows: Preferentially determine severe: If L skel ≥ T skel (indicating the existence of a long tear) or D in ≥ T in(Indicating a large area of ​​damage or a large hole), then immediately remove D. S The condition is deemed serious, regardless of its area A. j How small.

[0083] Regress to area determination: If none of the above priority determination criteria for severity are met (i.e., neither a long tear nor a large hole), then D S The decision reverts to rule A, meaning it is entirely based on its area A. j The severity is determined by comparison with T1 and T2 to determine whether it is mild or moderate.

[0084] This hierarchical rule explicitly defines how morphological features (skeleton / inscribed circle) assist area features (A). j This is achieved through a high-priority severe veto rule.

[0085] Damage location D L The determination is based on R j The coordinates of the centroid (c x ,c y The system predefines the layout coordinate areas of the book's inner pages (e.g., text area, margin area, illustration area). This is determined by judging (c...). x ,c y Which predefined region does D fall into to determine D? L The layout coordinate area can be automatically obtained by pre-matching templates to standard book pages or by layout analysis algorithms (such as projection methods).

[0086] The definition of layout coordinate areas should be based on the geometric properties of a standard book page. For example, on an A4-sized page image, the margin area can be predefined as an area more than 2cm from the image edge; the text area is defined by a set of rectangular boxes B = {Box1, Box2, ...} determined based on projection methods or deep learning layout analysis algorithms; and the illustration area is defined as any large non-text area that does not belong to the former two. If (c x ,c y If the text falls within the main text area, then D L This is the main text area; otherwise, if it falls within 2cm of the page margin, it is considered the margin blank area.

[0087] During step S204, the repair suggestion generation module 140 will use the damage feature vector (D) extracted in step S203. T D S D L ), which serves as the input to the internal rule-based decision tree g(·).

[0088] The decision tree g(·) is designed to cover all 3×3×3=27 main combinations, ensuring that recommendations are provided for any detected impairments. For cases where the feature vector is not explicitly defined in the decision tree (i.e., no match), g(·) should output a default safety recommendation R. default If atypical damage is detected, it is recommended to first perform digital archiving and consult a senior restoration expert; do not attempt to handle it yourself at this time. This mechanism ensures that the system will not output uncertain results under any circumstances.

[0089] The decision tree g(·) is a logical structure built from expert knowledge in the field of book restoration, used to map the combination of input feature vectors to a specific restoration suggestion R. sug The mapping relationship of the decision tree g(·) can be represented as: R sug =g(D T D S D L ); This decision tree contains multiple rules. For example, a rule can be defined as: if D T For stains, D L This is the margin area, and D S If it is slight, then the output R of g(·) is... sug Recommendation: Use Type A chemical dry cleaning agent for spot cleaning.

[0090] Another rule can be defined as: if D T For stains, and D L If it is the main text area, then R sug Warning: High-risk procedure. Recommendation: Prioritize high-resolution digital archiving, followed by evaluation by a professional restorer (Type B wet cleaning method).

[0091] Another rule can be defined as: if D T For damage, D S It is moderate, and D L If it is a blank area in the page margin, then R sug Recommendation: Use acid-free repair paper tape (T-1 type) for physical reinforcement.

[0092] Repair suggestion generation module 140 will eventually match the R sug Output to the results display interface.

[0093] This invention also provides a book page soiling detection and suggestion device, which is a hardware implementation of the aforementioned book page soiling detection and suggestion system.

[0094] The device for detecting soiled pages in books includes a processor, a memory, and a bus for connecting the processor and the memory.

[0095] Memory is used to store computer programs (instructions), which are executed by the processor to implement the aforementioned... Figure 2 The method for detecting soiled pages in books is shown. The memory can also be used to store various data required during method execution, such as the original image I. in Preprocessed image I pre Deep convolutional neural network model files and damage masks

[0096] The processor is used to execute computer programs stored in memory to implement specific tasks. Figure 2 Steps S201 to S204 are shown.

[0097] Specifically, the processor executes the program to achieve Figure 1 The functions of each module: The processor executes instructions to implement the functions of the image preprocessing module 120, to perform the color space conversion and brightness equalization operations in step S201.

[0098] The processor executes instructions to implement the function of the damage detection module 130, loads the deep convolutional neural network model, and performs damage detection and localization in step S202.

[0099] The processor executes instructions to implement the function of the repair suggestion generation module 140, performing damage feature analysis in step S203 and repair suggestion generation in step S204.

[0100] In addition, the processor executes instructions to actively control the input / output (I / O) interfaces in the system. Specifically, the processor drives the image acquisition interface 110 to perform image capture operations and actively queries and reads the metadata (e.g., DPI values) of the acquired image. After S204 is completed, the processor outputs the damage mask Y (or its visual overlay image) and text restoration suggestions R. sug The device may also include an image acquisition interface for connecting an external image acquisition device (e.g., a high-resolution scanner 10 with an optical resolution of at least 600 dpi or an industrial camera with macro capabilities) to acquire raw images. The result display interface 160 sends and drives the display of the results on a display device. in .

[0101] The device may also include a results display interface for connecting to an external display device (e.g., display screen 8) to present visual annotations of the detected damage locations and generated text repair suggestions. sug .

[0102] The book inner page damage detection and suggestion system described in this embodiment of the invention is implemented through a book inner page damage detection and suggestion device. The image preprocessing module 120, the damage detection module 130, and the repair suggestion generation module 140 are software functional modules stored in memory and executed by a processor. In other embodiments, these modules can also be implemented in hardware (e.g., dedicated logic circuits).

[0103] This invention also provides a computer-readable storage medium.

[0104] The computer-readable storage medium stores a computer program (instructions).

[0105] When this computer program is executed by the processor, it is used to achieve the aforementioned Figure 2 The complete steps of the recommended method for detecting soiling of book inner pages (i.e., steps S201 to S204) are shown.

[0106] The computer-readable storage medium can be any non-volatile or volatile storage medium, such as read-only memory (ROM), random access memory (RAM), hard disk drive (HDD), solid-state drive (SSD), or optical disc (CD-ROM).

Claims

1. A system for detecting and recommending repair of page damage in a book, the system comprising: The method comprises the following steps: An image preprocessing module is configured to perform color space conversion and brightness equalization on the obtained original image of the book inner page to output a standardized image to be detected; A damage detection module is connected to the image preprocessing module and configured to load a pre-trained deep convolutional neural network model to perform pixel-level semantic segmentation on the standardized image to be detected to output a damage mask identifying damage categories and positions; A repair suggestion generation module is connected to the damage detection module and configured to analyze the damage mask to determine the type, degree and position features of the damage area and generate a repair suggestion matching the features according to an internal rule-based decision tree.

2. The book interior page smudging detection repair recommendation system of claim 1, wherein, The image preprocessing module is specifically configured to: convert the original image from an RGB color space to a CIELAB color space to separate a brightness channel from a chroma channel.

3. The system for detecting and repairing suggestions for book page defacement according to claim 2, wherein, The image preprocessing module is further configured to: perform brightness equalization only on the brightness channel; recombine the equalized brightness channel with the chroma channel and convert back to the RGB color space to obtain the standardized image to be detected.

4. The system for detecting and suggesting repair of damage to a book's interior pages of claim 1, wherein, The deep convolutional neural network model in the damage detection module is trained using a combined loss function composed of a multi-class cross-entropy loss and a multi-class Dice loss.

5. The system for detecting and repairing suggestions for book page defacement of claim 1, wherein, The repair suggestion generation module is specifically configured to: perform post-processing on the damage mask to identify one or more independent damage areas.

6. The system for detecting and repairing recommendations for book page smudging of claim 5, wherein, The repair suggestion generation module is further configured to: extract a set of damage feature vectors for each damage area, the damage feature vectors including at least damage type, damage degree and damage position.

7. The system for detecting and repairing suggestions for book page defacement according to claim 5, wherein, The repair suggestion generation module is further configured to: input the damage feature vectors into the rule-based decision tree as inputs to match and output the repair suggestion A method for detecting and repairing damage on a book inner page, comprising the following steps: S201, image preprocessing, performing color space conversion and brightness equalization on the obtained original image of the book inner page to output a standardized image to be detected; S202, damage detection, loading a pre-trained deep convolutional neural network model to perform pixel-level semantic segmentation on the standardized image to be detected to output a damage mask identifying damage categories and positions; S203, damage feature analysis, analyzing the damage mask to determine the type, degree and position features of the damage area; S204, repair suggestion generation, generating a matching repair suggestion according to the features determined in the damage feature analysis step.

8. A device for detecting and suggesting repair of smudging of a book interior page, characterized by The method comprises the following steps: A processor, a memory and a computer program stored on the memory, wherein the computer program is executed by the processor to implement the method for detecting and repairing damage on a book inner page according to any one of claims 8.

9. A book interior page smudging detection repair suggestion storage medium, characterized by, A computer program is stored on the computer readable storage medium, wherein the computer program is executed by the processor to implement the method for detecting and repairing damage on a book inner page according to any one of claims 8.

Images