Calligraphy and painting digital asset generation method, system, device and medium

Abstract

The embodiment of the application discloses a calligraphy and painting digital asset generation method, system, device and medium. The method comprises the following steps: acquiring at least one candidate frame returned by a visual large model; performing boundary continuity verification and object category verification on each candidate frame to obtain a boundary continuity verification score and an object category verification score; calculating an audit priority score of each candidate frame according to the confidence, the boundary continuity verification score and the object category verification score corresponding to each candidate frame; dividing the candidate frames into multiple priority queues according to the audit priority scores; directly putting the candidate frames belonging to the automatic passing queue into an asset generation process, pushing the candidate frames belonging to the user audit queue to a client for user audit, and putting the candidate frames of the user audit queue into the asset generation process when the candidate frames of the user audit queue pass the audit. The method can reduce the workload of user audit candidate frames and improve the calligraphy and painting asset generation efficiency.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to methods, systems, equipment and media for generating digital assets of calligraphy and painting. Background Technology

[0002] Digital processing of paintings, calligraphy, documents, and cultural heritage images is an important foundational task in the field of cultural heritage protection. Existing digital processing technologies for paintings, calligraphy, documents, and cultural heritage images can be broadly categorized as follows.

[0003] The first category consists of automatic cropping, skew correction, and subject cropping techniques for scanned documents or ordinary images. These techniques typically extract the main content area from the scanned or captured image through edge detection, grayscale transformation, contour detection, target subject detection, and rectangular bounding box cropping. For example, a scanned document cropping system usually converts the scanned document into an image, performs grayscale conversion and edge detection, and then performs rotational correction and cropping based on the contour area. General subject cropping techniques crop an image to a suitable display size based on the location of the target object, its bounding rectangle, or a prominent area.

[0004] The second category is the identification and color correction technology of reference objects such as color charts, grayscale charts, and rulers. This type of technology usually first detects the color patches in the image, determines the position, spacing, and arrangement of the color patches, and then performs color correction, exposure evaluation, or white balance adjustment based on the difference between the captured color of the color patch and the standard color.

[0005] The third category includes techniques for the identification, cropping, restoration, and quality assessment of calligraphy works, paintings, or images of cultural heritage. For example, calligraphy image cropping and recognition techniques typically involve first locating the paper area, and then cropping and recognizing individual characters within the calligraphy work. Cultural heritage digitization standards emphasize controlling the quality of digitized images through indicators such as color, grayscale, sharpness, noise, and resolution, and often use color charts, grayscale charts, or other reference targets for quality assessment.

[0006] The fourth category is manual image editing or annotation software. This type of software allows users to manually drag and drop cropping frames, create annotation frames, adjust image boundaries, and export annotation results. However, it typically does not understand the semantic differences in painting cores, mountings, scrolls, titles, seals, color charts, etc., requiring users to determine the attributes of all areas themselves.

[0007] In recent years, with the rapid development of large-scale visual model technology, some studies have begun to explore the use of large-scale models for region recognition in calligraphy and painting images. Large-scale models can recognize various semantic regions such as color cards, painting cores, and seals, which has improved the level of intelligence in the digitization of calligraphy and paintings to a certain extent.

[0008] However, although large-scale visual models offer a new technological path for the digitization of calligraphy and painting, existing technologies still have the following major problems: 1. The candidate bounding boxes returned by the large model are not stable. The visual large model can identify areas such as color swatches, painting cores, and seals, but the returned bounding boxes may have problems such as missing frames, misaligned frames, over-cropping, and incorrect classification. For example, the color swatch is only partially bounded, the grayscale ruler is split, the lower boundary of the painting core is unclear from the mounting boundary, and the external seal is mistaken for the background.

[0009] 2. Traditional cropping techniques typically output only a single cropped image, lacking an awareness of the semantic assets of calligraphy and painting. In the images collected from calligraphy and painting collections, the mounting, inscriptions, collection labels, external seals, color charts, grayscale rulers, and scales outside the painting are not all useless backgrounds. On the one hand, these areas should not be included in the final image for appreciation; on the other hand, they may have value for color correction, archiving, measurement, or authentication.

[0010] 3. Manual review is labor-intensive. In batch digitization scenarios of calligraphy and paintings, if each large model candidate box requires manual confirmation by the user, the user needs to check each piece and each box, which is very time-consuming. Purely manual annotation is inefficient, while purely automatic recognition is prone to producing erroneous assets. Summary of the Invention

[0011] This application provides a method, system, device, and medium for generating digital assets of calligraphy and painting, which can solve the above-mentioned technical problems.

[0012] In a first aspect, embodiments of this application provide a method for generating digital assets of calligraphy and painting, which includes: The visual large model is used to perform region recognition on the original acquired images of calligraphy and painting. At least one candidate box is returned by the visual large model. Each candidate box contains region type, boundary coordinates and confidence score. Perform boundary continuity verification on each candidate box to obtain a boundary continuity verification score, and perform object category verification on each candidate box to obtain an object category verification score; The review priority score for each candidate box is calculated based on the confidence level, boundary continuity verification score, and object category verification score corresponding to each candidate box. The candidate boxes are divided into multiple priority queues based on the review priority score, and the multiple priority queues include at least an automatic approval queue and a user review queue. Candidate boxes belonging to the automatic approval queue are directly entered into the asset generation process, while candidate boxes belonging to the user review queue are pushed to the client for user review. When a candidate box in the user review queue passes the review, the asset generation process begins.

[0013] In some embodiments, performing boundary continuity verification on each candidate box to obtain a boundary continuity verification score includes: For each boundary of the candidate box, an inner pixel band is constructed inside the boundary and an outer pixel band is constructed outside the boundary. Calculate the color similarity, brightness difference, texture difference, and edge intensity between the inner pixel band and the outer pixel band; The boundary anomaly score of the boundary is calculated based on the color similarity, the brightness difference, the texture difference, and the edge intensity. The boundary continuity verification score of the candidate box is calculated based on the boundary anomaly score of each boundary.

[0014] In some embodiments, calculating the boundary anomaly score based on the color similarity, the brightness difference, the texture difference, and the edge intensity includes: The similarity between the inner and outer edges of the boundary is determined based on the color similarity, the brightness difference, and the texture difference. The boundary anomaly score is determined based on the internal and external similarity and the edge strength.

[0015] In some embodiments, performing object category validation on each candidate box to obtain an object category validation score includes: The object image is cropped from the candidate box; Extract the visual features of the object image; The visual features are compared with the target category samples in the category prototype library to obtain the object category verification score. The category prototype library stores multiple category samples of different object types, and the target category sample is the category sample in the category prototype library that corresponds to the region type.

[0016] In some embodiments, calculating the review priority score for each candidate box based on the confidence level, the boundary continuity verification score, and the object category verification score corresponding to each candidate box includes: Determine the semantic conflict score between each candidate box and other candidate boxes in the original captured image of the calligraphy and painting; The object value of each candidate box is determined based on the pre-defined correspondence between object value and region type, and the region type of each candidate box. For each candidate box, a weighted fusion process is performed on the confidence score, the boundary continuity verification score, the object category verification score, the semantic conflict score, and the object value to obtain the review priority score for each candidate box.

[0017] In some embodiments, the user review queue includes a low-priority queue, a medium-priority queue, and a high-priority queue. Candidate boxes in the low-priority queue and the medium-priority queue carry a random inspection flag, and the high-priority queue carries a mandatory inspection flag. After pushing candidate boxes belonging to the user review queue to the client for user review, the method further includes: Obtain a batch retry instruction sent by the user through the client for multiple specified candidate boxes, wherein the batch retry instruction carries an error reason description corresponding to each specified candidate box; In response to the batch retry instruction, candidate boxes in the high-priority queue of the original calligraphy and painting images are regenerated using the visual big model to obtain multiple new candidate boxes, each carrying a regenerated label. The multiple new candidate boxes are pushed to the client for user review.

[0018] In some embodiments, the asset generation process includes the following steps: The original captured images of calligraphy and painting are cropped based on the boundary coordinates of the finally confirmed candidate boxes to obtain cropped images corresponding to each candidate box. Based on the cropped image corresponding to each candidate box and the region type and boundary coordinates corresponding to each candidate box, digital assets of calligraphy and painting corresponding to the original acquired image of calligraphy and painting are generated. The region type includes painting core, mounting, background, color card, gray scale ruler, scale ruler, seal, inscription, title label or collection label.

[0019] Secondly, embodiments of this application also provide a system for generating digital assets of calligraphy and painting, which includes: The candidate box acquisition unit is used to acquire at least one candidate box returned by the visual large model after performing region recognition on the original acquired image of calligraphy and painting. Each candidate box includes region type, boundary coordinates and confidence score. The verification unit is used to perform boundary continuity verification on each candidate box to obtain a boundary continuity verification score, and to perform object category verification on each candidate box to obtain an object category verification score. The calculation unit is used to calculate the review priority score of each candidate box based on the confidence level, the boundary continuity verification score and the object category verification score corresponding to each candidate box. A queue partitioning unit is used to divide the candidate box into multiple priority queues according to the review priority score, wherein the multiple priority queues include at least an automatic approval queue and a user review queue. The asset generation unit is used to directly enter the asset generation process for candidate boxes belonging to the automatic pass queue, and to push candidate boxes belonging to the user review queue to the client for user review. When the candidate box in the user review queue passes the review, it enters the asset generation process.

[0020] Thirdly, embodiments of this application also provide a computer device, which includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the above-described method.

[0021] Fourthly, embodiments of this application also provide a computer-readable storage medium storing a computer program, the computer program including program instructions that, when executed by a processor, can implement the above-described method.

[0022] This application provides a method, system, device, and medium for generating digital assets of calligraphy and painting. Firstly, this application uses a large visual model to identify multiple candidate boxes of different region types in the original acquired images of calligraphy and painting that require region cropping, thus improving the richness of the digital assets and increasing the efficiency of digital asset generation compared to full manual annotation. Secondly, this application automatically scores each candidate box output by the large visual model (i.e., calculates a review priority score), and determines whether to push the candidate box to the user for review based on the score, eliminating the need for full manual review of candidate boxes. Therefore, this application can reduce the workload of manual review of candidate boxes while ensuring the quality of candidate box generation. Attached Figure Description

[0023] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 A flowchart illustrating the method for generating digital assets of calligraphy and painting provided in this application embodiment; Figure 2 A schematic diagram of a sub-process of the method for generating digital assets of calligraphy and painting provided in an embodiment of this application; Figure 3 This is a schematic diagram of another sub-process of the method for generating digital assets of calligraphy and painting provided in the embodiments of this application; Figure 4 This is a schematic diagram of another sub-process of the method for generating digital assets of calligraphy and painting provided in the embodiments of this application; Figure 5This is a schematic diagram of another sub-process of the method for generating digital assets of calligraphy and painting provided in the embodiments of this application; Figure 6 A schematic block diagram of a system for generating digital assets of calligraphy and painting provided in an embodiment of this application; Figure 7 A schematic block diagram of a computer device provided in an embodiment of this application. Detailed Implementation

[0025] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0026] It should be noted that any AI models, software tools, or components not belonging to this company appearing in the embodiments of this application are merely illustrative examples and do not represent actual use. All user personal information involved in the embodiments of this application has been obtained by authorized entities (who have known and consented) or fully authorized by all parties through various legal and compliant means. The collection, storage, use, processing, transmission, provision, and disclosure of the information, data, and signals involved all comply with relevant laws and regulations and do not violate public order and good morals.

[0027] It should be understood that, when used in this specification and the appended claims, the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0028] It should also be understood that the terminology used in this application specification is for the purpose of describing particular embodiments only and is not intended to limit the application. As used in this application specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0029] It should also be further understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0030] This application provides a method, system, device, and medium for generating digital assets of calligraphy and painting.

[0031] The execution entity of the method for generating digital assets of calligraphy and painting can be the digital asset generation system of calligraphy and painting provided in the embodiments of this application, or a computer device that integrates the digital asset generation system of calligraphy and painting. The digital asset generation system of calligraphy and painting (hereinafter referred to as the system) can be implemented in hardware or software. The computer device can be a terminal or a server. The terminal can be a smartphone, tablet computer, handheld computer or laptop computer, etc.

[0032] Figure 1 This is a flowchart illustrating the method for generating digital assets of calligraphy and painting provided in this application. Figure 1 As shown, the method includes the following steps S110-S150.

[0033] S110. Obtain at least one candidate box returned by the visual large model after performing region recognition on the original acquired image of calligraphy and painting, each candidate box containing region type, boundary coordinates and confidence level.

[0034] In this embodiment, the user first uploads the original captured images of the calligraphy and painting through the client. The original captured images refer to the original images obtained by digitally capturing the calligraphy and painting works, such as images obtained by taking pictures of the calligraphy and painting works using a high-precision scanner or camera. The original captured images typically contain various semantic regions of different area types, such as the painting core, mounting, background, color chart, grayscale ruler, scale, seal, inscription, title label, and collection label.

[0035] After the client uploads the original captured images of the calligraphy and painting to the backend server, the server saves the original captured images and creates a processing task. The server then calls a large visual model to perform region recognition on the original captured images of the calligraphy and painting. The large visual model can be an object detection model or a segmentation model based on the Transformer architecture.

[0036] Specifically, the visual big data model can perform region recognition on the original captured images of calligraphy and paintings based on preset prompts. For example, the prompt could be: "You are the visual analysis model in the calligraphy and painting digitization image processing system. Please analyze the original captured images of calligraphy and paintings uploaded by users and identify candidate regions that can be used for cropping, removal, storage, review, or training data generation." In some instances, prompts can be versioned. Firstly, users can manually modify prompts and save versions; secondly, updates can be rule-based or semi-automatic based on category prototype libraries, asset libraries, and historical audit results in the database. For example: INSERT INTO prompt_versions SELECT New prompt word versions generated from the current category prototype library, asset statistics, and historical audit errors.

[0037] After analyzing the image, the large model returns at least one candidate bounding box. Each candidate bounding box contains the following information: Region type: Indicates the object category corresponding to the candidate box as determined by the large model, such as painting core, mounting, background, color card, grayscale ruler, scale ruler, seal, inscription, title label or collection label. Different candidate boxes correspond to different region types.

[0038] Boundary coordinates: Represent the position of the candidate box in the original image, usually expressed as [x...]. min, y min , x max , y max The format is ], where the origin is the top left corner of the image and the coordinates are integer pixels.

[0039] Confidence: This indicates the degree of certainty that the visual model is in recognizing the candidate box. It is usually a value between 0 and 1, with a higher value indicating that the model is more confident.

[0040] S120. Perform boundary continuity verification on each candidate box to obtain a boundary continuity verification score, and perform object category verification on each candidate box to obtain an object category verification score.

[0041] The boundary continuity verification score is used to indicate whether the boundary of the candidate box is located on the edge of the real object, and the object category verification score is used to indicate whether the type of the real object in the candidate box is consistent with the region type.

[0042] This embodiment requires performing boundary continuity verification and object category verification on each candidate box obtained in step S110. The purpose of the boundary continuity verification is to determine whether the boundary of the candidate box is located on the edge of the real object, or cut inside the object or in the middle of the background. The purpose of the object category verification is to determine whether the object within the candidate box is consistent with the region type declared in the large model.

[0043] For boundary continuity verification: For each boundary of the candidate bounding box (top, bottom, left, and right), the system extracts pixel bands on both the inside and outside of the boundary and compares the differences in image features between the inner and outer pixel bands. If the inner and outer pixel bands of a certain boundary are very similar and the edge intensity is low, it indicates that the boundary may not fall on the edge of the real object, and the candidate bounding box may have omissions, be off-center, or not completely encompass the object.

[0044] The system calculates the boundary continuity verification score of the candidate box based on the verification results of the four boundaries. This score indicates whether the boundary of the candidate box lies on the edge of the real object. The lower the score, the more reliable the boundary.

[0045] In some embodiments, please refer to Figure 2 The system can perform boundary continuity verification through the following steps: a1201. For each boundary of the candidate box, an inner pixel band is constructed inside the boundary and an outer pixel band is constructed outside the boundary.

[0046] Specifically, for each boundary (top, bottom, left, and right boundary) of the candidate box, the system constructs pixel bands on the inside and outside of the boundary, respectively.

[0047] Taking the right boundary as an example, assume the boundary coordinates of the candidate box are [x min , y min , x max , y max The preset sampling band width is w (e.g., w = 10 pixels): The inner pixel band is: [x max - w, y min , x max , y max ], which is a rectangular strip with a width of w that is close to the inside of the right frame.

[0048] The outer pixel band is: [x max , y min, x max +w, y max ], which is a rectangular strip with a width of w that is right next to the outside of the right frame.

[0049] Similarly, for the other three boundaries: Left boundary: inner pixel band is [x min , y min , x min +w, y max The outer pixel band is [x] min - w, y min ,x min , y max ].

[0050] Upper boundary: the inner pixel band is [x min , y min , x max , y min +w], the outer pixel band is [x min , y min - w,x max , y min ].

[0051] Lower boundary: the inner pixel band is [x min , y max - w, x max, y max The outer pixel band is [x] min , y max ,x max , y max +w].

[0052] It should be noted that the sampling bandwidth w can be adjusted according to the image resolution. For high-resolution images, a larger w value (e.g., 20 pixels) can be used; for low-resolution images, a smaller w value (e.g., 5 pixels) can be used. This parameter can be preset by the system or configured by the user according to the actual scenario.

[0053] For example: Suppose the candidate bounding box has bbox = [77, 87, 371, 526], and w = 10: The inner pixel band of the right boundary is [361, 87, 371, 526] (x from 361 to 371), and the outer pixel band is [371, 87, 381, 526] (x from 371 to 381).

[0054] a1202. Calculate the color similarity, brightness difference, texture difference, and edge intensity between the inner pixel band and the outer pixel band.

[0055] Specifically, the system extracts features from the inner and outer pixel bands respectively and calculates the differences between them.

[0056] Color similarity calculation: The system extracts color features from the inner and outer pixel bands respectively. In one implementation, pixels are converted from RGB space to HSV space, generating color histograms (e.g., dividing hue (H) and saturation (S) into 32×32 intervals). The similarity between the two histograms is then calculated. Methods such as histogram intersection, Bach distance, or chi-square distance can be used. Higher similarity indicates a closer similarity in the color composition of the inner and outer pixel bands.

[0057] Brightness difference calculation: The system extracts the brightness component values ​​of each pixel in the inner and outer pixel bands. In one implementation, the RGB values ​​are weighted and summed according to preset weights to obtain a grayscale value as the brightness component value. Then, the inner brightness statistics (such as mean and standard deviation) and the outer brightness statistics are calculated, and the brightness difference is determined based on the difference between the two. The larger the difference, the more significant the brightness difference between the inner and outer pixel bands.

[0058] As an alternative, the luminance component value can also be obtained by converting the pixel from RGB space to Lab color space (L channel value) or from HSV color space (V channel value). These alternatives can also achieve the purpose of this application.

[0059] Texture difference calculation: The system extracts texture features from the inner and outer pixel bands. In one implementation, Local Binary Pattern (LBP) is used to extract texture histograms, and the distance between the two histograms is calculated. In another implementation, Gray-Level Co-occurrence Matrix (GLCM) is used to extract texture features (such as contrast, correlation, energy, homogeneity, etc.), and the distance between feature vectors is calculated. The greater the texture difference, the more different the texture structures of the inner and outer pixel bands are.

[0060] Edge strength calculation: The system detects edge strength on the candidate box boundaries. Specifically, gradient operators (such as the Sobel operator, Prewitt operator, or Canny edge detection operator) are used to calculate the gradient magnitude of each pixel on the boundary. Then, the edge strength of the boundary is determined based on the gradient magnitudes of all pixels on the boundary.

[0061] In one implementation, the average gradient magnitude of all pixels on the boundary can be taken as the edge strength.

[0062] For example, for the right boundary, the system calculates a color similarity of 0.92 (very similar), a brightness difference of 8 (very small), a texture similarity of 0.88 (very similar), and an edge strength of 15 (low) between the inner and outer pixel bands on the right. These metrics combined indicate that the inner and outer sides of the boundary are visually highly consistent and may not be the edge of a real object.

[0063] a1203. Calculate the boundary anomaly score of the boundary based on the color similarity, the brightness difference, the texture difference, and the edge intensity.

[0064] Specifically, the system integrates color similarity, brightness difference, texture difference, and edge strength to calculate the boundary anomaly score.

[0065] First, the system determines the internal and external similarity of the boundary based on color similarity, brightness difference, and texture difference. Specifically, the inverse normalized values ​​of color similarity, brightness difference, and texture difference are weighted and fused. The higher the internal and external similarity, the more the inside and outside of the boundary resemble the same region.

[0066] Then, the system determines the boundary anomaly score based on the internal and external similarity and the edge strength. The basic logic of the boundary anomaly score is: the higher the internal and external similarity and the lower the edge strength, the higher the boundary anomaly score; conversely, the lower the internal and external similarity or the higher the edge strength, the lower the boundary anomaly score.

[0067] In one implementation, the edge error score is calculated using the following formula: edge_error_score = inner_outer_similarity×low_edge_strength_factor Here, inner_outer_similarity represents the inner-outer similarity, and low_edge_strength_factor represents the inverse normalized value of the edge strength (i.e., 1 minus the normalized edge strength). The value of edge_error_score ranges from 0 to 1, with higher values ​​indicating a more suspicious boundary.

[0068] For example, if a boundary has inner_outer_similarity = 0.85 and low_edge_strength_factor = 0.8, then edge_error_score = 0.85 × 0.8 = 0.68, which is a high anomaly score. This boundary is very likely to have cut inside the object or in the middle of the background.

[0069] If a boundary has an inner_outer_similarity of 0.15 (large difference between inside and outside) and a low_edge_strength_factor of 0.1 (very hard edge), then edge_error_score = 0.15 × 0.1 = 0.015, which is a low anomaly score. This boundary is very likely to fall accurately on the edge of the real object.

[0070] a1204. Calculate the boundary continuity verification score of the candidate box based on the boundary anomaly score of each boundary.

[0071] Specifically, after obtaining the boundary anomaly scores of each of the four edges of the candidate box, the system comprehensively calculates the boundary continuity verification score of the candidate box.

[0072] In one implementation, the maximum value of the boundary anomaly scores among the four edges is taken as the boundary continuity verification score of the candidate box.

[0073] In another implementation, the average boundary anomaly score of the four edges is taken as the boundary continuity verification score, or the number of edges with anomaly scores higher than a preset threshold is taken as the scoring basis.

[0074] The boundary continuity check score ranges from 0 to 1. The lower the score, the more reliable the four boundaries of the candidate box are, and the higher the score, the greater the possibility of boundary anomalies.

[0075] For example, if a candidate box has anomaly scores of [0.02, 0.05, 0.68, 0.03] for its four edges (the lower boundary is clearly abnormal), and the maximum value is taken, the boundary continuity verification score is 0.68. This candidate box will be marked as requiring user attention.

[0076] This application's embodiments integrate four dimensions: color, brightness, texture, and edge intensity, avoiding misjudgments that may result from a single feature. For example, relying solely on color may not be able to distinguish areas with different textures but similar colors, while combining texture features can solve this problem.

[0077] Object category validation: The system crops the object image from the candidate box, extracts the visual features of the image (such as color features, texture features, shape features, or depth features), and then compares the similarity of the features with high-quality samples of the corresponding type in the category prototype library.

[0078] The system calculates an object category validation score for the candidate box based on the comparison results. This score indicates whether the object within the candidate box matches its declared category. The higher the score, the more closely the object within the box matches the declared category.

[0079] In some embodiments, please refer to Figure 3 The system can perform object category validation through the following steps: b1201. Crops the object image from the candidate box.

[0080] Specifically, the system crops the corresponding object image from the original captured image of the calligraphy and painting based on the boundary coordinates of the candidate bounding boxes. This cropping operation is the same as regular image cropping, that is, based on [x... min , y min , x max , y max Extract the corresponding rectangular region from the original image.

[0081] The cropped object image retains the color and resolution information of the original image, which is used for subsequent feature extraction.

[0082] For example, for a candidate box {type: "color_card", bbox: [400, 100, 480, 300]}, the system crops a rectangular region from the original image with x ranging from 400 to 480 and y ranging from 100 to 300 to obtain a partial image of the color card.

[0083] b1202. Extract the visual features of the object image.

[0084] Specifically, the system extracts features from the cropped object image. The visual features are mathematical vectors that can characterize the content attributes of the object, and may include one or more of the following: color features, texture features, shape features, and depth features.

[0085] In one implementation, the system concatenates the above features into a comprehensive feature vector for subsequent similarity comparison.

[0086] b1203. Compare the visual features with the target category samples in the category prototype library to obtain the object category verification score. The category prototype library stores multiple category samples of different object types, and the target category sample is the category sample in the category prototype library that corresponds to the region type.

[0087] Specifically, the system compares the extracted visual features with the target category samples in the category prototype library for similarity.

[0088] The category prototype library is a pre-built and continuously updated database in the system backend, storing high-quality category samples of multiple different object types. Each sample contains the visual feature vector of the object image and its corresponding category label. Categories that can be stored in the category prototype library include, but are not limited to: painting core, mounting, background, color card, grayscale ruler, scale ruleer, internal seal, external seal, inscription, label, collection label, restoration label, etc.

[0089] The target category sample refers to the category sample in the category prototype library that corresponds to the region type of the current candidate box. For example, if the region type of the candidate box is "color_card", the system retrieves all samples of the "color_card" category from the category prototype library for comparison.

[0090] The system calculates the similarity (such as cosine similarity, inverse normalization of Euclidean distance, etc.) between the feature vector of the candidate bounding box object image and the feature vector of the target category sample, obtaining an object category verification score. The higher the score, the better the object in the candidate bounding box matches that category.

[0091] In one implementation, the system can compare not only with the target category but also with all categories in a category prototype library to discover if there are other, more closely matching categories. If a candidate object shows higher similarity to other categories, it indicates a potential error in the category declaration of the larger model, and the system can generate a class suggestion.

[0092] At this point, when the similarity between the object within the candidate box and the declared category (i.e., the object category specified by the corresponding region type) is higher than the first threshold, it is determined to be a category match; when the similarity between the object within the candidate box and the declared category is lower than the second threshold but the similarity with other categories is higher than the third threshold, a class suggestion is generated; when the similarity between the object within the candidate box and all categories is lower than the fourth threshold, the candidate box is marked as a new category candidate. Wherein, the second threshold is less than the first threshold, the third threshold is greater than or equal to the first threshold, and the fourth threshold is less than the first threshold.

[0093] If the similarity of a candidate object to all categories is below a preset threshold, it indicates that the object may belong to a new category that has not yet been included. The system can then mark it as a candidate for a new category and leave it to the user to judge.

[0094] For example, suppose the large model returns a candidate bounding box with type: "internal_seal". The system crops the image within the bounding box, extracts depth features, and compares the similarity with samples of "internal_seal" in the category prototype library. The similarity score is 0.91, which is much higher than the preset matching threshold of 0.7. The object category verification score is 0.91, indicating that the object within the bounding box highly matches the seal category, and the category claim is reliable.

[0095] Conversely, if the similarity is only 0.32, which is far below the threshold, and the similarity with the "background" category reaches 0.85, it indicates that the category declaration of the large model is incorrect, and the object inside the box may actually be the background. The system will generate a class suggestion.

[0096] S130. Calculate the review priority score for each candidate box based on the confidence level, boundary continuity verification score, and object category verification score corresponding to each candidate box.

[0097] In this embodiment, the review priority score is a quantitative indicator used to measure the urgency of a candidate box requiring user review. The higher the score, the more worthy the user's attention is, and the more it should be pushed to the user for review; the lower the score, the more credible the candidate box is, and it can automatically enter the subsequent process.

[0098] In some embodiments, please refer to Figure 4 The system can calculate the review priority score for each candidate box through the following steps: S1301. Determine the semantic conflict score between each candidate box and other candidate boxes in the original image of the calligraphy and painting.

[0099] Semantic conflict refers to a contradiction between the region type of a candidate box and the common sense in the field of calligraphy and painting that should be followed by its spatial location in the image.

[0100] It should be noted that the system has pre-defined scores for when a semantic conflict is triggered and scores for when it is not. For example, the score for when a semantic conflict is triggered is 1, and the score for when it is not triggered is 0.

[0101] Specifically, the semantic conflict score includes at least one of the following: A semantic conflict is triggered when the coordinates of the candidate boxes for color cards overlap or contain the coordinates of the candidate boxes for painting cores. A semantic conflict is triggered when the coordinates of the background candidate box overlap or contain the coordinates of the core candidate box. A semantic conflict is triggered when the coordinates of the outer seal-type candidate box are located inside the painting core-type candidate box.

[0102] S1302. Determine the object value of each candidate box based on the preset correspondence between object value and region type, and the region type of each candidate box.

[0103] In this embodiment, the system determines the object archive value (object value) for each candidate box based on a preset mapping table. The mapping table records the archive value scores corresponding to different region types.

[0104] In the context of digitizing calligraphy and paintings, different types of areas possess different archival values. For example: Painting core: The file has high value, but large models usually identify it more accurately. The value score can be set to 0.3 (medium).

[0105] Internal seal and inscription: These have extremely high value for art historical research and are worth confirming by users even with a high degree of confidence. The value score is set at 0.9 (extremely high).

[0106] Collection labels and external seals: have authentication and traceability value, with a value score of 0.8 (high).

[0107] Color card and grayscale: have color calibration value, but are not artistic in nature, and their value score is set to 0.2 (low).

[0108] Background: Usually has no file value, value rating is set to 0.0 (no value).

[0109] The introduction of object value enables the system to identify high-value objects, which will be given a higher review priority even if their physical boundaries are clear and their categories match, thereby ensuring that important cultural relic information will not be missed in the automatic processing flow.

[0110] S1303. For each candidate box, a weighted fusion process is performed on the confidence level, the boundary continuity verification score, the object category verification score, the semantic conflict score, and the object value to obtain the review priority score for each candidate box.

[0111] In one implementation, the formula for calculating the review priority score is: review_score=a×uncertainty_score+b×boundary_score+c× low_similarity_score+d×semantic_conflict_score+e×high_asset_value_score; Here, uncertainty_score represents the uncertainty score of the model, uncertainty_score = 1 - confidence, where confidence is the confidence level; boundary_score represents the boundary continuity verification score, indicating the degree of abnormality of the clipping boundary; low_similarity_score = 1 - similarity_score, similarity_score is the object category verification score, semantic_conflict_score is the semantic conflict score, and high_asset_value_score is the object value.

[0112] Among them, a, b, c, d, and e are preset weighting coefficients that can be adjusted according to actual business needs.

[0113] For example, in a scenario where boundary accuracy is important (to avoid missing frames or over-cropping), b can be set to a larger value; in a scenario where high-value objects are important (to avoid missing seals or inscriptions), e can be set to a larger value.

[0114] S140. Divide the candidate box into multiple priority queues according to the review priority score. The multiple priority queues include at least an automatic approval queue and a user review queue.

[0115] In this embodiment, candidate boxes in the queue are automatically passed. If the review priority score is lower than the preset first queue threshold, it indicates that the candidate box is highly reliable. Candidate boxes belonging to this queue can directly enter the asset generation process without user review.

[0116] Candidate boxes in the user review queue refer to those with a review priority score higher than or equal to a preset first queue threshold, indicating that the candidate box has a certain degree of uncertainty or high value. Candidate boxes belonging to this queue need to be pushed to the client for user review.

[0117] Optionally, the user review queue can be further subdivided into low-priority queues, medium-priority queues, and high-priority queues for hierarchical management of candidate boxes that require review.

[0118] In some embodiments, the user review queue includes a low-priority queue, a medium-priority queue, and a high-priority queue, wherein the review priority score of the candidate box in the low-priority queue is lower than that of the candidate box in the medium-priority queue, the review priority score of the candidate box in the medium-priority queue is lower than that of the candidate box in the high-priority queue, the candidate boxes in the low-priority queue and the medium-priority queue carry a random inspection label, and the candidate boxes in the high-priority queue carry a mandatory inspection label.

[0119] Among them, the candidate boxes in the low-priority queue are generally reliable, but there is a slight uncertainty. The candidate boxes in this queue carry a sampling inspection flag. The system can randomly select a portion of the boxes according to a preset ratio (such as 10%) and push them to the user for review. The remaining boxes can be automatically released or simply recorded, or all boxes can be pushed to the user but carry a sampling inspection flag, and the user can decide whether to conduct a sampling inspection and the number of boxes to be inspected. Candidate boxes in the medium-priority queue may have some boundary deviation or category uncertainty, but the system can correct them with a high probability through automatic correction (such as expansion, contraction, or edge snapping). Candidate boxes in this queue also carry a sampling check flag (if automatic correction has been performed, an automatic correction flag must also be carried). The system can randomly select a portion of the boxes according to a preset ratio (such as 20%) and push them to the user for review. The remaining boxes can be automatically released, simply recorded, or pushed to the user in full, but with a sampling check flag, allowing the user to decide whether to conduct a sampling check and the number of boxes to be checked.

[0120] Candidate boxes in the high-priority queue contain serious anomalies, such as high boundary anomaly scores, category mismatches, semantic conflicts, or extremely high object value. These candidate boxes carry a mandatory inspection label, and the system forcibly pushes them to the user for review; the user must process each box individually.

[0121] S150. Candidate boxes belonging to the automatic pass queue are directly entered into the asset generation process, and candidate boxes belonging to the user review queue are pushed to the client for user review. When the candidate boxes in the user review queue are approved, the asset generation process begins.

[0122] In this example, candidate boxes assigned to the automatic approval queue are directly sent to the asset generation process. The asset generation process includes cropping the original image based on the boundary coordinates of the candidate boxes and generating corresponding asset types (such as core images, color charts, etc.) based on the region type. This step achieves the effect of "the vast majority of trustworthy candidate boxes automatically passing," requiring no user intervention on the trustworthy candidate boxes and significantly reducing the workload of manual review.

[0123] In some embodiments, candidate boxes from the automatic pass queue and the user review queue, as well as the original captured images of calligraphy and painting, need to be sent to the client. Users can select the queue to be displayed through the client. For example, if the client needs to review candidate boxes, they can click to display the user review queue (specifically, they can choose to display one or more of the low-priority queue, medium-priority queue, and high-priority queue). When the entire candidate box needs to be displayed, they can click to display all queues.

[0124] For candidate boxes that are assigned to the user review queue, the system pushes their information to the client. The client displays the original image on the interface, with these candidate boxes overlaid on top of the image. Users can click on a candidate box to view a magnified preview of its contents and determine whether the box is correct.

[0125] User review of candidate boxes can include: Confirmation passed: The candidate box is correct, and the system will send it to the asset generation process.

[0126] Drag and drop adjustment: The candidate box is basically correct but the boundary is off. The user can drag and adjust the boundary coordinates.

[0127] Modify Category: The object in the candidate box is correct, but the area type is incorrect. The user modifies its category.

[0128] Add to retry: The candidate box has a serious error and the user has marked it as to be retried, so that it can be re-identified by the large model.

[0129] Invalid tag: The candidate box is completely wrong and the user has deleted it.

[0130] Users who confirm their selection (including after dragging and adjusting or modifying the category) will proceed to the asset generation process.

[0131] In some embodiments, after pushing the candidate boxes belonging to the user review queue to the client for user review, the method further includes step AC: A. Obtain batch retry instructions sent by the user through the client for multiple specified candidate boxes, wherein the batch retry instructions carry an error reason description corresponding to each specified candidate box.

[0132] Specifically, when a user reviews candidate boxes in a high-priority queue, if multiple candidate boxes have obvious recognition errors (such as misaligned boxes, missing boxes, incorrect categories, etc.), the user does not need to manually drag and drop them one by one to correct them. The user can select multiple candidate boxes in the client interface and choose the "Add to Retry" option.

[0133] The client displays a retry reason form, where the user selects or enters a description of the error reason for each specified candidate box. Error reasons may include, but are not limited to: The box is too small and does not completely encompass the target object; The frame is too large, containing too many irrelevant areas; The position of the box is offset from the target object; Category error; the object inside the box does not match the declared category. The target object was not identified (missing frame); Other custom reasons.

[0134] After user confirmation, the client generates a batch retry command and sends it to the server. This command includes a description of the error reason for each specified candidate box.

[0135] B. In response to the batch retry instruction, the candidate boxes of the high-priority queue in the original acquired image of the calligraphy and painting are regenerated through the visual big model to obtain multiple new candidate boxes, and the new candidate boxes carry regenerated labels.

[0136] After receiving the batch retry instruction, the server does not call the visual big model separately for each candidate box, but merges all candidate boxes to be retried into one or a few retry requests to reduce token consumption and the number of interface calls.

[0137] In one implementation, the retry request is constructed as follows: the entire original image, the local image near each anomaly candidate box, the original boundary coordinates and category of each anomaly candidate box, the boundary anomaly description, the category similarity description, and the supplementary prompts input by the user are combined into a structured request and sent to the large visual model.

[0138] The large model reanalyzes the region based on the above information, returning corrected boundary coordinates and / or categories. The system marks these new candidate boxes as regenerated and saves the retry record.

[0139] For example, a user finds four problematic bounding boxes in an image: two color swatch boxes only partially enclose the target object, one painting core box has its bottom right corner cut off, and one stamp box is incorrectly marked as background. The user selects these four boxes on the client side and adds a comment: "These boxes do not completely enclose the target object; please try again." The server sends these four problematic boxes, along with the original image, a partial image, and the prepared error message, to the larger model. A single request corrects all four boxes.

[0140] C. Push the multiple new candidate boxes to the client for user review.

[0141] Specifically, after the large visual model returns the revised candidate boxes, the system pushes them to the client. The new candidate boxes carry regeneration labels, making it easy for users to distinguish on the interface which are the original candidate boxes and which are regenerated after retries.

[0142] Users can review the regenerated candidate boxes again—if everything is correct, click "Approve" to proceed with asset generation; if there are still problems, they can continue to adjust or retry.

[0143] This embodiment merges multiple exception candidate boxes into a single large model retry request. Compared to calling each exception box separately, this significantly reduces token consumption and the number of API calls, thereby lowering system operating costs.

[0144] It should be noted that the batch retries provided in this application differ from ordinary retries. In this application, the user can select multiple exception boxes at once. Instead of calling the large model one by one, the system submits the entire image, the local image, the original candidate boxes (bboxes), the anomaly detection results, and the error reasons selected by the user to the large model for retries, thereby reducing the number of calls and token consumption. For example: INSERT INTO retry_batch SELECT Full image, multiple partial images, original bounding box set, anomaly detection results, and user error reason set.

[0145] In some embodiments, such as Figure 5 As shown, the asset generation process includes the following steps: S1501. Based on the boundary coordinates of the finally confirmed candidate boxes, the original acquired images of calligraphy and painting are cropped to obtain cropped images corresponding to each candidate box.

[0146] After the system obtains the finally confirmed candidate boxes (including candidate boxes that are automatically passed through the queue and candidate boxes that have been approved by the user), it crops the original acquired image according to the boundary coordinates of each candidate box.

[0147] The final confirmed candidate boxes include: Automatically pass through the original candidate boxes in the queue (without user modification); User review and approval or drag and drop the adjusted candidate box; After batch retries, candidate boxes are regenerated and confirmed to be successful.

[0148] The cropping operation is based on the [x] of each candidate box. min , y min , x max , y max The coordinates are used to extract the corresponding rectangular region from the original image to obtain the cropped image.

[0149] S1502. Based on the cropped image corresponding to each candidate box and the region type and boundary coordinates corresponding to each candidate box, generate the digital asset of the calligraphy and painting corresponding to the original acquired image of the calligraphy and painting. The region type includes painting core, mounting, background, color card, gray scale ruler, scale, seal, inscription, title label or collection label.

[0150] The system categorizes cropped images into different asset types based on the region type of each candidate box, and the boundary coordinates are used to indicate the position of the corresponding asset in the original captured image of the calligraphy and painting.

[0151] The asset types include, but are not limited to: painting core images, painting core appreciation images (based on painting core images that have undergone further processing (such as removing color charts, rulers, and other auxiliary materials) to obtain a clean version suitable for appreciation and display), mounting archive images, color chart images, grayscale ruler images, ruler images, seal images, and title label images. In one implementation, the system can further generate a reference object mask and a training image without reference objects. The reference object mask marks the pixel areas of reference objects such as color charts and grayscale rulers, making it easy to mask them in the training data. The training image without reference objects is a clean image of the calligraphy or painting obtained by removing auxiliary objects such as color charts from the original image, which can be used to train the visual model's appreciation ability.

[0152] This embodiment enables the independent output of multiple asset types. Instead of simply outputting a single cropped image, it generates various asset files based on semantic regions to meet different purposes such as appreciation and display, archiving, color correction, and scale measurement.

[0153] In one implementation, the method further includes a category prototype library update step: obtaining the user's review and confirmation result for the candidate box, wherein the review and confirmation result includes confirmation passed, category modified, new category added, or marked as invalid; storing the objects in the candidate boxes that the user has confirmed as passed as positive samples into the category prototype library; storing the objects in the candidate boxes after the user has modified the category into the category prototype library according to the modified category; and storing the objects in the candidate boxes with newly added categories as new category samples into the category prototype library.

[0154] In one implementation, the category prototype library can support users importing or adding categories, and performing incremental updates in the database. For example: UPSERT category_library SET category='New Category', sample=User Confirmed Tile, feature=Tile Feature, version=version+1.

[0155] In summary, firstly, this application, through a large visual model, can identify multiple candidate boxes of different region types in the original acquired images of calligraphy and paintings that require region cropping, thereby increasing the richness of digital assets of calligraphy and paintings. Furthermore, compared to full manual annotation, it improves the efficiency of generating digital assets of calligraphy and paintings. Secondly, this application automatically scores each candidate box output by the large visual model (i.e., calculates the review priority score), and determines whether to push the candidate box to the user for review based on the score, eliminating the need for full manual review of candidate boxes. Therefore, this application can reduce the workload of manual review of candidate boxes while ensuring the quality of candidate box generation.

[0156] Figure 6 This is a schematic block diagram of a system for generating digital assets of calligraphy and painting provided in an embodiment of this application. Figure 6 As shown, corresponding to the above-described method for generating digital assets of calligraphy and painting, this application also provides a system 800 for generating digital assets of calligraphy and painting. This system 800 includes a unit for executing the above-described method for generating digital assets of calligraphy and painting, and can be configured in terminals such as desktop computers, tablet computers, and laptops. For details, please refer to... Figure 6 The digital asset generation system for calligraphy and painting 800 includes a candidate box acquisition unit 801, a verification unit 802, a calculation unit 803, a queue division unit 804, and an asset generation unit 805.

[0157] The candidate box acquisition unit 801 is used to acquire at least one candidate box returned by the visual large model after performing region recognition on the original acquired image of calligraphy and painting. Each candidate box includes region type, boundary coordinates and confidence score. The verification unit 802 is used to perform boundary continuity verification on each candidate box to obtain a boundary continuity verification score, and to perform object category verification on each candidate box to obtain an object category verification score. The calculation unit 803 is used to calculate the review priority score of each candidate box based on the confidence level, the boundary continuity verification score and the object category verification score corresponding to each candidate box. The queue partitioning unit 804 is used to divide the candidate box into multiple priority queues according to the review priority score, and the multiple priority queues include at least an automatic approval queue and a user review queue. The asset generation unit 805 is used to directly enter the asset generation process for candidate boxes belonging to the automatic pass queue, push candidate boxes belonging to the user review queue to the client for user review, and enter the asset generation process when the candidate boxes in the user review queue pass the review.

[0158] In some embodiments, when performing the step of performing boundary continuity verification on each candidate box and obtaining a boundary continuity verification score, the verification unit 802 is specifically used for: For each boundary of the candidate box, an inner pixel band is constructed inside the boundary and an outer pixel band is constructed outside the boundary. Calculate the color similarity, brightness difference, texture difference, and edge intensity between the inner pixel band and the outer pixel band; The boundary anomaly score of the boundary is calculated based on the color similarity, the brightness difference, the texture difference, and the edge intensity. The boundary continuity verification score of the candidate box is calculated based on the boundary anomaly score of each boundary.

[0159] In some embodiments, when performing the step of calculating the boundary anomaly score of the boundary based on the color similarity, the brightness difference, the texture difference, and the edge intensity, the verification unit 802 is specifically used for: The similarity between the inner and outer edges of the boundary is determined based on the color similarity, the brightness difference, and the texture difference. The boundary anomaly score is determined based on the internal and external similarity and the edge strength.

[0160] In some embodiments, when performing the step of performing object category verification on each candidate box and obtaining an object category verification score, the verification unit 802 is specifically used for: The object image is cropped from the candidate box; Extract the visual features of the object image; The visual features are compared with the target category samples in the category prototype library to obtain the object category verification score. The category prototype library stores multiple category samples of different object types, and the target category sample is the category sample in the category prototype library that corresponds to the region type.

[0161] In some embodiments, when the calculation unit 803 performs the step of calculating the review priority score of each candidate box based on the confidence level, the boundary continuity verification score, and the object category verification score corresponding to each candidate box, it is specifically used for: Determine the semantic conflict score between each candidate box and other candidate boxes in the original captured image of the calligraphy and painting; The object value of each candidate box is determined based on the pre-defined correspondence between object value and region type, and the region type of each candidate box. For each candidate box, a weighted fusion process is performed on the confidence score, the boundary continuity verification score, the object category verification score, the semantic conflict score, and the object value to obtain the review priority score for each candidate box.

[0162] In some embodiments, the user review queue includes a low-priority queue, a medium-priority queue, and a high-priority queue. The digital asset generation system 800 further includes a batch retry unit, which is used for: Obtain a batch retry instruction sent by the user through the client for multiple specified candidate boxes, wherein the batch retry instruction carries an error reason description corresponding to each specified candidate box; In response to the batch retry instruction, candidate boxes in the high-priority queue of the original calligraphy and painting images are regenerated using the visual big model to obtain multiple new candidate boxes, each carrying a regenerated label. The multiple new candidate boxes are pushed to the client for user review.

[0163] In some embodiments, the asset generation unit 805 specifically executes the asset generation process through the following steps: The original captured images of calligraphy and painting are cropped based on the boundary coordinates of the finally confirmed candidate boxes to obtain cropped images corresponding to each candidate box. Based on the cropped image corresponding to each candidate box and the region type and boundary coordinates corresponding to each candidate box, digital assets of calligraphy and painting corresponding to the original acquired image of calligraphy and painting are generated. The region type includes painting core, mounting, background, color card, gray scale ruler, scale ruler, seal, inscription, title label or collection label.

[0164] In summary, firstly, this application, through a large visual model, can identify multiple candidate boxes of different region types in the original acquired images of calligraphy and paintings that require region cropping, thereby increasing the richness of digital assets of calligraphy and paintings. Furthermore, compared to full manual annotation, it improves the efficiency of generating digital assets of calligraphy and paintings. Secondly, this application automatically scores each candidate box output by the large visual model (i.e., calculates the review priority score), and determines whether to push the candidate box to the user for review based on the score, eliminating the need for full manual review of candidate boxes. Therefore, this application can reduce the workload of manual review of candidate boxes while ensuring the quality of candidate box generation.

[0165] It should be noted that those skilled in the art can clearly understand that the specific implementation process of the above-mentioned digital asset generation system for calligraphy and painting and its various units can be referred to the corresponding descriptions in the aforementioned method embodiments. For the sake of convenience and brevity, these details will not be repeated here.

[0166] The aforementioned digital asset generation system for calligraphy and painting can be implemented as a computer program, which can, for example... Figure 7 It runs on the computer device shown.

[0167] Please see Figure 7 , Figure 7 This is a schematic block diagram of a computer device provided in an embodiment of this application. The computer device 900 can be a terminal or a server. The terminal can be an electronic device with communication functions, such as a smartphone, tablet, laptop, desktop computer, personal digital assistant, or wearable device. The server can be a standalone server or a server cluster composed of multiple servers.

[0168] See Figure 7 The computer device 900 includes a processor 902, a memory, and a network interface 905 connected via a system bus 901. The memory may include a non-volatile storage medium 903 and internal memory 904.

[0169] The non-volatile storage medium 903 can store an operating system 9031 and a computer program 9032. The computer program 9032 includes program instructions that, when executed, cause the processor 902 to perform a method for generating digital assets of calligraphy and painting.

[0170] The processor 902 provides computing and control capabilities to support the operation of the entire computer device 900.

[0171] The internal memory 904 provides an environment for the operation of the computer program 9032 in the non-volatile storage medium 903. When the computer program 9032 is executed by the processor 902, the processor 902 can execute a method for generating digital assets of calligraphy and painting.

[0172] This network interface 905 is used for network communication with other devices. Those skilled in the art will understand that... Figure 7 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device 900 to which the present application is applied. The specific computer device 900 may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0173] The processor 902 is used to run a computer program 9032 stored in the memory to perform the following steps: The visual large model is used to perform region recognition on the original acquired images of calligraphy and painting. At least one candidate box is returned by the visual large model. Each candidate box contains region type, boundary coordinates and confidence score. Perform boundary continuity verification on each candidate box to obtain a boundary continuity verification score, and perform object category verification on each candidate box to obtain an object category verification score; The review priority score for each candidate box is calculated based on the confidence level, boundary continuity verification score, and object category verification score corresponding to each candidate box. The candidate boxes are divided into multiple priority queues based on the review priority score, and the multiple priority queues include at least an automatic approval queue and a user review queue. Candidate boxes belonging to the automatic approval queue are directly entered into the asset generation process, while candidate boxes belonging to the user review queue are pushed to the client for user review. When a candidate box in the user review queue passes the review, the asset generation process begins.

[0174] It should be understood that in the embodiments of this application, the processor 902 may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0175] It will be understood by those skilled in the art that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program includes program instructions and can be stored in a storage medium, which is a computer-readable storage medium. The program instructions are executed by at least one processor in the computer system to implement the process steps of the embodiments of the above methods.

[0176] Therefore, this application also provides a storage medium. This storage medium can be a computer-readable storage medium. The storage medium stores a computer program, wherein the computer program includes program instructions. When executed by a processor, the program instructions cause the processor to perform the following steps: The visual large model is used to perform region recognition on the original acquired images of calligraphy and painting. At least one candidate box is returned by the visual large model. Each candidate box contains region type, boundary coordinates and confidence score. Perform boundary continuity verification on each candidate box to obtain a boundary continuity verification score, and perform object category verification on each candidate box to obtain an object category verification score; The review priority score for each candidate box is calculated based on the confidence level, boundary continuity verification score, and object category verification score corresponding to each candidate box. The candidate boxes are divided into multiple priority queues based on the review priority score, and the multiple priority queues include at least an automatic approval queue and a user review queue. Candidate boxes belonging to the automatic approval queue are directly entered into the asset generation process, while candidate boxes belonging to the user review queue are pushed to the client for user review. When a candidate box in the user review queue passes the review, the asset generation process begins.

[0177] The storage medium can be any computer-readable storage medium capable of storing program code, such as a USB flash drive, portable hard drive, read-only memory (ROM), magnetic disk, or optical disk.

[0178] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this application.

[0179] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For example, the division of each unit is merely a logical functional division, and there may be other division methods in actual implementation. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed.

[0180] The steps in the methods of this application embodiment can be adjusted, merged, or deleted according to actual needs. The units in the apparatus of this application embodiment can be merged, divided, or deleted according to actual needs. Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0181] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a terminal, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application.

[0182] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for generating digital assets of calligraphy and painting, characterized in that, include: The visual large model is used to perform region recognition on the original acquired images of calligraphy and painting. At least one candidate box is returned by the visual large model. Each candidate box contains region type, boundary coordinates and confidence score. A boundary continuity check is performed on each candidate box to obtain a boundary continuity check score, and an object category check is performed on each candidate box to obtain an object category check score, wherein the boundary continuity check score is used to indicate whether the boundary of the candidate box is located on the edge of a real object; The review priority score for each candidate box is calculated based on the confidence level, boundary continuity verification score, and object category verification score corresponding to each candidate box. The candidate boxes are divided into multiple priority queues based on the review priority score, and the multiple priority queues include at least an automatic approval queue and a user review queue. Candidate boxes belonging to the automatic approval queue are directly entered into the asset generation process, while candidate boxes belonging to the user review queue are pushed to the client for user review. When a candidate box in the user review queue passes the review, the asset generation process begins.

2. The method according to claim 1, characterized in that, The step of performing boundary continuity verification on each candidate box to obtain a boundary continuity verification score includes: For each boundary of the candidate box, an inner pixel band is constructed inside the boundary and an outer pixel band is constructed outside the boundary. Calculate the color similarity, brightness difference, texture difference, and edge intensity between the inner pixel band and the outer pixel band; The boundary anomaly score of the boundary is calculated based on the color similarity, the brightness difference, the texture difference, and the edge intensity. The boundary continuity verification score of the candidate box is calculated based on the boundary anomaly score of each boundary.

3. The method according to claim 2, characterized in that, The step of calculating the boundary anomaly score based on the color similarity, brightness difference, texture difference, and edge intensity includes: The similarity between the inner and outer edges of the boundary is determined based on the color similarity, the brightness difference, and the texture difference. The boundary anomaly score is determined based on the internal and external similarity and the edge strength.

4. The method according to claim 1, characterized in that, The step of performing object category validation on each candidate box to obtain an object category validation score includes: The object image is cropped from the candidate box; Extract the visual features of the object image; The visual features are compared with the target category samples in the category prototype library to obtain the object category verification score. The category prototype library stores multiple category samples of different object types, and the target category sample is the category sample in the category prototype library that corresponds to the region type.

5. The method according to claim 1, characterized in that, The step of calculating the review priority score for each candidate box based on the confidence level, boundary continuity verification score, and object category verification score corresponding to each candidate box includes: Determine the semantic conflict score between each candidate box and other candidate boxes in the original captured image of the calligraphy and painting; The object value of each candidate box is determined based on the pre-defined correspondence between object value and region type, and the region type of each candidate box. For each candidate box, a weighted fusion process is performed on the confidence score, the boundary continuity verification score, the object category verification score, the semantic conflict score, and the object value to obtain the review priority score for each candidate box.

6. The method according to claim 1, characterized in that, The user review queue includes a low-priority queue, a medium-priority queue, and a high-priority queue. Candidate boxes in the low-priority and medium-priority queues carry a random inspection flag, and the high-priority queue carries a mandatory inspection flag. After pushing candidate boxes belonging to the user review queue to the client for user review, the method further includes: Obtain a batch retry instruction sent by the user through the client for multiple specified candidate boxes, wherein the batch retry instruction carries an error reason description corresponding to each specified candidate box; In response to the batch retry instruction, candidate boxes in the high-priority queue of the original calligraphy and painting images are regenerated using the visual big model to obtain multiple new candidate boxes, each carrying a regenerated label. The multiple new candidate boxes are pushed to the client for user review.

7. The method according to claim 1, characterized in that, The asset generation process includes the following steps: The original captured images of calligraphy and painting are cropped based on the boundary coordinates of the finally confirmed candidate boxes to obtain cropped images corresponding to each candidate box. Based on the cropped image corresponding to each candidate box and the region type and boundary coordinates corresponding to each candidate box, digital assets of calligraphy and painting corresponding to the original acquired image of calligraphy and painting are generated. The region type includes painting core, mounting, background, color card, gray scale ruler, scale ruler, seal, inscription, title label or collection label.

8. A system for generating digital assets of calligraphy and painting, characterized in that, include: The candidate box acquisition unit is used to acquire at least one candidate box returned by the visual large model after performing region recognition on the original acquired image of calligraphy and painting. Each candidate box includes region type, boundary coordinates and confidence score. The verification unit is used to perform boundary continuity verification on each candidate box to obtain a boundary continuity verification score, and to perform object category verification on each candidate box to obtain an object category verification score, wherein the boundary continuity verification score is used to indicate whether the boundary of the candidate box is located on the edge of a real object; The calculation unit is used to calculate the review priority score of each candidate box based on the confidence level, the boundary continuity verification score and the object category verification score corresponding to each candidate box. A queue partitioning unit is used to divide the candidate box into multiple priority queues according to the review priority score, wherein the multiple priority queues include at least an automatic approval queue and a user review queue. The asset generation unit is used to directly enter the asset generation process for candidate boxes belonging to the automatic pass queue, and to push candidate boxes belonging to the user review queue to the client for user review. When the candidate box in the user review queue passes the review, it enters the asset generation process.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method for generating digital assets of calligraphy and painting as described in any one of claims 1-7.

10. A storage medium, characterized in that, The storage medium stores a computer program, which includes program instructions that, when executed by a processor, cause the processor to perform the method for generating digital assets of calligraphy and painting as described in any one of claims 1-7.

Images