Intelligent retrieval method and device for oracle rubbing

Abstract

The invention provides an intelligent retrieval method and device for an oracle rubbing, and the method comprises the steps: obtaining a first training sample with character division and character labeling, and training a character detection model according to the first training sample; performing character division and character labeling on each character in the rubbing data set according to the trained character detection model to obtain an expanded data set; performing environment simulation processing on the extended data set and the first training sample to obtain a second training sample; training a comparison model according to the second training sample to obtain a trained comparison model; associating the feature code of each character in the second training sample with the own character label, and establishing a feature database for character search; the environment simulation processing is performed on the training sample, so that the application range of the trained comparison model to a worse scene is expanded, and the accuracy of matching recognition of the comparison model to the oracle rubbing is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence, in particular to a method and device for intelligent retrieval of oracle bone inscriptions. BACKGROUND

[0002] Oracle bone inscriptions are the earliest known mature Chinese character system. Traditional oracle bone inscription processing relies on experts to transcribe, segment and compare characters one by one. An average-sized rubbing often contains dozens to hundreds of characters, and manual positioning and retrieval is extremely time-consuming. The same form, incomplete carving and rubbing noise make large-scale digital processing extremely difficult. In the past decade, the rise of deep convolutional networks and target detection frameworks has greatly promoted character detection and recognition in natural images. However, existing methods are mostly targeted at street signs or printed scenes, assuming that character boundaries are neat, strokes are continuous, and corpus is easy to read, which makes it difficult to be directly applied to oracle bone images with black lines, cracks and broken strokes. Most existing digital oracle bone platforms manually annotate character boxes and use neural network models for character matching, which not only has low accuracy, but also lacks robustness to large noise, light and rubbing differences, making it even more difficult to accurately match.

[0003] Therefore, it is urgent to overcome the defects of the prior art in the technical field. SUMMARY

[0004] The present application aims to solve the technical problem of how to improve the matching and recognition accuracy of neural network models for oracle bone characters.

[0005] The present application provides a method for intelligent retrieval of oracle bone inscriptions, comprising: obtaining a first training sample with character segmentation and character annotation, and training a character detection model according to the first training sample; segmenting and annotating each character in the rubbing dataset according to the trained character detection model to obtain an expanded dataset; performing environment simulation processing on the expanded dataset and the first training sample to obtain a second training sample, so as to improve the matching degree of the second training sample with different real scenes; extracting and encoding features of each character in the second training sample, training a comparison model according to the second training sample after feature encoding, and obtaining a trained comparison model; associating the feature encoding of each character in the second training sample with its own character annotation, and establishing a feature database for character search.

[0006] Preferably, the first training sample with character segmentation and character annotation is obtained, and the character detection model is trained according to the first training sample, specifically comprising: preprocessing the first training sample; obtaining the bounding box information and the classification information of each character according to the character division and the character annotation of each character in the preprocessed first training sample, encoding according to the bounding box information and the classification information of each character to obtain the true value encoding of each character; In each round of iterative training, the character detection model obtains the predicted encoding of each character according to the preprocessed first training sample, obtains the loss function of the character detection model according to the true value encoding and the predicted encoding of each character, and updates the character detection model according to the loss function. When the number of to-be-updated iteration rounds is greater than or equal to a preset maximum number of rounds, output the updated character detection model in the last round of iteration as the trained character detection model.

[0007] Preferably, the preprocessing of the first training sample specifically includes: The preprocessing includes one or more of the following: bounding box information loading on each character box in the first training sample, random scaling according to a preset scaling factor, random cropping, random adjustment of brightness within a preset brightness range, random adjustment of contrast within a preset contrast range, and random rotation within a preset angle range.

[0008] Preferably, the loss function of the character detection model obtained according to the true value encoding and the predicted encoding of each character specifically includes: obtaining the correlation between the true value encoding and the predicted encoding of the corresponding character according to the true value encoding and the predicted encoding of each character, and the corresponding expression is: ; wherein, is a matrix element, is a predicted classification vector in the predicted encoding of the corresponding character, is a real classification vector in the true value encoding of the corresponding character, is a real bounding box vector in the true value encoding of the corresponding character, is a predicted bounding box vector in the predicted encoding of the corresponding character, is a generalized intersection over union between the real bounding box vector and the predicted bounding box vector, is a norm loss between the real bounding box vector and the predicted bounding box vector, , and are weight parameters; obtaining the loss function of the character detection model according to the correlation between the true value encoding and the predicted encoding of each character, and the corresponding expression is: ; wherein L is a loss function of the character detection model, is a quality focus loss of the classification information, is a regression loss, is a compensation loss.

[0009] Preferably, the environment simulation processing on the augmented data set and the first training sample obtains a second training sample, so as to improve the matching degree of the second training sample with different real scenes, and specifically includes: obtaining each divided character box in the augmented data set and the first training sample and the corresponding differential sample of each character box; performing one or more of noise processing, erosion and expansion processing and blur processing on each character box and the corresponding differential sample of each character box according to a first preset probability; all processed character boxes and the corresponding differential sample of each character box are taken as the second training sample.

[0010] Preferably, each sample performs one or more of noise processing, erosion and expansion processing and blur processing according to a first preset probability, and specifically includes: a plurality of small spots are randomly generated in all divided character boxes in the augmented data set and the first training sample with a first preset probability, for simulating environmental noise; the strokes in all divided character boxes in the augmented data set and the first training sample are subjected to erosion operation or expansion operation by random floating point numbers with a first preset probability; each region in all divided character boxes in the augmented data set and the first training sample is subjected to focus type blur or jitter type blur according to a second preset probability with a first preset probability.

[0011] Preferably, the features of each character in the second training sample are extracted and encoded, the trained comparative model is obtained according to the second training sample after feature encoding, and specifically includes: the loss function expression of the comparative model is: ; wherein N is the number of images input in a single batch, τ is a temperature hyperparameter, and K is the total capacity of the negative sample queue, is an InfoNCE loss function, and are taken as positive sample pairs the i-th of the N images input in a single batch, and j is the j-th of the historical k vector queue, is a negative sample pair vector, and exp(*) is an exponential function.

[0012] Preferably, the oracle bone script intelligent retrieval method further comprises: inputting the input oracle bone script into the trained character detection model, performing character division on the input oracle bone script through the trained character detection model, and obtaining an input character frame of the input oracle bone script; inputting the input character frame required by the user into the trained comparison model, performing feature extraction on the input character frame through the comparison model, and obtaining feature encoding; comparing the feature encoding of the input character frame with the feature encoding of all characters in the feature database, and selecting the feature encoding of all matching characters in the feature database that match the input character frame; obtaining character labels of all matching characters according to the feature encoding of all matching characters, obtaining an oracle bone script corresponding to the matching characters according to the character labels of all matching characters, and a position of the matching characters in the oracle bone script.

[0013] In a second aspect, an oracle bone script intelligent retrieval device is provided, comprising at least one processor and a memory in communication connection with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the processor to execute the oracle bone script intelligent retrieval method.

[0014] In a third aspect, the present application further provides a non-volatile computer storage medium, wherein the computer storage medium stores computer executable instructions, and the computer executable instructions are executed by one or more processors to complete the method of the first aspect.

[0015] In a fourth aspect, a chip is provided, comprising a processor and an interface, which is used to call and run a computer program stored in a memory to execute the method of the first aspect.

[0016] In a fifth aspect, a computer program product containing instructions is provided, which, when executed on a computer or processor, causes the computer or processor to execute the method of the first aspect.

[0017] In a sixth aspect, an oracle bone script intelligent retrieval system is provided, comprising the oracle bone script intelligent retrieval device of the second aspect and using the oracle bone script intelligent retrieval method of the first aspect.

[0018] The application provides an oracle bone script expansion sheet intelligent retrieval method and device, which comprises the following steps: obtaining a first training sample with character division and character labeling, training a character detection model according to the first training sample; performing character division and character labeling on each character in an expansion sheet data set according to the trained character detection model to obtain an expanded data set; performing environment simulation processing on the expanded data set and the first training sample to obtain a second training sample; training a comparison model according to the second training sample to obtain a trained comparison model; associating the feature coding of each character in the second training sample with the character labeling of the character to establish a feature database for character search; and performing environment simulation processing on the training sample to improve the application range of the trained comparison model to a more severe scene and improve the accuracy of matching and identification of the comparison model to the oracle bone script expansion sheet. Further, the feature database is established to quickly obtain the associated expansion sheet of the input character of a user, and is used for quickly performing comparison research. BRIEF DESCRIPTION OF DRAWINGS

[0019] In order to more clearly illustrate the technical solutions of the embodiments of the application, the following will briefly introduce the drawings needed to be used in the embodiments of the application. Obviously, the drawings described below are only some of the embodiments of the application, and for those skilled in the art, other drawings can also be obtained from these drawings without creative labor.

[0020] Figure 1 It is a method flowchart of an oracle bone script expansion sheet intelligent retrieval method provided by the embodiments of the application. Figure 2 It is a schematic diagram of character segmentation, identification and retrieval in an oracle bone script expansion sheet intelligent retrieval method provided by the embodiments of the application. Figure 3 It is a method of obtaining a character detection model in an oracle bone script expansion sheet intelligent retrieval method provided by the embodiments of the application. Figure 4 It is a schematic diagram of character segmentation, identification and retrieval in an oracle bone script expansion sheet intelligent retrieval method provided by the embodiments of the application. Figure 5 It is a method flowchart of environment simulation processing in an oracle bone script expansion sheet intelligent retrieval method provided by the embodiments of the application. Figure 6 It is a flowchart of environment simulation processing in an oracle bone script expansion sheet intelligent retrieval method provided by the embodiments of the application. Figure 7 It is a method flowchart of environment simulation processing in another oracle bone script expansion sheet intelligent retrieval method provided by the embodiments of the application. Figure 8A method flow chart of character retrieval implementation in a method for intelligent oracle bone script expansion sheet retrieval provided by an embodiment of the present application; Figure 9 A method flow chart of comparison model and oracle bone script expansion sheet retrieval in a method for intelligent oracle bone script expansion sheet retrieval provided by an embodiment of the present application; Figure 10 A device schematic diagram of a device for intelligent oracle bone script expansion sheet retrieval provided by an embodiment of the present application. DETAILED DESCRIPTION

[0021] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely intended to explain the present application and should not be used to limit the present application.

[0022] Unless otherwise required by context, the term "comprises" in the specification and claims is to be construed as open-ended, i.e., as "comprises but not limited to". In the description of the specification, the terms "one embodiment", "some embodiments", "exemplary embodiment", "example", "specific example" or "some examples" are intended to mean that the particular feature, structure, material or characteristic being described in connection with such terms in the embodiment or example is included in at least one embodiment or example of the disclosure. The illustrative representations of the above terms do not necessarily refer to the same embodiment or example. In addition, the particular features, structures, materials or characteristics described in connection with the above terms can be included in any suitable manner in any one or more embodiments or examples, i.e., although they are carried in the embodiments or examples of the above terms due to the order of appearance and location, they are not limited to being carried in combination by one embodiment or example.

[0023] In the description of the present application, the terms "first", "second" are only used for description purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features limited by "first", "second" can be explicitly or implicitly included in one or more features. In the description of the embodiments of the present disclosure, unless otherwise specified, the meaning of "multiple" is two or more. In addition, for example, in the description, the same type of nouns can also be described as two independent individuals by adding "A", "B" at the end, in which case the features limited by "A", "B" are only used for the purpose of distinguishing the same type of individual description, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features.

[0024] In describing some embodiments, the use of "coupled" or "coupling" and "connected" or "connecting" along with their derivatives, can be used. For example, some embodiments can be described as being "connected" where directly connected or physically connected with the two components that are "connected" directly physically touching one another. As another example, some embodiments can be described as being "coupled" where directly coupled or electrically coupled with the two components that are "coupled" directly electrically touching one another. However, it should be understood that "connected" or "coupled" can also mean that two or more components are not directly physically or electrically touching one another, but nonetheless still co-operate or interact with one another. For example, "optically coupled", "wirelessly connected", etc. The embodiments disclosed herein are not necessarily limited to the inventive concept.

[0025] In the description of the application, the expression "A and / or B" (wherein A and B represent specific features) is used in the sense that the expression includes the following three combinations: A alone, B alone, and a combination of A and B.

[0026] As used herein, "about," "approximately," or "substantially" means an acceptable range of error for a given value or measurement; i.e., ±10% of the given value or measurement, unless otherwise specifically stated.

[0027] The embodiment provides an Oracle bone inscriptions intelligent retrieval method, as shown in the method flowchart. Figure 1 The method flowchart comprises the following steps. In step 101, a first training sample with character division and character annotation is obtained, and a character detection model is trained according to the first training sample.

[0028] In the embodiment, the first training sample can be an Oracle Bone Inscriptions Multi-modal Dataset (OBIMD for short), and the first training sample includes a plurality of oracle bone inscriptions samples, the oracle bone inscriptions sample refers to a picture of an oracle bone inscriptions tablet on which a plurality of oracle bone inscriptions characters are engraved. The character division refers to dividing different characters by a character box according to the positions of the characters on the oracle bone inscriptions sample and the sizes of the characters, so as to position the positions of the characters on the oracle bone inscriptions sample and clearly define the size ranges of the characters, avoiding mutual influence between different characters. The character annotation refers to numbering each character on the oracle bone inscriptions sample and corresponding each number to character information of the corresponding character, so as to clearly define the character information of each character on the oracle bone inscriptions tablet.

[0029] In this embodiment, the character detection model is used to divide each character on the oracle bone inscription according to the position and size, and to identify and classify each character. Since each oracle bone sample in the first training sample already has character division and character identification, the first training sample has correct values for division and identification classification. After the initialization of the character detection model is constructed, the first training sample can be input into the character detection model, and the character detection model outputs the predicted character division and the predicted character identification. The predicted values of the character division and the character identification are compared with the correct values, and through multiple iterations, the predicted values gradually converge to the correct values, thereby completing the training of the character detection model.

[0030] In step 102, each character in the oracle bone data set is divided and labeled according to the trained character detection model, and an expanded data set is obtained.

[0031] In this embodiment, considering that subsequent training of the contrast model is needed, the sample capacity of the first training sample may not be enough, so additional training samples need to be introduced. The oracle bone data set can be the oracle bone inscription copy system. Each oracle bone in the oracle bone data set is not divided and labeled for each character, so the oracle bone data set needs to be divided and labeled by the trained character detection model to obtain the position and information of each character in the oracle bone data set, ensuring that the oracle bone data set can be used as a sample set for training the contrast model.

[0032] In this embodiment, considering that the commonly used training samples, such as the oracle bone multi-modal data set and the oracle bone inscription copy system, are all oracle bone samples processed by artificial processing. After artificial processing, the oracle bone sample picture pixels are clear, the strokes of the oracle bone characters are clear, and there are basically no excessive stains as noise interference on the oracle bone. Therefore, if these training samples are directly used to train the contrast model, the contrast model trained will only be applicable to oracle bone samples with clear and less noise interference. For oracle bone samples with more noise interference, such as newly unearthed oracle bone, the photo taken on site may have unclear imaging, many noise points, and noise interference of broken and dirty oracle bone. It is difficult to input these oracle bone samples with more noise interference into the contrast model to output accurate prediction results. Therefore, in order to avoid the above problems, the following step design is involved in this embodiment: In step 103, the expanded data set and the first training sample are subjected to environment simulation processing to obtain a second training sample, so as to improve the matching degree of the second training sample with different real scenes.

[0033] The environment simulation processing is adding random noise, randomly eroding, expanding, deforming strokes, or randomly increasing dynamic blur to the extended data set and the tablet samples in the first training samples, to simulate the sample obtained in a relatively harsh scene, and the second training sample after the environment simulation processing is used to train the subsequent comparative model, so as to improve the applicable scene range of the comparative model.

[0034] In step 104, the features of each character in the second training sample are extracted and encoded, and the comparative model is trained according to the second training sample after feature encoding, to obtain the trained comparative model.

[0035] Further, in the embodiment, the comparative model is used to recognize a single character and compare it with the sample characters stored in its own database, so as to obtain the meaning and classification of the character, therefore, the feature extraction is to extract the strokes and styles of the character according to the feature type, and the feature encoding is obtained according to the feature type, and the feature encoding is in the form of vector. The training of the comparative model is to input the feature encoding of different characters, and compare the distance between the feature encodings of different characters, the closer the distance between the characters, the more likely they are the same character, and the farther the distance between the characters, the more likely they are different characters, and the training of the comparative model is completed through multiple rounds of iteration.

[0036] Further, in the embodiment, considering that the study of oracle bone characters usually needs to compare the same character on different tablets, to study the different writing methods of the same character or the meaning of the character on different tablets, etc., a function is needed to enable the user to input the corresponding character, obtain all tablets containing the character in the pre-set database, and know the position of the character in each tablet, to facilitate the user to compare and study the character on different tablets, therefore, the embodiment also relates to the following method steps: In step 105, the feature encoding of each character in the second training sample is associated with its own character label, to establish a feature database for character search.

[0037] In this embodiment, by associating the feature code with the character label of itself and storing it in the feature database, when the user inputs the corresponding input character, the comparison model extracts the features of the input character, obtains the feature code of the input character, and finds all reference characters similar to the input character (i.e. the characters that can be regarded as the same as the input character) from the feature database, and obtains the character label of all found reference characters, according to the character label of the reference character, the reference character can be obtained The position of the patch and on the patch, so as to output all patches containing the reference character and the position of the reference character on the corresponding patch, so as to facilitate the user to read and study. As shown in Figure 2 The schematic diagram for character segmentation and recognition in the patch sample and retrieval in this embodiment is shown in

[0038] Further, in this embodiment, the training of the character detection model involves the following design: the first training sample with character division and character label is obtained, and the character detection model is trained according to the first training sample, as shown in Figure 3 and Figure 4 The training flow chart of the character detection model is shown in Figure 4 The method flow includes: In step 201, the first training sample is preprocessed.

[0039] In this embodiment, the character detection model is initialized, and the preprocessing is to input the first training sample into the character detection model, and the first training sample is preprocessed by the character detection model, and the preprocessing includes one or more of the following: loading the bounding box information of each character box in the first training sample, randomly scaling according to a preset scale, randomly cropping, randomly adjusting the brightness in a preset brightness range, randomly adjusting the contrast in a preset contrast range, and randomly rotating according to a preset angle range.

[0040] In this embodiment, the preset scale, the preset brightness range, the preset contrast range and the preset angle range are set by the person skilled in the art according to the actual situation, the preset scale of the random scaling can be 0.2 times-5 times, the preset contrast range can be ±0.4, and the preset angle range can be ±20 degrees. The probability of random rotation can be 0.5.

[0041] In step 202, the bounding box information and classification information of each character are obtained according to the character division and character label of each character in the preprocessed first training sample, the bounding box information and classification information of each character are encoded to obtain the true value code of each character.

[0042] In the embodiment, the frame information includes the center point coordinates of the character frame and the length and width of the character frame, and the classification information is the character number of the corresponding character. The true value encoding is the real information related to the character.

[0043] In step 203, in each round of iterative training, the character detection model obtains the prediction encoding of each character according to the preprocessed first training sample, obtains the loss function of the character detection model according to the true value encoding and the prediction encoding of each character, and updates the character detection model according to the loss function.

[0044] In the embodiment, the preprocessed first training sample is input into the backbone network of the character detection model for convolution feature extraction. The backbone network can adopt a deep residual network structure, which is used to convert the image in the input first training sample into a feature map with four levels of different spatial resolutions. The feature map is mapped through a lightweight channel mapping module, the channel number is unified, and a fifth level of spatial scale is additionally generated, forming a feature pyramid. The feature pyramid provides multi-resolution information for the subsequent encoder based on cross-scale dynamic attention.

[0045] In the embodiment, the encoder includes a multi-layer Transformer structure based on dynamic deformable attention. Each layer of the Transformer structure in the encoder focuses on information aggregation of a small number of key positions, obtains context on feature maps of different scales through adaptive offset sampling, and greatly improves the representation ability of large and small size characters. After the output of the encoder, the decoder stage is entered. The query self-attention is performed inside the decoder layer to interact information between different queries, and then the cross-scale deformable attention is used to align the query and the encoder feature, and finally the feature update is completed through the feedforward network. The decoder iterates multiple times to generate a series of candidate boxes and classification scores.

[0046] In the embodiment, the prediction encoding includes prediction frame information and prediction classification information; the character detection model has vectors of all classification types of characters inside the model. For the prediction of classification information of different characters, the classification information of each character in the input first training sample is extracted and encoded, the encoded classification information is compared with all classification types of all characters inside the character detection model, and a multi-class confidence distribution of the encoded classification information relative to all classification types is obtained. According to the multi-class confidence distribution, the classification type to which the classification information of the corresponding character tends to be closer can be determined.

[0047] The correlation between the true value encoding and the prediction encoding of each character is obtained according to the true value encoding and the prediction encoding of each character, and the corresponding expression is: ; wherein, is the matrix element, is the predicted class vector in the predicted encoding of the corresponding character, is the true class vector in the ground truth encoding of the corresponding character, is the true bounding box vector in the ground truth encoding of the corresponding character, is the predicted bounding box vector in the predicted encoding of the corresponding character, is the generalized intersection over union between the true bounding box vector and the predicted bounding box vector, the greater the geometric overlap, the greater the the greater the overlap, the greater the value and the closer to 1, the smaller the overlap, the smaller the value. is the norm loss between the true bounding box vector and the predicted bounding box vector, used to compare the coordinate difference of the center of the bounding box; may be 1, may be 5, may be 2. The above formula is used to obtain the case where the distance between the predicted encoding and the ground truth encoding is minimized (i.e., the correlation is maximized), and the case where the distance between the predicted encoding and the ground truth encoding is minimized is corresponded, which is used for subsequent loss function calculation.

[0048] The loss function of the character detection model is obtained according to the correlation between the ground truth encoding and the predicted encoding of each character, and the corresponding expression is: ; In this embodiment, is the quality focus loss of the classification information, which is used to encourage the model to give higher confidence on the positive samples, and to punish the wrong classification. is the regression loss, which is used to measure the absolute error of the predicted bounding box information and the true bounding box information in the center coordinates and width and height. is used to compensate for the collateral effects of regression when GIoU is low. Wherein, , and are controlled and set by hyperparameters.

[0049] After the calculation is completed, the system performs back propagation on all trainable parameters, and adjusts the network weights according to the adaptive momentum and stochastic gradient descent (Adam+SGD) optimization algorithm. This iteration is performed until the preset training round is completed, and during the iteration, periodic verification can be inserted to monitor the performance of the model on the test set.

[0050] In the inference phase, the system only needs to perform input loading and simple preprocessing, without loading real box annotations and without data augmentation. The preprocessed images enter the same backbone, pyramid, encoder, decoder, and detection head to generate a batch of candidate boxes and their corresponding classification scores. Subsequently, non-maximum suppression (NMS) is performed on all candidates to filter out high-overlap and low-confidence boxes. For each remaining box, its normalized coordinates are converted back to the original image space, and the final output is selected according to the score threshold. The final result includes several bounding boxes and corresponding class confidences, indicating which characters are detected and their positions in the image. If subsequent semantic retrieval or contrastive learning is required, the box regions can be cropped to obtain single-character images, which are then sent to the representation model to generate feature vectors for approximate search. Thus, the end-to-end process seamlessly connects from supervised detection to unsupervised representation retrieval, completing the precise search and matching of single characters in the original map.

[0051] In step 204, when the to-be-updated iteration round is greater than or equal to the preset maximum round, the updated character detection model in the last iteration round is output as the trained character detection model.

[0052] Further, before training the contrast model in the embodiment, the first training sample also needs to be subjected to environment simulation processing, and therefore the embodiment also relates to the following design: The environment simulation processing is performed on the augmented data set and the first training sample to obtain a second training sample, so as to improve the matching degree of the second training sample with different real scenes, such as Figure 5 As shown in the figure, the method flow includes: In step 301, each divided character box in the augmented data set and the first training sample and the corresponding differential sample of each character box are obtained.

[0053] It should be noted that in the embodiment, the map data set and the first training sample are subjected to differential processing when importing pictures, and the corresponding differential sample is also processed when processing each image subsequently, so that each divided character box also has a corresponding differential sample. The differential processing is that differences are introduced between two views in the pixel dimension through random different preprocessing, but they are semantically derived from the same original character image, forming a positive sample pair, one of which is sent to the k branch and is subsequently encoded into a k vector to enter the queue, and the other is sent to the q branch and is encoded into a q vector.

[0054] In step 302, one or more of noise processing, erosion and dilation processing, and blur processing are performed on each character box and the corresponding differential sample of each character box according to a first preset probability.

[0055] In this embodiment, the first preset probability is set by those skilled in the art based on actual circumstances. Specifically, the first preset probability can be 50%.

[0056] In step 303, all processed character frames and the differential samples corresponding to each character frame are used as the second training samples.

[0057] like Figure 6 The diagram shown is a schematic representation of the environmental simulation process in this embodiment.

[0058] Each sample undergoes one or more of the following processing methods according to a first preset probability: noise reduction, erosion and dilation processing, and blurring processing. Figure 7 As shown, it specifically includes: In step 401, multiple small spots are randomly generated within all the divided character frames in the expanded dataset and the first training sample with a first preset probability to simulate environmental noise.

[0059] In this embodiment, the environmental simulation processing is performed on the character frames after each character in the expanded dataset and the first training sample. It should be noted that before performing the environmental simulation processing, considering that oracle bone rubbings are often in single-channel grayscale form, it is necessary to first convert the grayscale character frames into 3×8-bit RGB images.

[0060] In this embodiment, custom-defined verification noise can be used to simulate small blank or excessively inked spots that appear on oracle bone rubbings during the rubbing, photographing, scanning, or transmission of ancient books, thus enriching the training sample distribution. This is achieved by randomly determining the number and position of white or black dots based on the image size and specified ratio. Specifically, at random coordinate positions in the image, the pixel value is set to 0 (i.e., black dot) or 255 (i.e., white dot).

[0061] In step 402, the characters in all the divided character frames in the expanded dataset and the first training sample are subjected to stroke erosion or dilation operations using random floating-point numbers with a first preset probability.

[0062] In this embodiment, the erosion and dilation process simulates the subtle breaks in the lines of characters on real oracle bone rubbings caused by long-term preservation or rubbing techniques, as well as the possible thickening of lines due to ink diffusion. During model training, incorporating erosion and dilation reduces the probability of misidentifying the same character as different views due to variations in stroke thickness or minor breaks during comparative learning. In this embodiment, the erosion and dilation process uses randomly generated floating-point numbers between 0 and 3 to distinguish between "erosion" and "dilation" operations, and randomly selects kernels of sizes 1 to 3 to implement the corresponding erosion and dilation operations.

[0063] In step 403, each region within all the divided character frames in the expanded dataset and the first training sample is subjected to either focus-type blurring or jitter-type blurring according to a first preset probability and a second preset probability.

[0064] In this embodiment, the blurring process is used to add random Gaussian blur with a 50% probability to the image of each character frame to simulate blurring caused by focusing or shaking during actual shooting or scanning. This helps to test the encoder's ability to capture global and local features, and at the same time, to prevent overfitting to high-frequency noise, it allows the model to encode the glyphs from a more macroscopic level. The blurring process can be implemented by setting the Gaussian kernel size k between 3 and 15 and randomly selecting an odd number, and the standard deviation σ being random within the range of 0.1 to 1.0, and then calling the Gaussian blur function to perform convolution blurring on the image of the character frame.

[0065] The aforementioned preprocessing steps aim to transform oracle bone rubbings into three-channel tensors suitable for comparative learning, providing a reliable feature foundation for unsupervised representation learning and subsequent retrieval tasks. First, the conversion from grayscale to RGB images ensures the compatibility of the model input, allowing the original single-channel brightness information to be smoothly integrated into the ResNet structure based on ImageNet pre-trained weights without introducing additional noise. Next, through custom noise processing and erosion / dilation operations, multi-source micro-deformations such as ink stains, paper stains, stroke breaks, and diffusion commonly encountered during rubbing, photography, and preservation are simulated. These semantically irrelevant perturbations force the encoder to focus on the global shape and texture of the bone-carved characters, thereby improving its ability to discriminate the essential structure of the characters. Furthermore, a random Gaussian blur with a 50% probability is added to simulate focal length drift and jitter noise. This step strengthens the model's robustness to different resolutions and low-frequency features while suppressing overfitting to high-frequency noise. Finally, the dual-view model generates positive sample pairs C×H×W tensors with rich differences in scale, position, rotation angle, and color style through a series of geometric and color enhancements. This design ensures that the InfoNCE loss in the momentum contrastive learning framework meets the dual discriminative requirements of "similarity of the same rubbing under varying conditions" and "distinguishability between different rubbings".

[0066] Furthermore, the process involves extracting and encoding features from each character in the second training sample, and then training the comparison model based on the feature-encoded second training sample to obtain the trained comparison model. The training process of the comparison model specifically includes: The generated positive sample pair C×H×W tensor is input into the encoder q / k branch, and the resulting positive sample pair vector is obtained after encoding. The q branch participates in reverse updates, while the k branch updates via momentum. They are incorporated into a circular negative sample queue, which stores historical k vectors. The negative sample vector is constructed, and then the InfoNCE function is used, and the loss function expression of the contrast model is: ; Wherein, N is the number of images in a single batch input, that is, the number of corresponding batch positive sample pairs, τ is a temperature hyperparameter, and K is the total capacity of the negative sample queue, is the InfoNCE loss function, and together constitute a positive sample pair, The i-th vector from the k branch in the corresponding batch, is the positive sample vector of the q branch corresponding to it, i is the i-th, that is, the i-th positive sample pair, of the N images input in the corresponding batch, and j is the j-th in the historical k vector queue, is the negative sample pair vector, and exp(*) is an exponential function. The InfoNCE loss function increases the difference between the positive sample and the entire negative sample, and reduces the difference between the positive sample pairs. The loss function is backpropagated to update the weights in the q branch. In the subsequent tile retrieval process, the learned features can be used to construct an efficient feature vector index (such as k-NN based neighbor search), to realize automatic matching of homologous tiles in a cross-batch, cross-device environment. This method not only improves the depth representation ability of the model for oracle bone characters, but also provides a low-annotation, quickly expandable intelligent retrieval solution for oracle bone databases and AI large models.

[0067] Further, in the present embodiment, after the contrast model and the character detection model are trained, the feature encoding of all characters in the second training sample is associated with the character label of itself. Thus, after obtaining the feature encoding of a character, the tile where the corresponding character is located and the specific position on the tile can be known, and the corresponding association relationship, the feature encoding of each character in the second training sample, and the character label are collectively used as the feature database, and the search function of the tile associated with the character is realized through the feature database. The implementation method of the corresponding search function is as follows: As Figure 8 shown, the oracle bone tile intelligent retrieval method further comprises: In step 501, the input tile is input to the trained character detection model, the input tile is divided into characters by the trained character detection model, and the input character frame of the input tile is obtained.

[0068] In step 502, the input character frame required by the user is input to the trained contrast model, and the contrast model extracts features from the input character frame and obtains feature encoding.

[0069] In step 503, the feature encoding of the input character box is compared and identified with the feature encodings of all characters in the feature database, and the feature encodings of all matching characters that match the input character box in the feature database are selected.

[0070] It should be noted that in this embodiment, the three characters with the highest similarity to the input character box in the feature database can be selected as the matching characters.

[0071] In step 504, the character annotations of all matching characters are obtained according to the feature encodings of all matching characters, rubbings corresponding to the matching characters are obtained according to the character annotations of all matching characters, and the positions of the matching characters in the rubbings.

[0072] As Figure 9 shown, it is a schematic diagram of the method flow of the comparison model and the oracle bone rubbing retrieval.

[0073] To more intuitively reflect the above method flow, the following is an example: Select rubbing h3320 and identify the fourth oracle bone character in the first column. After loading the rubbing image, the character detection model will segment and sort the 7 oracle bone characters on the rubbing. After the user manually selects number 4, the comparison model will display the extraction of the features of the oracle bone character corresponding to number 4, and make a corresponding speculation with modern Chinese characters according to the extracted features. The characters in the input character box are compared with all characters in the feature database by the comparison model to obtain the confidence ranking between the characters in the input character box and other characters, and the character with the highest confidence is matched with the character in the character box. Suppose the user selects the first confidence Chinese character "伐" and clicks on the "Rubbing Retrieval" text box, the system will automatically retrieve all rubbings that contain the oracle bone character "伐" at the same time, such as h32175, h32045, etc.

[0074] Embodiment 2: As Figure 10 shown, it is a schematic diagram of the oracle bone rubbing intelligent retrieval device of the embodiment of the present invention. The oracle bone rubbing intelligent retrieval device of this embodiment includes one or more processors 41 and a memory 42.

[0075] The processor 41 and the memory 42 can be connected by a bus or other means, Figure 10 taking the connection by bus as an example.

[0076] The memory 42, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs and non-volatile computer-executable programs, such as the oracle bone rubbing intelligent retrieval method in the above embodiment. The processor 41 executes the oracle bone rubbing intelligent retrieval method by running the non-volatile software programs and instructions stored in the memory 42.

[0077] Memory 42 can include high-speed random access memory and can also include nonvolatile memory, such as at least one disk storage device, flash memory device, or other nonvolatile solid-state memory device. In some embodiments, memory 42 can optionally include memory that is remotely located with respect to processor 41, such as memory that is connected to processor 41 via a network. Examples of such networks include, but are not limited to, the Internet, intranet, local area network, mobile communications network, and combinations thereof.

[0078] The program instructions / modules are stored in the memory 42, when executed by the one or more processors 41, perform the Oracle chart intelligence retrieval method in the above embodiments, for example, perform each step shown in each flowchart described above.

[0079] The embodiment of the present application further provides a computer storage medium, which has computer program instructions stored thereon; the computer program instructions are executed by a processor to implement the Oracle chart intelligence retrieval method provided by the embodiment of the present application.

[0080] The above only describes the preferred embodiments of the present application and is not used to limit the present application, and any modification, equivalent replacement, and improvement within the spirit and principle of the present application should be included in the protection scope of the present application.

Claims

1. A method for intelligent retrieval of oracle bone inscription rubbings, characterized in that, include: Obtain the first training sample with character segmentation and character annotation, and train the character detection model based on the first training sample; Based on the trained character detection model, each character in the rubbing dataset is segmented and labeled to obtain an expanded dataset; The expanded dataset and the first training sample are subjected to environmental simulation processing to obtain a second training sample, so as to improve the matching degree of the second training sample with different real-world scenarios; Features are extracted and encoded for each character in the second training sample. The comparison model is then trained based on the second training sample with the encoded features to obtain the trained comparison model. The feature codes of each character in the second training sample are associated with their own character annotations to establish a feature database for character search.

2. The intelligent retrieval method for oracle bone inscription rubbings according to claim 1, characterized in that, The step of obtaining a first training sample with character segmentation and character annotation, and training the character detection model based on the first training sample, specifically includes: The first training sample is preprocessed; Based on the character segmentation and character annotation of each character in the preprocessed first training sample, the border information and classification information of each character are obtained. The border information and classification information of each character are then encoded to obtain the ground value code of each character. In each round of iterative training, the character detection model obtains the predicted encoding of each character based on the preprocessed first training sample, obtains the loss function of the character detection model based on the ground value encoding and predicted encoding of each character, and updates the character detection model based on the loss function. When the number of iterations is greater than or equal to the preset maximum number of iterations, the updated character detection model in the last iteration is output as the trained character detection model.

3. The intelligent retrieval method for oracle bone inscription rubbings according to claim 2, characterized in that, The preprocessing of the first training sample specifically includes: The preprocessing includes one or more of the following: loading border information for each character box in the first training sample, randomly scaling it according to a preset magnification, randomly cropping it, randomly adjusting the brightness within a preset brightness range, randomly adjusting the contrast within a preset contrast range, and randomly rotating it according to a preset angle range.

4. The intelligent retrieval method for oracle bone inscription rubbings according to claim 2, characterized in that, The loss function for obtaining the character detection model based on the truth encoding and predicted encoding of each character specifically includes: The correlation between the true value encoding and the predicted encoding of each character is obtained from the true value encoding and the predicted encoding of that character. The corresponding expression is: ； in, For matrix elements, This is the predicted classification vector in the predicted encoding of the corresponding character. This is the true classification vector in the truth value encoding of the corresponding character. The true border vector in the truth value encoding of the corresponding character. This refers to the predicted bounding box vector in the predictive encoding of the corresponding character. The generalized intersection-union ratio (CIU) is the ratio between the ground truth bounding box vector and the predicted bounding box vector. The norm loss is the difference between the ground truth bounding box vector and the predicted bounding box vector. , and All are weighted parameters; The loss function of the character detection model is obtained based on the correlation between the ground truth encoding and the predicted encoding of each character. The corresponding expression is: ； Where L is the loss function of the character detection model. For the loss of quality focus of classified information, To regress the loss, To compensate for the losses.

5. The intelligent retrieval method for oracle bone inscription rubbings according to claim 2, characterized in that, The process of performing environmental simulation processing on the expanded dataset and the first training samples to obtain second training samples, in order to improve the matching degree of the second training samples with different real-world scenarios, specifically includes: Obtain each segmented character frame and the corresponding differential sample from the augmented dataset and the first training sample; According to the first preset probability, one or more of the following are performed on each character frame and the differential sample corresponding to each character frame: noise processing, erosion and dilation processing, and blurring processing. All processed character frames and the differential samples corresponding to each character frame are used as the second training samples.

6. The intelligent retrieval method for oracle bone inscription rubbings according to claim 5, characterized in that, Each sample undergoes one or more of the following processing methods according to a first preset probability: noise reduction, erosion and dilation processing, and blurring processing. Specifically, this includes: Multiple small spots are randomly generated within all the divided character frames in the expanded dataset and the first training sample with a first preset probability to simulate environmental noise. With a first preset probability, the characters in all the divided character frames in the expanded dataset and the first training sample are subjected to stroke erosion or dilation operations using random floating-point numbers. With a first preset probability, each region within all the divided character frames in the expanded dataset and the first training sample is subjected to either focus-type blurring or jitter-type blurring with a second preset probability.

7. The intelligent retrieval method for oracle bone inscription rubbings according to claim 2, characterized in that, The process of extracting and encoding features from each character in the second training sample, and then training the comparison model based on the feature-encoded second training sample to obtain the trained comparison model, specifically includes: The loss function expression for the comparison model is: ； Where N is the number of images in a single batch of input, τ is the temperature hyperparameter, and K is the total capacity of the negative sample queue. For the InfoNCE loss function, and Together as positive sample pairs The i-th image among the N images input in a single batch, where j is the j-th image in the historical k-vector queue. Let be a vector of negative sample pairs, and exp(*) be an exponential function.

8. The intelligent retrieval method for oracle bone inscription rubbings according to claim 2, characterized in that, The intelligent retrieval method for oracle bone inscription rubbings also includes: The input rubbing is input into the trained character detection model, and the trained character detection model is used to segment the input rubbing into characters to obtain the input character frame of the input rubbing. The user inputs the required input character box into the trained comparison model, which extracts features from the input character box and obtains feature encoding. The feature code of the input character box is compared and identified with the feature codes of all characters in the feature database, and the feature codes of all matching characters in the feature database that match the input character box are selected. Based on the feature encoding of all matching characters, character labels for all matching characters are obtained. Based on the character labels of all matching characters, rubbings corresponding to the matching characters are obtained, as well as the positions of the matching characters in the rubbings.

9. A smart retrieval device for oracle bone inscription rubbings, characterized in that, The method includes at least one processor and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the processor for performing the intelligent retrieval method for oracle bone rubbings according to any one of claims 1-8.

10. A non-volatile computer storage medium, characterized in that, The computer storage medium stores computer program instructions, which, when executed by one or more processors, implement the intelligent retrieval method for oracle bone rubbings as described in any one of claims 1-8.

Images