Archaeological information extraction method, device, equipment and medium based on deep learning

Abstract

The present application relates to a method, device, equipment and medium for extracting archaeological information based on deep learning, wherein the method comprises: obtaining an archaeological file to be processed, and converting the archaeological file to be processed into image information to obtain an initial archaeological image; performing target detection on the initial archaeological image through a trained YOLO model to obtain a target detection result, and performing image segmentation according to the target detection result to obtain a segmented archaeological image, wherein the target detection result includes a plurality of prediction frames; performing OCR recognition on the segmented archaeological image based on the prediction frame, generating target annotations and serial number information, and formatting the segmented archaeological image according to the target annotations and serial number information to obtain target archaeological image information. The present application combines the target detection and OCR recognition of the YOLO model, can accurately segment out the archaeological image, and accurately annotate the segmented archaeological image, without the need for human intervention, and effectively improves the efficiency of archaeological information extraction.

Description

Technical Field

[0001] The present application relates to the field of computer vision technology, and in particular to a method, device, equipment and medium for extracting archaeological information based on deep learning. Background Art

[0002] With the deepening of archaeological research, the importance of morphological analysis and classification of artifacts in archaeological reports has become increasingly significant. Morphological analysis of archaeological artifacts often uses image processing technology to extract the contour feature points and size parameters of the artifact to assist classification and research. In the prior art, by obtaining a photo or line drawing of the artifact, extracting the contour curve and performing noise reduction processing, the feature points and size parameters of the artifact are finally obtained. In another prior art, the pdf is first converted into a picture, the converted picture is processed into a grayscale image and binarized, and then the contour is processed using an expansion algorithm to eliminate small contour breakpoints and clean up noise based on connectivity detection and use a closed operation to close the contour, and finally edge detection and automatic cropping are performed to obtain the segmented cultural relic image.

[0003] In these two methods of segmenting and extracting artifacts in archaeological reports, traditional image processing methods are used, which do not have the ability to be widely used in different scenes. Due to the problem of artifact image noise in archaeological reports due to reasons such as PDF clarity, the above two methods each propose a method for processing image noise, which is to use the connectivity of binary image pixels to detect small areas in the image and to record the width of all artifact points from top to bottom and the width position of each width to obtain the shape curve. Such methods are only suitable for simple artifact forms. For complex asymmetric images with many similar detailed structures, connectivity detection is difficult to distinguish irrelevant information from effective contours, and the edge information on both sides obtained by averaging the shape curve is very poor. Since these methods lack the ability of feature extraction and adaptive adjustment, the efficiency and accuracy of archaeological information extraction are low. Summary of the invention

[0004] The purpose of the embodiments of the present application is to propose an archaeological information extraction method, device, equipment and medium based on deep learning to improve the efficiency and accuracy of archaeological information extraction.

[0005] In order to solve the above technical problems, the embodiment of the present application provides an archaeological information extraction method based on deep learning, comprising:

[0006] Acquire the archaeological file to be processed, and convert the archaeological file to be processed into image information to obtain an initial archaeological image;

[0007] Performing target detection on the initial archaeological image through the trained YOLO model to obtain a target detection result, and performing image segmentation according to the target detection result to obtain a segmented archaeological image, wherein the target detection result includes multiple prediction boxes;

[0008] Based on the prediction frame, OCR recognition is performed on the segmented archaeological image to generate target annotation and serial number information, and the segmented archaeological image is formatted and named according to the target annotation and serial number information to obtain target archaeological image information.

[0009] In order to solve the above technical problems, the embodiment of the present application provides an archaeological information extraction device based on deep learning, comprising:

[0010] An archaeological file acquisition unit is used to acquire the archaeological file to be processed and convert the archaeological file to be processed into image information to obtain an initial archaeological image;

[0011] A target detection unit is used to perform target detection on the initial archaeological image through a trained YOLO model to obtain a target detection result, and perform image segmentation according to the target detection result to obtain a segmented archaeological image, wherein the target detection result includes a plurality of prediction boxes;

[0012] An image naming unit is used to perform OCR recognition on the segmented archaeological image based on the prediction box, generate target annotation and serial number information, and format the segmented archaeological image according to the target annotation and the serial number information to obtain target archaeological image information.

[0013] To solve the above technical problems, a technical solution adopted by the present invention is: to provide a computer device, including one or more processors; a memory for storing one or more programs, so that one or more processors can implement any one of the above-mentioned archaeological information extraction methods based on deep learning.

[0014] In order to solve the above technical problems, a technical solution adopted by the present invention is: a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, any one of the above-mentioned deep learning-based archaeological information extraction methods is implemented.

[0015] The embodiment of the present invention provides an archaeological information extraction method, device, equipment and medium based on deep learning. The method includes: obtaining an archaeological file to be processed, and converting the archaeological file to be processed into image information to obtain an initial archaeological image; performing target detection on the initial archaeological image through a trained YOLO model to obtain a target detection result, and performing image segmentation according to the target detection result to obtain a segmented archaeological image, wherein the target detection result includes multiple prediction frames; performing OCR recognition on the segmented archaeological image based on the prediction frame, generating target annotations and serial number information, and formatting the segmented archaeological image according to the target annotations and the serial number information to obtain target archaeological image information. The embodiment of the present invention can accurately segment out the archaeological image and accurately annotate the segmented archaeological image by converting the archaeological file to be processed into an archaeological image, and combining the target detection and OCR recognition of the YOLO model, and does not require human intervention, thereby effectively improving the efficiency of archaeological information extraction. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the scheme in the present application, a brief introduction is given below to the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0017] Figure 1 It is a flowchart of the implementation process of the archaeological information extraction method based on deep learning provided in the embodiment of the present application;

[0018] Figure 2 It is a flowchart for implementing the first sub-process in the archaeological information extraction method based on deep learning provided in an embodiment of the present application;

[0019] Figure 3 It is a flowchart for implementing the second sub-process in the archaeological information extraction method based on deep learning provided in an embodiment of the present application;

[0020] Figure 4 It is a flowchart for implementing the third sub-process in the archaeological information extraction method based on deep learning provided in an embodiment of the present application;

[0021] Figure 5 It is a flowchart for implementing the fourth sub-process in the archaeological information extraction method based on deep learning provided in an embodiment of the present application;

[0022] Figure 6 It is a flowchart for implementing the fifth sub-process in the archaeological information extraction method based on deep learning provided in an embodiment of the present application;

[0023] Figure 7It is a schematic diagram of generating target archaeological image information provided by an embodiment of the present application;

[0024] Figure 8 It is a flowchart for implementing a process in an archaeological information extraction method based on deep learning provided in another embodiment of the present application;

[0025] Fig. 9 is a schematic diagram of a mask displayed when the mouse is hovered over the preview area provided in an embodiment of the present application;

[0026] Fig.10 is a schematic diagram of a blue anchor point set by clicking the left button of a mouse provided in an embodiment of the present application;

[0027] Fig.11 is a schematic diagram of a red anchor point set by right-clicking a mouse provided in an embodiment of the present application;

[0028] Fig.12 It is a schematic diagram of the index input box in the lower right corner of the object card for changing the order of the object card provided in the embodiment of the present application;

[0029] Fig.13 It is a schematic diagram of a title input box above an artifact card for changing the title of an artifact card provided in an embodiment of the present application;

[0030] Fig.14 It is a schematic diagram of an archaeological information extraction device based on deep learning provided in an embodiment of the present application;

[0031] Fig.15 It is a schematic diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0032] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present application; the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit the present application; the terms "including" and "having" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of the present application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0033] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0034] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0035] The present invention is described in detail below with reference to the accompanying drawings and embodiments.

[0036] It should be noted that the archaeological information extraction method based on deep learning provided in the embodiments of the present application is generally executed by a computer device, and accordingly, the archaeological information extraction device based on deep learning is generally configured in a computer device.

[0037] See also Figure 1 , Figure 1 A specific implementation of the archaeological information extraction method based on deep learning is shown.

[0038] It should be noted that if there are substantially the same results, the method of the present invention is not limited to Figure 1 The process sequence shown is limited to the following steps:

[0039] S1: Acquire the archaeological file to be processed, and convert the archaeological file to be processed into image information to obtain an initial archaeological image.

[0040] Specifically, an archaeological file to be processed is obtained, and the archaeological file to be processed is a PDF format file, and there is artifact image information corresponding to archaeology in the archaeological file to be processed. In an embodiment of the present application, the archaeological file to be processed needs to be converted into image information to obtain an initial archaeological image. Wherein, the initial archaeological image can be in JPEG format or PNG format.

[0041] See also Figure 2 , Figure 2 A specific implementation of step S1 is shown, which is described in detail as follows:

[0042] S11: When the user uploads the archaeological file to be processed, the archaeological file to be processed is transmitted to the back end in the form of FormData through the front end;

[0043] S12: Obtain the number of pages of the archaeological file to be processed, and return the number of pages and the file name of the archaeological file to be processed to the front end to obtain the request information of the front end for the archaeological file to be processed;

[0044] S13: Convert the archaeological file to be processed in the request information into image information to obtain the initial archaeological image.

[0045] The embodiment of the present application provides an archaeological information extraction system. The user selects and uploads the archaeological file to be processed in the interface of the front end, and the front end passes the archaeological file to be processed to the back end in the form of FormData. After receiving the archaeological file to be processed, the back end saves it to the local uploads folder, and enables the pdfinfo command tool to analyze the archaeological file to be processed and obtain the number of pages of the archaeological file to be processed. Then, the back end returns the number of pages and the file name of the archaeological file to be processed to the front end as a JSON response. After receiving the file name and the number of pages, the front end requests the image of a certain page from the back end through the number of pages and generates the corresponding request information. At this time, the back end converts the archaeological file to be processed into a PNG image according to the request information through the pdftoppm tool in the poppler tool library. During the conversion, pdftoppm uses the -singlefile parameter to ensure that the file name of the image is unique to avoid file overwriting. The converted initial image is saved in PNG format, and the file name consists of the file name and the number of pages of the archaeological file to be processed, such as "example.pdf-page-1.png". Among them, the front end refers to the user end, and the back end refers to the service end.

[0046] In a specific embodiment, after the initial archaeological image is generated, the backend reads the image file corresponding to the initial archaeological image and converts it into Base64 format, because Base64 encoding is a way to convert binary data into text format, which is suitable for transmitting images or other binary data. The backend returns the converted Base64 string as a JSON response to the frontend. After the frontend receives the Base64-encoded image, it uses HTML to The src attribute of the tag displays the image in the preview area. Because the image has been converted to a Base64 string, the front end does not need to make an additional HTTP request to obtain the image file, and can directly display the image through the Base64 data source.

[0047] In another specific embodiment, in order to optimize the user operation experience, parallel processing is referenced in the embodiment of the present application. When the user jumps to the current page, the back end will automatically process the subsequent pages at the same time. When the user jumps to the next page, the back end no longer needs to re-upload the processing operation, saving the user's waiting time to achieve a faster response effect. In the processAllPagesSegment function, the front end divides the processing tasks of each PDF page into multiple batches (for example, 3 pages per batch) and processes each batch concurrently. The processing of each batch includes operations such as uploading images and segmenting images. Through the uploadAndSegmentPage function, the front end uploads each page and processes the image segmentation. The image of each page will be uploaded to the server first, the server generates the image and returns the Base64-encoded data, and the front end receives and processes it and displays it. Async / await and delay (setTimeout) are used to ensure that tasks are performed in order and pause and wait for user interaction when necessary.

[0048] S2: Performing target detection on the initial archaeological image through the trained YOLO model to obtain a target detection result, and performing image segmentation according to the target detection result to obtain a segmented archaeological image. The target detection result includes multiple prediction boxes.

[0049] The embodiment of the present application aims to automatically identify, cut and name the elements such as line drawings, legends, serial numbers and artifacts in archaeological documents through image recognition and natural language processing technology to realize visual analysis and storage of archaeological data. The YOLO v10 model has efficient target detection capability, so the YOLO v10 model is used as the target detection model in the embodiment of the present application. Therefore, the initial archaeological image is automatically identified and the coordinates are extracted by the trained YOLO model to perform target detection on the initial archaeological image to obtain the target detection result. Different prediction frames are included in the target detection result, and the prediction frame includes an overall frame, a line drawing frame, a serial number frame and a legend frame.

[0050] See also Figure 3 , Figure 3 A specific implementation of step S2 is shown, which is described in detail as follows:

[0051] S21: Image path verification of initial archaeological images;

[0052] S22: If the image path verification is valid, a prediction directory is generated according to the number of pages and the file name of the archaeological file to be processed;

[0053] S23: Identify and extract coordinates of the line elements of the initial archaeological image based on the prediction catalog through the trained YOLO model to generate the target detection result.

[0054] Specifically, before implementing step S21, in order to prevent the request of the same page from being processed repeatedly, the front end controls the processing of each page through this.pageLocks[page]. Only when a certain page is not locked, the pdf page is uploaded and processed from the front end through the / uploadimg route. In an embodiment of the present application, when the initial archaeological image is uploaded to the back end, the initial archaeological image path verification will be performed first to check whether the initial archaeological image path is valid. If the path is invalid, an error response is returned; if the image path verification is valid, a unique prediction directory (predict{page_id}) is generated for each initial archaeological image based on the number of pages and the file name of the archaeological file to be processed. Then, the line drawing elements of the initial archaeological image are identified and the coordinates are extracted based on the prediction directory by the trained YOLO model to generate the target detection result. Specifically, the trained YOLO model is then used to perform page analysis, and the trained YOLO model best.pt is loaded. The back end will call the yolo predict command, which will perform target detection on the uploaded image and generate results. The following parameters are involved: model={model_path} is the specified YOLO model path (best.pt); source={image_path} is the image path; save_txt=True is to save the detection results as a .txt file; name=predict{page_id} is the specified output directory name; project={file_name} is the specified root directory for storing output results. This command will generate a new directory (for example, predict{page_id}) under the runs / detect directory, which contains the target detection results (.txt file) and the image of the annotated box (*.jpg or *.png file);

[0055] S24: Perform image segmentation according to the target detection result to obtain the segmented archaeological image.

[0056] See also Figure 4 , Figure 4 A specific implementation of step S24 is shown, which is described in detail as follows:

[0057] S241: parsing the label file corresponding to the target detection result to obtain target label information, wherein the target label information includes the location and category information of each target;

[0058] S242: Segment the initial archaeological image according to the location and category information of the target in the target label information to obtain the segmented archaeological image, and save the segmented archaeological image in a preset folder.

[0059] Specifically, in the backend / segmentimg route, the backend will look for the label folder (labels) in the predict{page_id} directory. This folder stores the target information after the YOLO model detects the image, and each detected object will have a corresponding .txt file. The backend calls the process_image function to process the target detection results. The first step is to parse the label file corresponding to the target detection result. The backend reads and parses the label file (.txt file) generated by the YOLO model, that is, to generate the target label information. The target label information contains the location, category and other information of each target. The next step is to segment the image. According to the information in the target label information, the area containing the target in the image is intercepted, and each target area is extracted and saved as a new image, that is, the segmented archaeological image is obtained (for example, the image of each target is saved separately).

[0060] Furthermore, the segmented archaeological images will be saved in the Result_single folder. This folder contains each segmented archaeological image, usually saved in PNG or JPG format. Finally, the backend will traverse the image files in the Result_single folder, package the URL address of each image (generated by url_for) into a JSON response and return it to the frontend as the segmentation result.

[0061] See also Figure 5 , Figure 5 A specific implementation method before step S2 is shown, which is described in detail as follows:

[0062] S201: Acquire a sample archaeological image, and perform image preprocessing on the sample archaeological image to generate a preprocessed archaeological image;

[0063] S202: construct a configuration file, and write the dataset path, category data and category information in the pre-processed archaeological image into the configuration file;

[0064] S203: Perform model training on the YOLO model based on the preprocessed archaeological image and the configuration file according to a preset number of iterations to generate the trained YOLO model.

[0065] Specifically, sample archaeological images are obtained, wherein the sample archaeological images can be artifact images (such as pottery jars, stone axes, jade rings, etc.) and six tomb images commonly used in archaeological reports. The sample archaeological images are image preprocessed to generate preprocessed archaeological images. Image preprocessing may include image cropping and image enhancement, etc. A training data set is constructed based on the preprocessed archaeological images, a configuration file is constructed, and the data set path, category data and category information in the preprocessed archaeological images are written into the configuration file; the YOLO model is trained based on the preprocessed archaeological images and the configuration file according to a preset number of iterations to generate the trained YOLO model.

[0066] In a specific embodiment, the training data set includes 2,400 annotated images from multiple archaeological reports. By constructing a yaml configuration file, the training data set path, number of categories, and category information are written into it, and the pre-trained weights of YOLO v10 are loaded in the model training, and the training is completed within 100 epochs. The trained model can identify and annotate information such as archaeological artifacts and tomb locations in the archaeological report.

[0067] S3: Based on the prediction frame, perform OCR recognition on the segmented archaeological image to generate target annotation and serial number information, and format the segmented archaeological image according to the target annotation and the serial number information to obtain the target archaeological image information.

[0068] In an embodiment of the present application, Paddle OCR is used to perform OCR recognition on the segmented archaeological image based on the prediction frame, generate target annotations and serial number information, and format the segmented archaeological image according to the target annotations and the serial number information to obtain the target archaeological image information. Since Paddle OCR has high-precision Chinese recognition capabilities, it is particularly suitable for situations where archaeological documents contain a large number of Chinese annotations. Therefore, in an embodiment of the present application, Paddle OCR is used to perform OCR recognition, generate target annotations and serial numbers, and then name the segmented archaeological image in the format of "target annotation + serial number information".

[0069] See also Figure 6 and Figure 7 , Figure 6 A specific implementation of step S3 is shown. Figure 7 This is a schematic diagram of generating target archaeological image information provided by an embodiment of the present application, which is described in detail as follows:

[0070] S31: Taking the overall frame as a reference, extracting elements in the overall frame to obtain target elements;

[0071] S32: performing OCR recognition on the segmented archaeological image of the legend frame and the serial number frame to generate the target legend and the serial number information;

[0072] S33: If there is text information in the target caption, format the segmented archaeological image according to the text information and the sequence number information to obtain the target archaeological image information;

[0073] S34: If the text information does not exist in the target annotation, the text information in the previous target annotation and the serial number information are used to format the segmented archaeological image to obtain the target archaeological image information.

[0074] Specifically, taking the overall frame as a reference, the elements in the overall frame are extracted to obtain the target elements, wherein the target elements include elements such as captions, serial numbers and artifacts. Paddle OCR is then used to perform OCR recognition on the caption frame and the serial number frame on the segmented archaeological image to generate the target caption and serial number information. If there is text information in the target caption, the segmented archaeological image is formatted and named according to the text information and serial number information to obtain the target archaeological image information. If there is no text information in the target caption, the segmented archaeological image is formatted and named according to the text information and serial number information in the previous target caption to obtain the target archaeological image information. Among them, the target archaeological image information includes information such as the segmented archaeological image and naming information, as shown in the figure. Figure 7 shown.

[0075] See also Figures 8 to 13 , Figure 8 Another specific implementation of the embodiment of the present application is shown; Fig. 9 is a schematic diagram of a mask displayed when the mouse is hovered over the preview area provided in an embodiment of the present application; Fig.10 is a schematic diagram of a blue anchor point set by clicking the left button of a mouse provided in an embodiment of the present application; Fig.11 is a schematic diagram of a red anchor point set by right-clicking a mouse provided in an embodiment of the present application; Fig.12 It is a schematic diagram of the index input box in the lower right corner of the object card for changing the order of the object card provided in the embodiment of the present application; Fig.13 The schematic diagram of the title input box for changing the title of the artifact card above the artifact card provided in the embodiment of the present application is described in detail as follows:

[0076] S101: When the user marks the initial archaeological image on the front-end interface, the marking position and marking type are obtained;

[0077] S102: using a SAM model to segment the initial archaeological image based on the marker position and the marker type, and generating a mask array;

[0078] S103: drawing the mask array in the preview area of ​​the front-end interface through canvas.

[0079] Specifically, the above steps are to obtain the required artifact map through automatic segmentation of the yolo model, but in order to deal with some complex or personalized situations, the embodiment of the present application provides a manual segmentation module, and the embodiment of the present application adopts the SegmentAnything model (SAM model) for image segmentation. Among them, SegmentAnything is an image segmentation module based on deep learning, which combines the front-end and back-end components, allowing users to finely segment the image through interactive selection (click or drag). In an embodiment of the present application, after the user clicks on certain points on the initial archaeological image with the mouse on the front-end interface to mark the area to be segmented, the front-end sends the mark position and mark type to the back-end, and the back-end uses the SAM model to perform image segmentation on the initial archaeological image based on the mark position and mark type to generate a mask array. In order to reduce bandwidth usage, the mask data returned by the back-end is compressed and returned to the front-end through the API. After the front-end receives the segmentation result, the mask is drawn on the initial archaeological image using canvas, and the effect of each segmented area in the image can be seen.

[0080] like Fig. 9 As shown in the figure, whenever the user hovers the mouse over a point on the area to be segmented in the preview area, the system will automatically identify the area and generate a mask for the area. As long as the mouse is in the preview area, the front end will send the current mouse position information to the back end at a certain frequency. The back end will use the position information to segment the image to be segmented stored in the back end based on the SegmentAnything model, and return a mask array to the front end. The front end will use the canvas to draw the mask on the preview area based on the mask array, and draw the mask image in real time according to the movement of the user's mouse position.

[0081] like Fig.10 and Fig.11 As shown, the generated mask is the auxiliary information provided to the user by the embodiment of the present application. The user can click the left mouse button to fix the current mask, which is fixed by the blue anchor point and will not change with the mouse movement. At the same time, the preview area will be allowed to generate the next mask. Taking into account the complexity of the image to be segmented, in order to enhance the accuracy of mask generation, the user can click the right mouse button to set a red anchor point, which will set a negative weight for the nearby area, greatly reducing the possibility of the area being predicted by the model, that is, inhibiting the mask generation of the area. For the setting operation of the red and blue anchor points, the embodiment of the present application provides the user with the opportunity of redo and undo, which can revoke the anchor points that have been set and trace back the revoked anchor points. After the user adds several blue anchor points and red anchor points to obtain a satisfactory mask area, the mask area image can be segmented after the user confirms the segmentation, and the top of the tool area is added. At the same time, all masks in the preview area will be cleared.

[0082] Furthermore, if Figure 12 to Figure 13 As shown, users can display and drag the segmented images, sort them, clear and save images, rename images, etc. on the front-end interface.

[0083] In the front-end page of the embodiment of the present application, the VueDraggable component is used, which is a draggable list component encapsulated by the Vue component library vuedraggable. VueDraggable is bound to the cutOuts array, and each VueDraggable component corresponds to an element in the cutOuts array, which is rendered in the form of an image card through a v-for loop, where each image card item can display a title and can be clicked to edit (such as modifying the title name). These image cards allow the user to change the index of the array elements for reordering by dragging the list items. After dragging is completed, the order of the cutOuts array will be updated, and the order of the images will be displayed according to the new arrangement. In addition, you can click the input box in the lower right corner, such as Fig.12 As shown, directly enter the index of the card to change the position of the card.

[0084] In another specific embodiment, the title of each image item is generated by the ImageName method, which generates a short name based on the name field of the image and appends an index value to the name. When the user drags the image, VueDraggable automatically handles the order of elements in the array. Changing the order of the array changes the index value accordingly, so the second half of the title of the image card will also change in real time. Fig.13 As shown in the figure, by clicking on the image card title, the title option will become an input box, in which the user can freely edit the title of each image, and the system will update the image name in the cutOuts array in real time. Among them, the isEditing method is used to determine whether the current image is in the editing state. When the title input box is clicked, the startEditing method will be called to assign the parameter editingItem to the current item. If editingItem is equal to the current image, the image is in the editing state. After entering the editing mode, the value of the input box is initialized to the name of the image, and the input box is focused. The user can modify the text in the input box. When pressing Enter or losing focus, the saveEditing method will be triggered to save the modified name and update the corresponding item in the cutOuts array.

[0085] In another specific embodiment, the user can save each image to a specified folder by naming it with the title on the image card. When the user confirms to save, all image cards in the current workspace will be cleared for the user to perform the next segmentation operation. Each time you save, the system will download each image through an HTTP request and save it as a .png file with a new name. At the same time, the user page can clear all currently edited and sorted images and restore them to the initial state. The clearImages method will pop up a confirmation box, asking the user whether to confirm to clear all images. If the user confirms, the cutOuts array will be cleared and all images will be removed. After clearing, the cutOuts array is saved to local storage so that it can be restored to its original state when the page is refreshed.

[0086] In an embodiment of the present application, an archaeological file to be processed is obtained, and the archaeological file to be processed is converted into image information to obtain an initial archaeological image; target detection is performed on the initial archaeological image through a trained YOLO model to obtain a target detection result, and image segmentation is performed according to the target detection result to obtain a segmented archaeological image, wherein the target detection result includes a plurality of prediction boxes; OCR recognition is performed on the segmented archaeological image based on the prediction box to generate a target annotation and serial number information, and the segmented archaeological image is formatted and named according to the target annotation and serial number information to obtain target archaeological image information. The embodiment of the present invention can accurately segment out the archaeological image and accurately annotate the segmented archaeological image by converting the archaeological file to be processed into an archaeological image, and combining the target detection and OCR recognition of the YOLO model, and does not require human intervention, thereby effectively improving the efficiency of archaeological information extraction.

[0087] In the application embodiment, the whole process from report analysis to segmentation and annotation is automated, which reduces human intervention and improves processing efficiency. By using YOLO and Segment Anything models for multi-level segmentation and combining mouse annotation information, a more refined image segmentation effect is achieved. The embodiment of the application provides intuitive editing and saving functions, making it easier to adjust and save image processing results.

[0088] Please refer to Fig.14 , as a response to the above Figure 1 The present application provides an embodiment of an archaeological information extraction device based on deep learning, and the device embodiment is similar to Figure 1 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0089] like Fig.14As shown, the archaeological information extraction device based on deep learning in this embodiment includes: an archaeological file acquisition unit 41, a target detection unit 42 and an image naming unit 43, wherein:

[0090] The archaeological file acquisition unit 41 is used to acquire the archaeological file to be processed and convert the archaeological file to be processed into image information to obtain an initial archaeological image;

[0091] The target detection unit 42 is used to perform target detection on the initial archaeological image through the trained YOLO model to obtain a target detection result, and perform image segmentation according to the target detection result to obtain a segmented archaeological image, wherein the target detection result includes a plurality of prediction boxes;

[0092] The image naming unit 43 is used to perform OCR recognition on the segmented archaeological image based on the prediction box, generate target annotation and serial number information, and format the segmented archaeological image according to the target annotation and the serial number information to obtain the target archaeological image information.

[0093] Furthermore, the archaeological document acquisition unit 41 includes:

[0094] The archaeological file transfer unit is used to transfer the archaeological file to be processed to the back end in the form of FormData through the front end when the user uploads the archaeological file to be processed;

[0095] A request information acquisition unit, used to acquire the number of pages of the archaeological file to be processed, and return the number of pages and the file name of the archaeological file to be processed to the front end, so as to obtain the request information of the front end for the archaeological file to be processed;

[0096] An image conversion unit is used to convert the archaeological file to be processed in the request information into image information to obtain the initial archaeological image.

[0097] Further, the target detection unit 42 includes:

[0098] A path verification unit, used to verify the image path of the initial archaeological image;

[0099] A prediction directory generating unit, used for generating a prediction directory according to the number of pages and the file name of the archaeological file to be processed if the image path verification is valid;

[0100] A target detection result generating unit, configured to identify and extract coordinates of line elements of the initial archaeological image based on the prediction catalog by using the trained YOLO model, and generate the target detection result;

[0101] An image segmentation unit is used to perform image segmentation according to the target detection result to obtain the segmented archaeological image.

[0102] Furthermore, the image segmentation unit comprises:

[0103] A label file parsing unit, used to parse the label file corresponding to the target detection result to obtain target label information, wherein the target label information includes the location and category information of each target;

[0104] The segmented archaeological image generating unit is used to segment the initial archaeological image according to the position and category information of the target in the target label information to obtain the segmented archaeological image, and save the segmented archaeological image in a preset folder.

[0105] Furthermore, the device before the target detection unit 42 also includes:

[0106] A sample archaeological image acquisition unit, used to acquire a sample archaeological image and perform image preprocessing on the sample archaeological image to generate a preprocessed archaeological image;

[0107] A configuration file construction unit, used to construct a configuration file, and write the dataset path, category data and category information in the pre-processed archaeological image into the configuration file;

[0108] The model training unit is used to perform model training on the YOLO model based on the preprocessed archaeological image and the configuration file according to a preset number of iterations to generate the trained YOLO model.

[0109] Further, the image naming unit 43 includes:

[0110] An element extraction unit, used to extract elements in the overall frame based on the overall frame to obtain a target element;

[0111] An OCR recognition unit is used to perform OCR recognition on the segmented archaeological image of the legend frame and the serial number frame to generate target legend and serial number information;

[0112] A first format naming unit is used for, if there is text information in the target caption, formatting the segmented archaeological image according to the text information and the sequence number information to obtain the target archaeological image information;

[0113] The second format naming unit is used to format the segmented archaeological image using the text information and serial number information in the previous target annotation if the text information does not exist in the target annotation to obtain the target archaeological image information.

[0114] Furthermore, the archaeological information extraction device based on deep learning also includes:

[0115] A marking unit, used for obtaining a marking position and a marking type when a user marks the initial archaeological image on a front-end interface;

[0116] A mask array generating unit, used for performing image segmentation on the initial archaeological image based on the marker position and the marker type using a SAM model to generate a mask array;

[0117] The drawing unit is used to draw the mask array in the preview area of ​​the front-end interface through canvas.

[0118] To solve the above technical problems, the present application also provides a computer device. Fig.15 , Fig.15 This is a basic structural block diagram of the computer device in this embodiment.

[0119] The computer device 5 includes a memory 51, a processor 52, and a network interface 53 which are interconnected through a system bus. It should be noted that: Fig.15 Only a computer device 5 having three components, memory 51, processor 52, and network interface 53, is shown, but it should be understood that it is not required to implement all the components shown, and more or fewer components may be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), digital signal processors (DSP), embedded devices, etc.

[0120] Computer devices can be computing devices such as desktop computers, notebooks, PDAs, and cloud servers. Computer devices can interact with users through keyboards, mice, remote controls, touch pads, or voice control devices.

[0121] The memory 51 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 51 can be an internal storage unit of the computer device 5, such as a hard disk or memory of the computer device 5. In other embodiments, the memory 51 can also be an external storage device of the computer device 5, such as a plug-in hard disk, a smart memory card (SmartMedia Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the computer device 5. Of course, the memory 51 can also include both the internal storage unit of the computer device 5 and its external storage device. In this embodiment, the memory 51 is generally used to store the operating system and various application software installed on the computer device 5, such as the program code of the archaeological information extraction method based on deep learning, etc. In addition, the memory 51 can also be used to temporarily store various types of data that have been output or are to be output.

[0122] The processor 52 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chips in some embodiments. The processor 52 is generally used to control the overall operation of the computer device 5. In this embodiment, the processor 52 is used to run the program code stored in the memory 51 or process data, such as running the program code of the above-mentioned archaeological information extraction method based on deep learning, so as to implement various embodiments of the archaeological information extraction method based on deep learning.

[0123] The network interface 53 may include a wireless network interface or a wired network interface, and the network interface 53 is generally used to establish a communication connection between the computer device 5 and other electronic devices.

[0124] The present application also provides another embodiment, namely, providing a computer-readable storage medium, which stores a computer program, and the computer program can be executed by at least one processor to enable the at least one processor to perform the steps of an archaeological information extraction method based on deep learning as described above.

[0125] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods of each embodiment of the present application.

[0126] Obviously, the embodiments described above are only some embodiments of the present application, rather than all embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application is described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned specific implementation methods, or to perform equivalent replacement of some of the technical features therein. Any equivalent structure made using the contents of the present application specification and the accompanying drawings, directly or indirectly used in other related technical fields, is similarly within the scope of protection of the present application.

Claims

1. A method for extracting archaeological information based on deep learning, characterized in that: include: Acquire the archaeological file to be processed, and convert the archaeological file to be processed into image information to obtain an initial archaeological image; Performing target detection on the initial archaeological image through the trained YOLO model to obtain a target detection result, and performing image segmentation according to the target detection result to obtain a segmented archaeological image, wherein the target detection result includes multiple prediction boxes; Based on the prediction frame, the segmented archaeological image is subjected to OCR recognition to generate target annotation and serial number information, and the segmented archaeological image is formatted and named according to the target annotation and serial number information to obtain target archaeological image information; The prediction frame includes an overall frame, a line drawing frame, a serial number frame and a caption frame; based on the prediction frame, the segmented archaeological image is subjected to OCR recognition to generate target captions and serial number information; the segmented archaeological image is formatted and named according to the target captions and the serial number information to obtain target archaeological image information, including: Taking the overall frame as a reference, extracting elements in the overall frame to obtain target elements; Perform OCR recognition on the segmented archaeological image for the caption frame and the serial number frame to generate target caption and serial number information; If there is text information in the target caption, the segmented archaeological image is formatted and named according to the text information and the sequence number information to obtain the target archaeological image information; If the text information does not exist in the target annotation, the segmented archaeological image is formatted and named using the text information and serial number information in the previous target annotation to obtain the target archaeological image information, wherein the target archaeological image information includes the segmented archaeological image and the naming information.

2. The archaeological information extraction method based on deep learning according to claim 1 is characterized in that: Acquiring an archaeological file to be processed, and converting the archaeological file to be processed into image information to obtain an initial archaeological image, including: When the user uploads the archaeological file to be processed, the archaeological file to be processed is transmitted to the backend in the form of FormData through the frontend; Obtain the number of pages of the archaeological file to be processed, and return the number of pages and the file name of the archaeological file to be processed to the front end to obtain the request information of the front end for the archaeological file to be processed; The archaeological file to be processed in the request information is converted into image information to obtain the initial archaeological image.

3. The archaeological information extraction method based on deep learning according to claim 2 is characterized in that: Performing target detection on the initial archaeological image through the trained YOLO model to obtain a target detection result, and performing image segmentation according to the target detection result to obtain a segmented archaeological image, including: Image path verification of initial archaeological images; If the image path verification is valid, a prediction directory is generated according to the number of pages and the file name of the archaeological file to be processed; Identify and extract coordinates of line elements of the initial archaeological image based on the prediction catalog by using the trained YOLO model to generate the target detection result; Image segmentation is performed according to the target detection result to obtain the segmented archaeological image.

4. The archaeological information extraction method based on deep learning according to claim 3 is characterized in that: Performing image segmentation according to the target detection result to obtain the segmented archaeological image includes: Parsing the label file corresponding to the target detection result to obtain target label information, wherein the target label information includes the location and category information of each target; The initial archaeological image is segmented according to the location and category information of the target in the target label information to obtain a segmented archaeological image, and the segmented archaeological image is saved in a preset folder.

5. The archaeological information extraction method based on deep learning according to claim 1 is characterized in that: Performing target detection on the initial archaeological image by using the trained YOLO model to obtain a target detection result, and performing image segmentation according to the target detection result to obtain the segmented archaeological image, the method further includes: Acquire a sample archaeological image, and perform image preprocessing on the sample archaeological image to generate a preprocessed archaeological image; Constructing a configuration file, and writing the dataset path, category data and category information in the pre-processed archaeological image into the configuration file; The YOLO model is trained based on the preprocessed archaeological image and the configuration file according to a preset number of iterations to generate the trained YOLO model.

6. The archaeological information extraction method based on deep learning according to any one of claims 1 to 5, characterized in that: The method also includes: When the user marks the initial archaeological image on the front-end interface, the marking position and marking type are obtained; Using the SAM model to segment the initial archaeological image based on the marker position and the marker type to generate a mask array; The mask array is drawn in the preview area of ​​the front-end interface through canvas.

7. An archaeological information extraction device based on deep learning, characterized in that: include: An archaeological file acquisition unit is used to acquire the archaeological file to be processed and convert the archaeological file to be processed into image information to obtain an initial archaeological image; A target detection unit is used to perform target detection on the initial archaeological image through a trained YOLO model to obtain a target detection result, and perform image segmentation according to the target detection result to obtain a segmented archaeological image, wherein the target detection result includes a plurality of prediction boxes; An image naming unit is used to perform OCR recognition on the segmented archaeological image based on the prediction frame, generate a target caption and serial number information, and format the segmented archaeological image according to the target caption and the serial number information to obtain target archaeological image information; The prediction frame includes an overall frame, a line drawing frame, a sequence number frame and a legend frame; the image naming unit includes: An element extraction unit, used to extract elements in the overall frame based on the overall frame to obtain a target element; An OCR recognition unit is used to perform OCR recognition on the segmented archaeological image of the legend frame and the serial number frame to generate target legend and serial number information; A first format naming unit is used for, if there is text information in the target caption, formatting the segmented archaeological image according to the text information and the sequence number information to obtain the target archaeological image information; The second format naming unit is used to format the segmented archaeological image using the text information and serial number information in the previous target annotation if the text information does not exist in the target annotation, so as to obtain the target archaeological image information, wherein the target archaeological image information includes the segmented archaeological image and the naming information.

8. A computer device, characterized in that: It includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, it implements the archaeological information extraction method based on deep learning as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the archaeological information extraction method based on deep learning as described in any one of claims 1 to 6.

Images