A deep learning-based book cover and spine joint identification borrowing method

Abstract

The application discloses a kind of based on deep learning's book cover and book spine joint identification borrowing method, comprising the following steps: acquisition cover, book spine and binding image and pre-processing, get the image group of cover spine;Detect cover, book spine, binding line, thickness boundary and label area, get position record;According to position record, extract cover spine recognition unit;With candidate collection build node to recognition unit, by projection, field, thickness pairing gets entity graph;With binding line correction node input, by improved Equiformer gets coding graph;Shield mismatch cross face attention edge, to the candidate result that reserved edge propagation is kept;According to candidate result generation residual, combined with available state output borrowing or review record.The application relates to the technical field of image recognition and library borrowing management, improves the accuracy of library borrowing identification, reduces false borrowing and manual review amount.

Description

Technical Field

[0001] This invention relates to the fields of image recognition and book lending management technology, and in particular to a method for borrowing books based on the joint recognition of book covers and spines using deep learning. Background Technology

[0002] Current book lending management methods mostly rely on barcode scanning, RFID reading, single-cover recognition, or spine text recognition to confirm the library's holdings. Barcode scanning and RFID reading depend on the integrity of the tags; damaged, obscured, or detached tags can affect the lending confirmation results. Single-cover recognition focuses on the cover design, title text, and layout features, while spine text recognition focuses on the text arrangement within a narrow area and library tag information. Both methods are often compared based on independent recognition results, lacking a joint representation of the physical spatial relationships between the cover, spine, binding area, and spine thickness.

[0003] Existing deep learning-based book cover and spine recognition methods, when handling scenarios involving joint book cover and spine borrowing, tend to overlook factors such as the correspondence of binding hinges, consistency between cover and spine side fields, thickness differences between different book versions, and the borrowability status of candidate library holdings. When cover and spine images exhibit shooting angle shifts, partial occlusion, similar covers, or multiple versions of the same book, simple feature stitching or confidence ranking struggles to reliably distinguish target library holdings. Existing methods lack node pairing based on spine entity images, thickness version pairing, cross-face attention edge masking, and residual closure judgment, leading to increased errors in borrowing, missed borrowing, and manual verification. Therefore, providing a deep learning-based method for joint book cover and spine recognition and borrowing is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0004] One objective of this invention is to propose a method for joint identification and borrowing of book covers and spines based on deep learning. This invention utilizes image recognition, cover and spine entity images, and an improved Equiformer structure to achieve joint identification of the cover, spine, binding area, thickness, and collection status. It has the advantages of accurate identification, low mis-borrowing rate, low verification volume, and high borrowing efficiency.

[0005] A method for jointly recognizing and borrowing book covers and spines based on deep learning according to an embodiment of the present invention includes the following steps: Collect images of the cover, spine, and binding area of ​​the books to be borrowed, and perform preprocessing on the collected images to obtain a set of spine images; Region detection was performed on the spine image group to mark the cover area, spine area, binding hinge line, spine thickness boundary and collection label area, and the spine position record was obtained; Extract the ridge seal identification unit based on the ridge seal location record; Register the spine identification unit and candidate collection books as nodes, perform binding line projection pairing and field label pairing on the spine identification unit, perform version thickness pairing on the spine identification unit and candidate collection books, connect the paired nodes as graph edges, and obtain the spine entity graph. Input the spine solid diagram into the improved Equiformer structure, calculate the node projection number, node area mark and thickness direction mark based on the binding hinge line, and use the node projection number, node area mark and thickness direction mark to correct the node input vector of the spine solid diagram to obtain the spine coding diagram. In the spine coding diagram, cross-face attention edges are established between the cover side node and the spine side node. Cross-face attention edges with mismatched binding line projections, inconsistent field labels, or thickness results that do not pass the version thickness pairing are masked. Cross-face attention edges without masking are propagated by the same graph to obtain the propagation results of candidate library collection books. Based on the dissemination results of candidate library books, a spine identification residual is generated. Based on the spine identification residual and the borrowability status of candidate library books, the target library books are determined and a borrowing record is generated. If the target library books are not determined, a review record is generated.

[0006] Optionally, the operation of obtaining the ridge image group includes: Distortion correction, brightness normalization, and size normalization are performed on the cover image, spine image, and binding area image respectively. The preprocessed cover image, spine image, and binding area image are then grouped into a spine image set of the same book to be borrowed.

[0007] Optionally, the operation of obtaining the ridge cap position record includes: Mark the cover area within the cover image in the spine image group; Mark the spine region, spine thickness boundary, and collection label region within the spine image group; Mark the binding hinge line within the binding area image in the spine image group; The locations of the cover area, spine area, binding hinge line, spine thickness boundary, and collection label area are grouped into a spine location record.

[0008] Optionally, the operation of extracting the ridge identification unit based on the ridge position record includes: Based on the cover area, spine area, binding hinge line, spine thickness boundary, and collection label area recorded in the spine position record, the cover recognition unit, spine recognition unit, binding recognition unit, thickness recognition unit, and collection label unit are extracted respectively, and the cover recognition unit, spine recognition unit, binding recognition unit, thickness recognition unit, and collection label unit are grouped into a spine recognition unit.

[0009] Optionally, the operation of registering the spine identification unit and candidate library books as nodes includes: Convert the ridge seal recognition unit into a recognition node; Perform a library collection search on the collection label unit, cover recognition unit, and spine recognition unit to obtain candidate library collection books; Candidate books in the library collection are converted into library collection nodes, and the identification nodes and library collection nodes are used as nodes in the spine entity graph.

[0010] Optionally, the operation of connecting paired nodes as graph edges includes: Extract the cover side node and spine side node from the identification node, and form projection intervals according to the binding hinge line direction. Mark the cover side node and spine side node that overlap in the projection interval as binding line projection pairing. Extract the field tags from the cover side node and the spine side node respectively, and mark the cover side node and spine side node with the same field tags as field tag pairing passed; Based on the thickness recognition unit, the thickness results are obtained. The version thickness range of candidate books in the collection database is read. Candidate books whose thickness results fall within the version thickness range are marked as having passed version thickness matching. Connect the cover side node and spine side node that have passed the binding line projection and field mark pairing to form the cover spine image edge. Connect the recognition node and collection node corresponding to the thickness recognition unit that have passed the version thickness pairing to form the thickness version image edge. The spine entity image is formed by the recognition node, collection node, cover spine image edge and thickness version image edge.

[0011] Optionally, the operation of obtaining the ridge coding map includes: Using the binding hinge line as the projection reference, the identification nodes in the spine solid drawing are projected along the direction of the binding hinge line, and the node projection sequence number is configured according to the order of the projection positions. Configure node area markers based on the cover area, spine area, binding hinge line, spine thickness boundary, or collection label area corresponding to the identification node; Based on the inner and outer positions of the identification nodes relative to the thickness boundary of the spine and the thickness extension direction, thickness direction markers are configured; the original node features of the identification nodes and collection nodes are extracted, and the original node features are combined with the node projection number, node region marker, and thickness direction marker to form the corrected node input vector; The modified node input vector is input into the improved Equiformer structure, and equivariant coding is performed to obtain the ridge coding map.

[0012] Optionally, the operation of masking the cross-surface attention edge includes: Extract the cover side nodes corresponding to the cover area from the spine coding diagram, and extract the spine side nodes corresponding to the spine area from the node region markers. Establish a cross-surface attention edge between the cover side node and the spine side node; Read the binding line projection pairing results and field label pairing results corresponding to the nodes at both ends of the cross-face attention edge, and read the version thickness pairing results of the collection nodes associated with the cross-face attention edge; When the binding line projection pairing result is unsuccessful, the field mark pairing result is unsuccessful, or the version thickness pairing result is unsuccessful, the corresponding cross-surface attention edge will be marked as a shielded edge. When the binding line projection pairing result, field mark pairing result, and version thickness pairing result are all passed, the corresponding cross-face attention edge will be marked as a reserved edge; Remove the shielded edges from the attention calculation range of the improved Equiformer structure, and retain the edges within the attention calculation range of the improved Equiformer structure.

[0013] Optionally, the operation of obtaining the dissemination results of candidate library collection books includes: In the attention computation range of the improved Equiformer structure, retain the edges and stop feature transfer between the nodes at both ends of the masked edge; Perform isomorphic graph attention propagation on the cover side nodes and spine side nodes at both ends of the preserved edge to obtain cross-surface propagation features; The cross-surface propagation features, the node features of the corresponding recognition nodes of the thickness recognition unit, and the node features of the corresponding recognition nodes of the collection tag unit are aggregated into the collection node to form the propagation results of candidate collection books.

[0014] Optionally, the operation of determining the target library book based on the spine identification residual and the borrowability status of candidate library books includes: Extract the propagation matching component of the collection node from the propagation results of the candidate collection books, read the binding line projection pairing result, field label pairing result and version thickness pairing result corresponding to the collection node, and generate the spine recognition residual based on the propagation matching component, binding line projection pairing result, field label pairing result and version thickness pairing result; Candidate books whose spine recognition residuals meet the preset residual thresholds and whose borrowable status is available are identified as target books. Borrowing records are generated based on the target books. For candidate books whose spine recognition residuals do not meet the preset residual threshold or whose borrowable status is not borrowable, a review record will be generated.

[0015] The beneficial effects of this invention are: (1) The present invention forms a spine image group by collecting cover image, spine image and binding area image, and extracts spine recognition unit based on spine position record, so that the book recognition process no longer depends on a single cover image, a single spine text or external label, thus improving the integrity of book identity confirmation.

[0016] (2) The present invention establishes a spine entity map by binding line projection pairing, field mark pairing and version thickness pairing, and associates the cover side node, spine side node, thickness recognition unit and candidate library books to reduce the probability of misidentification in scenarios with similar covers, multiple versions of books with the same name and label occlusion.

[0017] (3) This invention improves the Equiformer structure to correct the node input vector and performs masking on cross-face attention edges that do not meet the pairing conditions, so that the attention propagation of the isomorphic graph is focused on the effective cover spine association; combined with the cover spine recognition residual and the borrowable status to determine the target library books, the amount of manual verification is reduced and the borrowing confirmation efficiency is improved. Attached Figure Description

[0018] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is a flowchart of a deep learning-based method for jointly recognizing and borrowing books based on their cover and spine, as proposed in this invention. Figure 2 This is a schematic diagram of the construction of the spine entity image for a book cover and spine joint recognition and borrowing method based on deep learning proposed in this invention; Figure 3 This is a schematic diagram illustrating the cross-surface attention propagation of a deep learning-based book cover and spine joint recognition and borrowing method proposed in this invention. Detailed Implementation

[0019] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0020] refer to Figures 1-3 A method for jointly recognizing and borrowing books based on deep learning through the combined recognition of book covers and spines includes the following steps: The system acquires images of the cover, spine, and binding area of ​​the book to be borrowed. Preprocessing is then performed on the acquired images to obtain a set of spine and cover images. During acquisition, the book is placed within the borrowing identification area, and image data is acquired from the cover orientation, spine orientation, and binding edge orientation. The cover image records the cover text, cover design, and cover layout; the spine image records the spine text, spine band, and library label; and the binding area image records the binding boundary between the cover and spine. After preprocessing, the cover, spine, and binding area images are bound to the same borrowing identification number, forming a set of spine and cover images. If image acquisition fails, image clarity is below a preset clarity threshold, or the book boundary is not fully captured, the borrowing identification number is written to the verification record.

[0021] Region detection is performed on the spine image set to calibrate the cover region, spine region, binding hinge line, spine thickness boundary, and collection label region, resulting in a spine position record. During region detection, the cover region is determined in the cover image, the spine region, spine thickness boundary, and collection label region are determined in the spine image, and the binding hinge line is determined in the binding region image. The spine position record includes the borrowing identification number, region category, region boundary coordinates, line segment endpoint coordinates, region source image, region confidence value, and region detection status. If any of the cover region, spine region, binding hinge line, spine thickness boundary, or collection label region is not calibrated, subsequent automatic borrowing confirmation is stopped, and a review record containing the failed region category is generated.

[0022] The spine identification unit is extracted based on the spine location record. During extraction, corresponding image blocks are extracted from the spine image group according to the location range of each area in the spine location record. Text recognition, texture extraction, boundary extraction, and label recognition are then performed on the extracted image blocks to form cover identification units, spine identification units, binding identification units, thickness identification units, and collection label units. The spine identification unit includes identification unit number, identification unit type, source region, image block location, image features, text field, field confidence value, boundary features, label recognition result, and extraction status. When the text field, collection label result, or thickness result is empty, the corresponding borrowing identification number is written to the verification record.

[0023] The spine identification unit and candidate books are registered as nodes. Binding line projection and field labeling are paired for the spine identification unit, and version thickness is paired between the spine identification unit and the candidate books. Nodes that pass the pairing are connected as graph edges to obtain the spine entity graph. During node registration, the spine identification unit is converted into an identification node, and the candidate books returned from the collection database are converted into collection nodes. The spine entity graph includes a node set and a graph edge set. The node set includes identification nodes and collection nodes, and the graph edge set includes cover spine graph edges, thickness and version graph edges, and candidate collection association edges. When the collection database does not return any candidate books, a record with no candidate books for review is generated; when multiple candidate books are returned, an independent collection node is created for each candidate book.

[0024] The spine cover entity image is input into the improved Equiformer structure. Using the binding hinge line as a reference, the node projection number, node region label, and thickness direction label are calculated. These elements are then used to correct the node input vector of the spine cover entity image, resulting in the spine cover coded image. The improved Equiformer structure reads the node set, edge set, and corrected node input vector from the spine cover entity image. Isovariant coding is performed on the identified nodes and collection nodes, outputting the spine cover coded image containing updated node features and edge structures. If the node projection number cannot be calculated, the node region label is missing, or the thickness direction label is missing, the corresponding node is marked as an abnormal node and added to the review record.

[0025] Cross-face attention edges are established between cover-side nodes and spine-side nodes in the spine coding diagram. Cross-face attention edges with mismatched binding line projections, inconsistent field labels, or thickness results not matched with version thickness are masked. Unmasked cross-face attention edges undergo isomorphic graph attention propagation to obtain the propagation results for candidate library collection books. When establishing cross-face attention edges, nodes whose node region labels correspond to the cover region are selected as cover-side nodes, and nodes whose node region labels correspond to the spine region are selected as spine-side nodes. Cross-face attention edges are marked as masked or retained edges based on the binding line projection matching results, field label matching results, and version thickness matching results. Masked edges do not enter the propagation edge set, while retained edges enter the attention calculation scope of the improved Equiformer structure. When no retained edges are found, a record with no valid cross-face association is generated for review.

[0026] Based on the dissemination results of candidate collection books, a spine identification residual is generated. The target collection book is determined based on the spine identification residual and the borrowability status of the candidate collection books, and a borrowing record is generated. If the target collection book is not determined, a review record is generated. When generating the spine identification residual, the dissemination matching component, binding line projection pairing result, field tag pairing result, and version thickness pairing result of the collection node are read to form dissemination residual, projection residual, field residual, and thickness residual, which are then merged into the spine identification residual. If the spine identification residual meets a preset residual threshold and the borrowability status of the candidate collection book is available, the candidate collection book is determined as the target collection book, and a borrowing record is generated. If the spine identification residual does not meet the preset residual threshold, the borrowability status of the candidate collection book is not available, or the residuals of multiple candidate collection books are close, a review record is generated.

[0027] In this embodiment, the operation to obtain the spine image group includes: Distortion correction, brightness normalization, and size normalization are performed on the cover image, spine image, and binding area image respectively. The preprocessed cover image, spine image, and binding area image are then grouped into a single spine image set for the same book to be borrowed. During distortion correction, the outer contour boundary of the book in the image is identified, and perspective distortion caused by the acquisition perspective is corrected according to the outer contour boundary. During brightness normalization, the grayscale distribution of the cover image, spine image, and binding area image is statistically analyzed, and the three types of images are adjusted to the same brightness range. During size normalization, the three types of images are scaled according to a preset input size while maintaining the proportion of the book boundaries. During grouping, the preprocessed cover image, spine image, and binding area image are associated with the same borrowing identification number to form a spine image set. The preset input size and brightness range are obtained from statistical analysis of the library's sample images or configured by the borrowing management system. All three types of images in the spine image group share the same borrowing identification number, and are categorized as cover image, spine image, and binding area image, respectively. Image preprocessing can employ one or more combinations of perspective transformation, affine transformation, histogram equalization, grayscale normalization, and bilinear scaling. If the proportion of boundary defects in any image exceeds a preset boundary defect threshold after preprocessing, a collection verification record is generated.

[0028] In this embodiment, the operation of obtaining the ridge cap position record includes: The cover region is defined within the cover image of the spine image group. During definition, edge detection is performed on the cover image to obtain the outer contour of the cover. Closed contour filtering is then applied to the outer contour, removing noisy contours with an area smaller than a preset cover area threshold. The largest rectangular area within the closed contour is selected as the cover region, and the coordinates of its top-left and bottom-right corners, as well as its region category, are recorded. The preset cover area threshold is determined by the distribution of cover areas in the collection samples or configured by the borrowing management system. Cover region definition can be achieved using a target detection network outputting detection boxes, or by combining edge detection with contour filtering. When the confidence value of the cover region is lower than the preset cover confidence threshold, a cover region verification record is generated.

[0029] Within the spine image group, the spine region, spine thickness boundary, and collection label region are calibrated. When calibrating the spine region, elongated region detection is performed on the spine image, extracting regions with a vertical or horizontal extension length exceeding a preset length threshold as the spine region. When calibrating the spine thickness boundary, parallel boundary lines are extracted on both sides of the spine region, and these two boundary lines are used as the spine thickness boundary. When calibrating the collection label region, high-contrast rectangular blocks, barcode blocks, or label text blocks are detected within the spine region, and the detection results are used as the collection label region. The preset length threshold is obtained statistically from the length of the spine regions of the collection samples or configured by the borrowing management system. The spine region can be obtained using an object detection network or elongated connected component filtering; the spine thickness boundary can be obtained using edge detection, Hough line detection, or segmentation network boundary output; the collection label region can be obtained using barcode detection, QR code detection, rectangular label detection, or text block detection. If the spine region, spine thickness boundary, or collection label region is not calibrated, a spine region verification record is generated.

[0030] In the binding area image within the spine image group, the binding hinge line is calibrated. During calibration, straight edge detection is performed on the binding area image to extract the continuous line segment with the largest grayscale change between the cover edge and the spine edge. The continuous line segments are then connected by breakpoints and filtered for directional consistency to obtain the binding hinge line, and the endpoint coordinates and direction of the binding hinge line are recorded. The direction of the binding hinge line is used for subsequent node projection sequence number calculation and binding line projection pairing. The binding hinge line can be obtained using Hough line detection, edge gradient peak detection, or semantic segmentation boundary extraction. When the binding hinge line cannot form a continuous line segment, a binding area verification record is generated.

[0031] The locations of the cover area, spine area, binding hinge line, spine thickness boundary, and collection label area are compiled into a spine location record. The spine location record includes the borrowing identification number, area category, area boundary coordinates, line segment endpoint coordinates, and the source image of the area; the cover area corresponds to the cover image, the spine area, spine thickness boundary, and collection label area correspond to the spine image, and the binding hinge line corresponds to the binding area image. The spine location record also includes the area confidence value and area detection status; when the area detection status is "detection failed," the corresponding borrowing identification number is written into the review record.

[0032] In this embodiment, the operation of extracting the ridge seal identification unit based on the ridge seal location record includes: Based on the cover area, spine area, binding hinge line, spine thickness boundary, and collection label area recorded in the spine position record, the cover recognition unit, spine recognition unit, binding recognition unit, thickness recognition unit, and collection label unit are extracted respectively, and the cover recognition unit, spine recognition unit, binding recognition unit, thickness recognition unit, and collection label unit are grouped into a spine recognition unit. When extracting the cover recognition unit, a cover image block is cropped from the cover area, and text recognition and pattern feature extraction are performed on the cover image block to obtain the cover text field, cover pattern features, and cover position features. When extracting the spine recognition unit, a spine image block is cropped from the spine area, and text line recognition, color band recognition, and layout feature extraction are performed on the spine image block to obtain the spine text field, spine color band features, and spine position features. When extracting the binding recognition unit, a binding neighborhood image block is cropped with the binding hinge line as the center, and binding texture features and binding line direction features are extracted. When extracting the thickness recognition unit, the distance between the two boundary lines is calculated based on the spine thickness boundary to obtain the thickness result and thickness boundary position. When extracting the collection label unit, a collection label image block is cropped from the collection label area, and label character recognition or barcode recognition is performed to obtain the collection label result. The spine recognition unit also includes field confidence values, label recognition confidence values, and thickness result confidence values. When the field confidence value, label recognition confidence value, or thickness result confidence value is lower than the corresponding threshold, the corresponding spine recognition unit is marked as a unit to be reviewed. Text recognition can use an OCR model; pattern feature extraction can use a convolutional neural network, a visual Transformer model, or a local feature descriptor; label recognition can use barcode recognition, QR code recognition, or label character recognition. When the cover text field, spine text field, collection label result, or thickness result is empty, a recognition unit review record is generated.

[0033] In this embodiment, the operation of registering the spine identification unit and candidate library books as nodes includes: The spine recognition unit is converted into a recognition node. During this conversion, a node record is created for each spine recognition unit. The node record includes the recognition unit type, source region, region location, image features, text fields, and boundary features. The cover recognition unit is converted into a cover side node, the spine recognition unit into a spine side node, the binding recognition unit into a binding node, the thickness recognition unit into a thickness node, and the collection label unit into a collection label node. Each recognition node also includes a node number, initial node region label value, node status, and node feature vector. The node feature vector of a recognition node can be obtained by concatenating image features, text field features, boundary position features, and label features.

[0034] The library's catalog search is performed on the catalog tag unit, cover recognition unit, and spine recognition unit to obtain candidate catalog books. During the catalog search, the catalog tag results in the catalog tag unit are precisely searched to obtain the first candidate set. The cover text field and cover pattern features in the cover recognition unit are similarly searched to obtain the second candidate set. The spine text field and spine color band features in the spine recognition unit are similarly searched to obtain the third candidate set. The first, second, and third candidate sets are merged and deduplicated to obtain the candidate catalog books. When the first, second, and third candidate sets are all empty, a review record is directly generated. When the number of candidate catalog books is greater than one, an independent catalog node is created for each candidate catalog book. The catalog search can use precise number search, text inverted index search, vector similarity search, and image feature similarity search. When the number of candidates returned by the catalog search exceeds the preset candidate number threshold, the candidate set is truncated according to the field matching score or image similarity score, and the candidate number exceeding the limit status is written to the review record.

[0035] Candidate books in the library's collection are converted into collection nodes, and the identification nodes and collection nodes are used as nodes in the spine entity graph. During the collection node conversion, a collection node record is created for each candidate book, including collection number, title, author, publisher, edition information, edition thickness range, and availability. The identification node and collection node are added to the same node set, which is then used as the node set in the spine entity graph. Each collection node participates in edition thickness matching, cross-face attention propagation, and spine identification residual calculation.

[0036] In this embodiment, the operation of connecting paired nodes as graph edges includes: The cover side node and spine side node are extracted from the identification nodes. Projection intervals are formed according to the binding hinge line direction. Cover side nodes and spine side nodes with overlapping projection intervals are marked as successful binding line projection pairing. When forming projection intervals, the boundary points of the cover side node's region are projected onto the binding hinge line direction to obtain the cover side projection start point and end point. The boundary points of the spine side node's region are also projected onto the binding hinge line direction to obtain the spine side projection start point and end point. When the interval formed by the cover side projection start point and end point overlaps with the interval formed by the spine side projection start point and end point, the binding line projection pairing is marked as successful. When the projection intervals do not overlap, the binding line projection pairing is marked as unsuccessful. The binding line projection pairing results include a pass mark, an unsuccessful mark, the cover side projection interval, and the spine side projection interval.

[0037] Field tags are extracted from the cover side node and spine side node separately. Cover side nodes and spine side nodes with identical field tags are considered successfully matched. During field tag extraction, the cover text field in the cover side node is categorized to obtain title, author, publisher, and category field tags; the spine text field in the spine side node is also categorized to obtain title, author, publisher, and category field tags. Character or semantic similarity comparisons are performed on the field content corresponding to the same type of field tags. If the field content is consistent or the similarity reaches a preset field threshold, the field tag matching is successful; if the field content is inconsistent and the similarity is below the preset field threshold, the field tag matching fails. The preset field threshold is obtained from the similarity statistics of text fields in the library collection or configured by the borrowing management system. The field tag matching result includes field type, field content, field similarity, and pass / fail status. If field tags cannot be extracted, field types cannot be categorized, or field content conflicts, a field matching review record is generated.

[0038] Based on the thickness recognition unit, the thickness results are obtained. The version thickness range of candidate books is retrieved from the library's collection database. Candidate books whose thickness results fall within the version thickness range are marked as having passed version thickness matching. When calculating the thickness result, the two spine thickness boundaries in the thickness recognition unit are read, the pixel distance between the two spine thickness boundaries is calculated, and the pixel distance is converted into a thickness value according to a calibrated ratio. The version thickness range of candidate books is retrieved from the library's collection database, including a minimum thickness value and a maximum thickness value. When the thickness result is greater than or equal to the minimum thickness value and less than or equal to the maximum thickness value, the version thickness matching is marked as passed. When the thickness result is less than the minimum thickness value or greater than the maximum thickness value, the version thickness matching is marked as failed. The version thickness matching result includes the thickness result, minimum thickness value, maximum thickness value, and pass status. If the thickness result cannot be calculated or the version thickness range of the candidate books is missing, a thickness matching verification record is generated.

[0039] Connect the cover side nodes and spine side nodes that have passed the binding line projection pairing and field label pairing to form the cover spine image edge. Connect the identification nodes and collection nodes corresponding to the thickness identification units that have passed the version thickness pairing to form the thickness version image edge. The spine entity image is formed by the identification nodes, collection nodes, cover spine image edges, and thickness version image edges. When creating the cover spine image edge, the cover side node number, spine side node number, binding line projection pairing result, and field label pairing result are written into the cover spine image edge record. When creating the thickness version image edge, the thickness node number, collection node number, thickness result, version thickness range, and version thickness pairing result are written into the thickness version image edge record. The spine entity image includes a node set and an image edge set. The node set includes identification nodes and collection nodes, and the image edge set includes cover spine image edges and thickness version image edges. When multiple candidate collection books exist, a thickness version image edge record is created for each collection node. When the cover spine image edge or the thickness version image edge is empty, a spine entity image verification record is generated.

[0040] In this embodiment, the operation to obtain the ridge coding map includes: Using the binding hinge line as the projection reference, the identification nodes in the spine solid diagram are projected along the direction of the binding hinge line, and node projection numbers are configured according to the order of projection positions. When configuring node projection numbers, the center point of the corresponding area of ​​each identification node is read, and the center point is projected onto the direction of the binding hinge line to obtain the node projection position. Incremental node projection numbers are assigned to the identification nodes according to the arrangement order of their projection positions from one end of the binding hinge line to the other. The node projection numbers are used to record the spatial arrangement order of the identification nodes along the direction of the binding hinge line. Before entering the corrected node input vector, the node projection number is converted into a node projection number code. If the node projection position cannot be obtained, the corresponding identification node is marked as a projection anomaly node.

[0041] Based on the cover area, spine area, binding hinge line, spine thickness boundary, or collection label area corresponding to the identification node, a node area marker is configured. When configuring the node area marker, the source area of ​​the identification node is read. If the source area is the cover area, a cover area marker is configured; if the source area is the spine area, a spine area marker is configured; if the source area is the binding hinge line, a binding area marker is configured; if the source area is the spine thickness boundary, a thickness area marker is configured; and if the source area is the collection label area, a label area marker is configured. The node area marker is added to the identification node using one-hot encoding or integer encoding. If the source area is missing, the corresponding identification node is marked as an area anomaly node.

[0042] Based on the inner and outer positions of the identification nodes relative to the spine thickness boundary and the thickness extension direction, thickness direction markers are configured. When configuring thickness direction markers, the two boundary lines of the spine thickness boundary are read, and the area between the two boundary lines is taken as the thickness extension area. When the center point of the identification node is within the thickness extension area, an inner thickness marker is configured; when the center point of the identification node is outside the thickness extension area, an outer thickness marker is configured. The thickness extension direction is determined according to the normal directions of the two spine thickness boundaries and encoded as a thickness direction marker. Thickness direction markers are used to distinguish the spatial orientation of cover side nodes, spine side nodes, and thickness nodes in the entity diagram. When the spine thickness boundary is missing, the thickness direction marker is configured as an anomaly marker, and a thickness direction verification record is generated.

[0043] The original node features of the recognition node and the collection node are extracted. These original node features are then combined with the node projection index, node region label, and thickness direction label to form the corrected node input vector. During the extraction of original node features, image features, text field features, boundary position features, and label features are extracted for the recognition node; collection field features, version thickness range features, and borrowable status features are extracted for the collection node. During combination, the original node features, node projection index encoding, node region label encoding, and thickness direction label encoding are concatenated in a fixed order and transformed into the input dimension of the improved Equiformer structure through a linear mapping layer, resulting in the corrected node input vector. The corrected node input vector serves as the node input for the improved Equiformer structure, replacing the original input that only contains visual or text features.

[0044] The corrected node input vector is input into the improved Equiformer structure, and isotropic coding is performed to obtain the spine coding map. During isotropic coding, the improved Equiformer structure reads the node set, graph edge set, and corrected node input vector of the spine entity map, and performs graph attention coding on the identified nodes and collection nodes, updating the node features while preserving the spatial structure of the cover, spine, binding edges, and thickness directions. After coding, the output is a spine coding map containing the updated node features and graph edge structure. The spine coding map includes updated identified node features, updated collection node features, graph edge structure, and node region labels. The improved Equiformer structure can be executed by the local computing unit or by the server. If the model loading fails or the coding output is empty, a model processing review record is generated.

[0045] In this embodiment, the operation of shielding the cross-surface attention edge includes: Extract the cover side nodes corresponding to the cover area and the spine side nodes corresponding to the spine area from the spine coding diagram. When extracting cover side nodes, read the nodes in the spine coding diagram whose node area is marked as the cover area. When extracting spine side nodes, read the nodes in the spine coding diagram whose node area is marked as the spine area. Combine the cover side nodes and spine side nodes to form candidate cross-face node pairs. Each candidate cross-face node pair includes a cover side node number, a spine side node number, and a corresponding borrowing identification number. If either the cover side node or the spine side node is empty, generate a cross-face node verification record.

[0046] Establish cross-face attention edges between the cover side nodes and the spine side nodes. When establishing cross-face attention edges, create a candidate cross-face attention edge for each candidate cross-face node pair, and record the cover side node number, spine side node number, node projection sequence difference, node region label, and thickness direction label. When multiple candidate books exist, the candidate cross-face attention edges are bound to the corresponding library nodes according to the library node numbers.

[0047] The system reads the binding line projection pairing results and field label pairing results corresponding to the nodes at both ends of the cross-face attention edge, and reads the version thickness pairing results of the collection nodes associated with the cross-face attention edge. When reading the binding line projection pairing results, it retrieves the cover and spine image edge records according to the cover side node number and the spine side node number. When reading the field label pairing results, it retrieves the field content comparison results according to the cover side node number and the spine side node number. When reading the version thickness pairing results, it retrieves the collection nodes according to the candidate collection books bound to the cross-face attention edge, and reads the thickness version image edge records between the collection nodes and the thickness nodes. The associated collection nodes are the collection nodes corresponding to the candidate collection books bound to the cross-face attention edge. If any pairing result is missing, the corresponding cross-face attention edge is marked as a masked edge.

[0048] When the binding line projection pairing result is unsuccessful, the field label pairing result is unsuccessful, or the version thickness pairing result is unsuccessful, the corresponding cross-surface attention edge is marked as a shielded edge. When marking a shielded edge, the edge state of the cross-surface attention edge is set to shielded state, and the shielded state is written to the attention edge state table. The cross-surface attention edge corresponding to the shielded state does not participate in the subsequent attention weight calculation. The shielded edge retains a record but is not included in the propagation edge set.

[0049] When the binding line projection pairing result, field label pairing result, and version thickness pairing result are all passed, the corresponding cross-face attention edge is marked as a reserved edge. When marking a reserved edge, the edge state of the cross-face attention edge is set to reserved state, and the reserved state is written to the attention edge state table. The cross-face attention edge corresponding to the reserved state enters the attention calculation range of the improved Equiformer structure. The reserved edge is used for cross-face feature transfer between the cover side node and the spine side node.

[0050] The shielded edges are removed from the attention calculation scope of the improved Equiformer structure, while the retained edges are retained. When removing shielded edges, the attention mask value corresponding to the shielded edge is set to an invalid value, stopping feature propagation between the cover-side node and the spine-side node. When retaining retained edges, the attention mask value corresponding to the retained edge is set to a valid value, allowing isomorphic graph attention propagation between the cover-side node and the spine-side node. The attention edge status table includes the cross-face attention edge number, cover-side node number, spine-side node number, associated collection node number, edge status, and attention mask value. When all cross-face attention edges are shielded edges, a verification record with no valid retained edges is generated.

[0051] In this embodiment, the operations for obtaining the dissemination results of candidate library collection books include: Within the attention calculation range of the improved Equiformer structure, the reserved edges are retained, and feature transfer between the nodes at both ends of the shielded edges is stopped. Before propagation, the attention edge state table is read, and cross-face attention edges with a reserved state are added to the propagation edge set, while cross-face attention edges with a shielded state are removed from the propagation edge set. If there are no shielded edges in the propagation edge set, the features of the nodes at both ends of the shielded edges are not updated. The propagation edge set is grouped according to the library node number, so that each candidate library book corresponds to a set of reserved edges. When the propagation edge set corresponding to any library node is empty, the corresponding library node is written into the review record.

[0052] An isomorphic graph attention propagation is performed on the cover-side nodes and spine-side nodes at both ends of the retained edge to obtain cross-face propagation features. During isomorphic graph attention propagation, the corrected node input vectors of the nodes at both ends of the retained edge are read. Key vector mapping and value vector mapping are performed on the corrected node input vectors of the cover-side nodes, and query vector mapping is performed on the corrected node input vectors of the spine-side nodes. The query vectors and key vectors are input into the attention mapping layer to obtain the retained edge attention weights. The value vectors are weighted using the retained edge attention weights to obtain the first propagation feature from the cover-side node to the spine-side node. A reverse mapping is performed on the spine-side nodes and cover-side nodes to obtain the second propagation feature from the spine-side node to the cover-side node. The first and second propagation features are merged to obtain the cross-face propagation features. The attention mapping layer calculates the retained edge attention weights based on the corrected node input vectors, node region labels, node projection indices, and thickness direction labels. If the attention weights are abnormal or the cross-face propagation features are empty, a propagation verification record is generated.

[0053] The cross-face propagation features, node features of the corresponding recognition nodes of the thickness recognition unit, and node features of the corresponding recognition nodes of the collection label unit are aggregated to the collection node to form the candidate collection book propagation results. During aggregation to the collection node, according to the candidate collection book number corresponding to the collection node, the cross-face propagation features generated by the thickness node, collection label node, and retained edges connected to the collection node are read; the cross-face propagation features, thickness node features, and collection label node features are concatenated and input into the collection node aggregation layer to obtain the updated features of the collection node; the updated features of the collection node, the collection node number, and the candidate collection book number are combined to form the candidate collection book propagation results. The candidate collection book propagation results are output separately for each candidate collection book, with each candidate collection book corresponding to one collection node updated feature and one propagation matching component. When the collection node updated feature is empty, a candidate propagation result verification record is generated.

[0054] In this embodiment, the operation of determining the target library book based on the spine identification residual and the borrowability status of the candidate library books includes: The propagation matching component of the collection node is extracted from the propagation results of the candidate collection books. The binding line projection pairing result, field label pairing result, and version thickness pairing result of the collection node are read. The spine recognition residual is generated based on the propagation matching component, binding line projection pairing result, field label pairing result, and version thickness pairing result. When extracting the propagation matching component, the updated features of the collection node are input into the matching mapping layer, and the propagation matching component between the candidate collection book and the book to be borrowed is output. When generating the spine recognition residual, the propagation matching component is compared with a preset matching threshold. If the propagation matching component is lower than the preset matching threshold, a propagation residual is generated. The binding line projection pairing result is read. If the binding line projection pairing result fails, a projection residual is generated. The field label pairing result is read. If the field label pairing result fails, a field residual is generated. The version thickness pairing result is read. If the version thickness pairing result fails, a thickness residual is generated. The propagation residual, projection residual, field residual, and thickness residual are merged to obtain the spine recognition residual. The preset matching threshold and preset residual threshold are obtained from the matching distribution of the collection samples and the statistics of manual review records, or configured by the borrowing management system. When the propagation matching component is empty, the pairing result is missing, or the spine recognition residual cannot be generated, a residual review record is generated.

[0055] Candidate books whose spine recognition residuals meet a preset residual threshold and whose borrowability status is "available" are identified as target books, and borrowing records are generated based on these target books. When identifying target books, the spine recognition residuals are compared with the preset residual thresholds. If the spine recognition residual is less than or equal to the preset residual threshold and the candidate book's borrowability status is "available," the candidate book is written into the target book field. When generating borrowing records, the target book's catalog number, borrower ID, and borrowing time are read, and these information are written into the borrowing record. If multiple candidate books meet the preset residual thresholds and are available, the candidate book with the smallest spine recognition residual is identified as the target book. If two or more candidate books have the same spine recognition residual or a difference less than a preset residual interval threshold, a verification record is generated. Before generating a borrowing record, the real-time borrowing status of candidate library books is read. If the real-time borrowing status is unavailable, already checked out, reserved, or prohibited, no borrowing record is generated, but a status review record is generated instead.

[0056] For candidate books whose spine recognition residuals do not meet the preset residual threshold or whose borrowability status is not "borrowable," a review record is generated. When generating the review record, candidate books with spine recognition residuals greater than the preset residual threshold are written into the recognition review item, and candidate books whose borrowability status is not "borrowable" are written into the status review item. The review record includes the candidate book number, spine recognition residual, propagation matching component, binding line projection pairing result, field tag pairing result, version thickness pairing result, and borrowability status. A review record is generated when the collection label cannot be recognized, field tags cannot be paired, thickness results cannot be calculated, there is no preserved edge between the cover side node and the spine side node, or all candidate books are unborrowable.

[0057] During model training, training samples include cover images, spine images, binding area images, collection labels, version thickness, target collection book annotations, and review status annotations. Training objectives include field consistency, thickness and version matching, collection node matching, and review status recognition. Review records are output to librarian terminals, self-service lending cabinet management interfaces, or the library server's review queue.

[0058] Example 1: To verify the feasibility of this invention in practice, it was applied to a self-service book borrowing scenario. In this scenario, the books to be borrowed exhibit issues such as worn-out library tags, reflective covers, narrow spine text, different versions of the same title, similar covers, and differences in thickness between versions. Using single barcode recognition or single cover recognition methods easily leads to problems such as unreadable tags, misjudgments of similar covers, missing spine fields, and version confusion. This invention improves the accuracy of book identification by simultaneously acquiring cover images, spine images, and binding area images, and jointly determining the correspondence between the cover, spine, binding hinge line, spine thickness boundary, and library tag.

[0059] During the application process, after the reader places the book into the self-service borrowing and recognition area, the system collects the cover image, spine image, and binding area image, and forms a spine image group after preprocessing; then the cover area, spine area, spine thickness boundary, collection label area, and binding hinge line are calibrated, and the corresponding cover recognition unit, spine recognition unit, binding recognition unit, thickness recognition unit, and collection label unit are extracted.

[0060] The system uses collection labels, cover text, cover patterns, spine text, and spine color bands to search for collections. It converts the spine recognition unit into recognition nodes, converts candidate collection books into collection nodes, and establishes graph edges based on binding line projection matching, field mark matching, and version thickness matching results to form a spine entity graph.

[0061] Subsequently, the system inputs the spine entity map into the improved Equiformer structure, configures the node projection sequence number, node region label, and thickness direction label, and combines these labels with the original node features to form the corrected node input vector. The improved Equiformer structure adds a node spatial label input channel and a cross-face attention edge shielding channel, so that the feature propagation between cover side nodes and spine side nodes is constrained by binding line projection, field consistency, and version thickness results.

[0062] During the cross-face propagation phase, the system establishes a cross-face attention edge between the cover side node and the spine side node. If any of the binding line projection pairing, field tag pairing, or version thickness pairing fails, the corresponding cross-face attention edge is blocked; if all three pairings succeed, the corresponding cross-face attention edge participates in the equal-variable graph attention propagation. After propagation, the system aggregates the cross-face propagation features, thickness node features, and collection tag node features to the collection node, obtaining the propagation results for the candidate collection books.

[0063] The system generates spine recognition residuals based on the dissemination results of candidate library books. When the spine recognition residual is less than or equal to a preset residual threshold, and the candidate library book is available for borrowing, the system identifies the target library book and generates a borrowing record; when the spine recognition residual exceeds the threshold, the candidate library book is not available for borrowing, the residuals of multiple candidate library books are close, there is no retained edge, the library label cannot be recognized, or the thickness result cannot be calculated, the system generates a review record.

[0064] To verify the effectiveness of this invention, multiple batches of books awaiting borrowing were selected for comparative processing. The samples covered ordinary books, books with damaged labels, books with similar covers, books of the same title but different editions, books with narrow spine text, books with minor thickness differences, and books that were unavailable for borrowing. The comparison methods included conventional label recognition, a combined confidence ranking method based on single cover and single spine, and the method of this invention. The comparison results are shown in the table below: Table 1: Comparison of the Implementation Effects of Joint Identification and Borrowing System with Spine Covering ; The data in the table shows that, with a total sample size of 6280 books, the method of this invention is superior to conventional label recognition methods and single cover and single spine joint confidence ranking methods in terms of recognition accuracy, anomaly interception capability and error control.

[0065] Regarding the accuracy of ordinary book identification, the conventional label identification method achieved 96.81%, the single cover and single spine joint confidence ranking method achieved 97.44%, and the method of this invention reached 99.32%, indicating that the invention has high identification stability in regular borrowing scenarios. Regarding the accuracy of identifying severely damaged labels, the conventional label identification method achieved 73.42%, the single cover and single spine joint confidence ranking method achieved 91.57%, and the method of this invention reached 97.86%, indicating that the invention can reduce the reliance on the integrity of the library's labels.

[0066] In terms of recognizing books with complex appearances, the accuracy rate for recognizing similar covers increased from 88.64% and 94.06% to 98.47%, the accuracy rate for recognizing different editions with the same name increased from 84.27% and 90.83% to 96.92%, and the accuracy rate for recognizing narrow-text spine samples increased from 86.95% and 92.38% to 97.35%. These data demonstrate that this invention, through matching the actual spine image, binding line projection, field markers, and edition thickness, can more accurately distinguish between books with similar covers, different editions with the same name, and books with narrow spine text.

[0067] Regarding error control, the average mis-borrowing rate of the method described in this invention is 0.29%, lower than the 2.18% of the conventional label recognition method and the 1.46% of the single cover and single spine joint confidence ranking method; the average missed borrowing rate is 0.61%, lower than 3.42% and 2.15%; and the comprehensive manual review rate is 2.43%, lower than 8.76% and 6.18%. In terms of anomaly interception, the method described in this invention achieves a 99.58% interception rate for unborrowable states and a 96.37% interception rate for abnormal edition thickness, both significantly higher than the two comparative methods. These results demonstrate that this invention can improve the accuracy of book identification, reduce mis-borrowing, missed borrowing, and manual review, and enhance the interception capabilities for unborrowable states and abnormal edition thickness.

[0068] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for jointly recognizing and borrowing books based on deep learning's cover and spine, characterized in that... Includes the following steps: Collect images of the cover, spine, and binding area of ​​the books to be borrowed, and perform preprocessing on the collected images to obtain a set of spine images; Region detection was performed on the spine image group to mark the cover area, spine area, binding hinge line, spine thickness boundary and collection label area, and the spine position record was obtained; Extract the ridge seal identification unit based on the ridge seal location record; Register the spine identification unit and candidate collection books as nodes, perform binding line projection pairing and field label pairing on the spine identification unit, perform version thickness pairing on the spine identification unit and candidate collection books, connect the paired nodes as graph edges, and obtain the spine entity graph. Input the spine solid diagram into the improved Equiformer structure, calculate the node projection number, node area mark and thickness direction mark based on the binding hinge line, and use the node projection number, node area mark and thickness direction mark to correct the node input vector of the spine solid diagram to obtain the spine coding diagram. In the spine coding diagram, cross-face attention edges are established between the cover side node and the spine side node. Cross-face attention edges with mismatched binding line projections, inconsistent field labels, or thickness results that do not pass the version thickness pairing are masked. Cross-face attention edges without masking are propagated by the same graph to obtain the propagation results of candidate library collection books. Based on the dissemination results of candidate library books, a spine identification residual is generated. Based on the spine identification residual and the borrowability status of candidate library books, the target library books are determined and a borrowing record is generated. If the target library books are not determined, a review record is generated.

2. The method for joint recognition and borrowing of book covers and spines based on deep learning according to claim 1, characterized in that, The operation of obtaining the ridge image group includes: Distortion correction, brightness normalization, and size normalization are performed on the cover image, spine image, and binding area image respectively. The preprocessed cover image, spine image, and binding area image are then grouped into a spine image set of the same book to be borrowed.

3. The method for joint recognition and borrowing of book covers and spines based on deep learning according to claim 2, characterized in that, The operation of obtaining the ridge cap position record includes: Mark the cover area within the cover image in the spine image group; Mark the spine region, spine thickness boundary, and collection label region within the spine image group; Mark the binding hinge line within the binding area image in the spine image group; The locations of the cover area, spine area, binding hinge line, spine thickness boundary, and collection label area are grouped into a spine location record.

4. The method for joint recognition and borrowing of book covers and spines based on deep learning according to claim 3, characterized in that, The operation of extracting the ridge identification unit based on the ridge position record includes: Based on the cover area, spine area, binding hinge line, spine thickness boundary, and collection label area recorded in the spine position record, the cover recognition unit, spine recognition unit, binding recognition unit, thickness recognition unit, and collection label unit are extracted respectively, and the cover recognition unit, spine recognition unit, binding recognition unit, thickness recognition unit, and collection label unit are grouped into a spine recognition unit.

5. The method for joint recognition and borrowing of book covers and spines based on deep learning according to claim 4, characterized in that, The operation of registering the spine identification unit and candidate library books as nodes includes: Convert the ridge seal recognition unit into a recognition node; Perform a library collection search on the collection label unit, cover recognition unit, and spine recognition unit to obtain candidate library collection books; Candidate books in the library collection are converted into library collection nodes, and the identification nodes and library collection nodes are used as nodes in the spine entity graph.

6. The method for joint recognition and borrowing of book covers and spines based on deep learning according to claim 5, characterized in that, The operation of connecting paired nodes into graph edges includes: Extract the cover side node and spine side node from the identification node, and form projection intervals according to the binding hinge line direction. Mark the cover side node and spine side node that overlap in the projection interval as binding line projection pairing. Extract the field tags from the cover side node and the spine side node respectively, and mark the cover side node and spine side node with the same field tags as field tag pairing passed; Based on the thickness recognition unit, the thickness results are obtained. The version thickness range of candidate books in the collection database is read. Candidate books whose thickness results fall within the version thickness range are marked as having passed version thickness matching. Connect the cover side node and spine side node that have passed the binding line projection and field mark pairing to form the cover spine image edge. Connect the recognition node and collection node corresponding to the thickness recognition unit that have passed the version thickness pairing to form the thickness version image edge. The spine entity image is formed by the recognition node, collection node, cover spine image edge and thickness version image edge.

7. The method for joint recognition and borrowing of book covers and spines based on deep learning according to claim 6, characterized in that, The operation of obtaining the ridge coding map includes: Using the binding hinge line as the projection reference, the identification nodes in the spine solid drawing are projected along the direction of the binding hinge line, and the node projection sequence number is configured according to the order of the projection positions. Configure node area markers based on the cover area, spine area, binding hinge line, spine thickness boundary, or collection label area corresponding to the identification node; Based on the inner and outer positions of the identification nodes relative to the thickness boundary of the spine and the thickness extension direction, thickness direction markers are configured; the original node features of the identification nodes and collection nodes are extracted, and the original node features are combined with the node projection number, node region marker, and thickness direction marker to form the corrected node input vector; The modified node input vector is input into the improved Equiformer structure, and equivariant coding is performed to obtain the ridge coding map.

8. The method for joint recognition and borrowing of book covers and spines based on deep learning according to claim 7, characterized in that, The operation of shielding the cross-face attention edge includes: Extract the cover side nodes corresponding to the cover area from the spine coding diagram, and extract the spine side nodes corresponding to the spine area from the node region markers. Establish a cross-surface attention edge between the cover side node and the spine side node; Read the binding line projection pairing results and field label pairing results corresponding to the nodes at both ends of the cross-face attention edge, and read the version thickness pairing results of the collection nodes associated with the cross-face attention edge; When the binding line projection pairing result is unsuccessful, the field mark pairing result is unsuccessful, or the version thickness pairing result is unsuccessful, the corresponding cross-surface attention edge will be marked as a shielded edge. When the binding line projection pairing result, field mark pairing result, and version thickness pairing result are all passed, the corresponding cross-face attention edge will be marked as a reserved edge; Remove the shielded edges from the attention calculation range of the improved Equiformer structure, and retain the edges within the attention calculation range of the improved Equiformer structure.

9. A method for jointly recognizing and borrowing book covers and spines based on deep learning according to claim 8, characterized in that, The steps for obtaining the dissemination results of candidate library collection books include: In the attention computation range of the improved Equiformer structure, retain the edges and stop feature transfer between the nodes at both ends of the masked edge; Perform isomorphic graph attention propagation on the cover side nodes and spine side nodes at both ends of the preserved edge to obtain cross-surface propagation features; The cross-surface propagation features, the node features of the corresponding recognition nodes of the thickness recognition unit, and the node features of the corresponding recognition nodes of the collection tag unit are aggregated into the collection node to form the propagation results of candidate collection books.

10. A method for jointly recognizing and borrowing book covers and spines based on deep learning according to claim 9, characterized in that, The operation of determining the target collection book based on the spine cover identification residual and the borrowability status of candidate collection books includes: Extract the propagation matching component of the collection node from the propagation results of the candidate collection books, read the binding line projection pairing result, field label pairing result and version thickness pairing result corresponding to the collection node, and generate the spine recognition residual based on the propagation matching component, binding line projection pairing result, field label pairing result and version thickness pairing result; Candidate books whose spine recognition residuals meet the preset residual thresholds and whose borrowable status is available are identified as target books. Borrowing records are generated based on the target books. For candidate books whose spine recognition residuals do not meet the preset residual threshold or whose borrowable status is not borrowable, a review record will be generated.

Images