A method for automatic identification and dosage confirmation of bone pins based on multi-feature collaborative reasoning

Abstract

The application discloses a kind of based on the automatic identification and dosage confirmation method of bone nail of multi-feature collaborative inference.The method first acquires the multi-view image sequence of bone nail box in initial state, obtains bone nail identification image sequence;Boundary information of bone nail placement position is extracted based on the image sequence, and bone nail structure atlas for indicating the relative position relationship of placement area is generated;The structure atlas is combined with image sequence, the image features of each placement area are extracted, and based on bone nail standard external shape feature comparison and identification, identification result is obtained;Identification result is compared with historical use behavior mode, potential anomaly is identified and corrected enhanced identification result is output;The above steps are executed before and after surgery respectively, enhanced identification result is obtained and difference comparison is carried out, and bone nail use difference information is obtained;Combined with surgery mark, identification confidence and acquisition time generate bone nail use confirmation information, and write in tamper-proof storage medium, realize the closed loop tracing of nail record.

Description

Technical Field

[0001] This invention relates to the field of computer image analysis technology, and in particular to a method for automatic identification and dosage confirmation of bone nails based on multi-feature collaborative reasoning. Background Technology

[0002] Bone screws used in orthopedic surgery are typically stored in specialized screw cases. To ensure standardized surgical procedures and accurate dosage, hospitals generally use manual methods to count bone screws before surgery and verify them after surgery, sometimes combined with photographic documentation or simple image recognition tools. Some technical solutions propose using single-view image acquisition and template comparison to identify the number of bone screws, and using differential methods to determine the consumption of bone screws before and after surgery, thereby assisting medical staff in confirming and recording the dosage.

[0003] However, existing technologies still have some limitations. First, single-view image acquisition is easily affected by factors such as light reflection, bone screw occlusion, or similar bone screw sizes, leading to insufficient recognition accuracy. Second, template comparison methods are difficult to cover complex scenarios with different models and placement methods, limiting the system's adaptability and robustness. Third, existing methods mainly rely on visual features and lack the ability to combine historical data or multi-dimensional information for correction, failing to effectively correct for identification anomalies or questionable results. Furthermore, consumable usage records typically rely on manual input or storage in ordinary databases, posing a risk of tampering and hindering the traceability of medical liability and safety compliance.

[0004] Therefore, there is an urgent need to propose a new method for automatic identification and dosage confirmation of bone screws in order to overcome the shortcomings of existing technologies. Summary of the Invention

[0005] This application provides a method for automatic identification and quantity confirmation of bone screws based on multi-feature collaborative reasoning, so as to improve the accuracy and stability of bone screw category and quantity identification.

[0006] This application provides a method for automatic identification and dosage confirmation of bone nails based on multi-feature collaborative reasoning, including:

[0007] Step S1: Collect a multi-view image sequence of the bone nail box in its initial state, and perform edge extraction, corner correction and perspective correction on the multi-view image sequence to obtain a bone nail recognition image sequence;

[0008] Step S2: Based on the bone screw recognition image sequence, extract the boundary information of the bone screw placement position and generate a bone screw structure map to represent the relative positional relationship of the bone screw placement area;

[0009] Step S3: Combine the bone screw structure atlas with the bone screw recognition image sequence to extract the image features of the bone screw in each placement area, and perform comparison recognition based on the standard shape features of the bone screw to obtain the recognition result containing bone screw category information and recognition confidence.

[0010] Step S4: Compare the identification results with the historical usage behavior pattern of bone screws, identify potential abnormal identification items and output the enhanced identification results after behavior pattern correction, wherein the historical usage behavior pattern of bone screws is constructed based on historical bone screw usage behavior data related to surgery type, doctor identification and bone screw specifications;

[0011] Step S5: Perform steps S1-S4 before and after the operation to obtain the enhanced recognition results before and after the use of the bone screw, and calculate the difference information between the two by comparing the corresponding positions to obtain the bone screw use difference information.

[0012] Step S6: Based on the bone screw usage difference information, combined with surgical identification information, recognition confidence level and image acquisition time, generate bone screw usage confirmation information, and write the bone screw usage confirmation information into an immutable storage medium to achieve closed-loop traceability of bone screw usage records.

[0013] The beneficial effects of the technical solution provided in this application include:

[0014] (1) By acquiring images from multiple perspectives and correcting perspective, and combining them with bone screw structure maps for feature alignment, the recognition error caused by occlusion, angle deviation or uneven lighting is effectively reduced. Compared with single image recognition, this method can significantly improve the accuracy and stability of bone screw category and quantity recognition. (2) By comparing and correcting the recognition results using historical usage behavior patterns related to surgical type, doctor identification and bone screw specifications, potential anomalies can be automatically detected and corrected, avoiding usage confirmation errors caused by misidentification, thereby enhancing the system's fault tolerance and intelligence level. (3) Recognition and comparison are performed before and after surgery, automatically generating bone screw usage results before and after and calculating differences, which can automatically confirm usage, replace manual counting, significantly reduce the workload of medical staff and reduce the risk of human statistical errors. (4) By writing bone screw usage confirmation information into an immutable storage medium, long-term retention and closed-loop traceability of surgical screw information can be achieved, ensuring the authenticity and integrity of the data, which helps with the division of responsibilities, medical compliance and the transparency of the hospital management system. Attached Figure Description

[0015] Figure 1 This is a flowchart of a method for automatic identification and dosage confirmation of bone nails based on multi-feature collaborative reasoning, provided in the first embodiment of this application. Detailed Implementation

[0016] Many specific details are set forth in the following description to provide a full understanding of this application. However, this application can be implemented in many other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this application; therefore, this application is not limited to the specific embodiments disclosed below.

[0017] The first embodiment of this application provides a method for automatic identification and dosage confirmation of bone nails based on multi-feature collaborative reasoning. Please refer to... Figure 1 This figure is a schematic diagram of the first embodiment of this application. The following is in conjunction with... Figure 1 The first embodiment of this application provides a detailed description of a method for automatic identification and dosage confirmation of bone nails based on multi-feature collaborative reasoning.

[0018] Step S1: Collect a multi-view image sequence of the bone nail box in its initial state, and perform edge extraction, corner correction and perspective correction on the multi-view image sequence to obtain a bone nail recognition image sequence.

[0019] In step S1, first place the bone screw box in the surgical preparation area or on a clean workbench, ensuring it is in its initial state and all bone screws are within their respective placement areas. The upper surface of the bone screw box should be clean and free of obvious liquid residue. To ensure image consistency, mount the main camera on a fixation bracket approximately 450mm above the bone screw box, and place an auxiliary camera on each side in front of the bone screw box, tilting the optical axis of each camera approximately 25° to 30° relative to the normal, thus obtaining three perspectives: top view, side oblique view, and side oblique view. Each camera is preferably an industrial camera with at least 2 megapixels and a 25mm fixed-focus low-distortion lens, with a fixed aperture in the range of F8 to F11 to increase depth of field and reduce image differences caused by factory tolerances. To suppress metallic reflections, install a linear polarizer in front of the lens, and arrange a ring of diffused light above it and two strip light sources on either side, with a color temperature of approximately 5000K and an illuminance controlled between 800 and 1200 lux. The light sources are also equipped with polarizers that intersect with the lens polarizer to reduce highlights. Exposure time and gain were coarsely adjusted and locked to fixed values ​​using the center-weighted mode of automatic exposure. White balance was set to a fixed color temperature mode to avoid color drift when switching viewing angles. To obtain a measurable spatial reference, four high-contrast corner markers were temporarily attached to the edge of the same plane of the pin box, or the inner edges of the four corners of the pin box were used directly as reference points. A millimeter ruler was placed on one side of the pin box for subsequent pixel-to-physical size conversion.

[0020] The acquisition process sequentially triggers three-way shooting: top view, left oblique view, and right oblique view. Three frames of raw images are continuously acquired from each viewpoint, and the median frame is selected to suppress occasional noise and minor jitter, forming the raw material for a multi-view image sequence. For each raw image frame, a color space conversion from BGR to grayscale is performed, followed by contrast-limited adaptive histogram equalization to enhance weak edges. Smoothing is then performed using a Gaussian kernel of 1.2 to 1.6 pixels. Canny edge extraction is then employed, with a low threshold set to 0.66 times the median grayscale gradient and a high threshold set to twice the low threshold. The resulting binary edge map undergoes one 3×3 erosion followed by two 3×3 dilation operations to remove isolated noise points and connect broken edges. To identify corner points, Shi-Tomasi corner detection is preferably used on the original grayscale image. The initial threshold is set to a quality factor of 0.01 and a minimum spacing of 10 pixels. Subpixel-level thinning iterations are terminated with a maximum of 30 iterations or a precision of 1e−3. If corner markers are used, the marked corner points are used as four reference points. If no markers are attached, the edge lines of the bone screw box are detected in the edge image using probabilistic Hough linear transform, and the intersection points of the interior corners are taken. After subpixel thinning, these are used as four reference points. Subsequently, based on the correspondence between the four reference points and the four corners of the target rectangular shape, the projective transformation parameters are estimated, and geometric shaping is performed on each frame of the image to ensure that the upper surface of the bone screw box is orthographically projected from the top view in the image, without perspective distortion. The shaped image is mapped to a uniform pixel grid according to a predetermined physical size. The top view frame is preferably corrected to, for example, 2200×1600 pixels, and the pixel spacing is calculated based on a millimeter scale, retaining at least the pixel precision corresponding to 0.01mm. To ensure reliable corner correction, a quality check is performed on the reshaped image: the Laplacian variance used for sharpness assessment should be no less than 200, the saturation pixel ratio at both ends of the histogram should not exceed 1% each, and the parallelism deviation of the reference edge in the horizontal and vertical directions should not exceed 0.2°; if any indicator fails to meet the standard, the image is reverted to the original frame of that viewpoint, the exposure or gain is automatically fine-tuned, and the image is re-acquired, with a maximum of two retries. If the image still fails to meet the standard, a manual review is prompted.

[0021] After geometric shaping, to facilitate region analysis in subsequent steps, an initial region of interest polygon is generated in the orthophoto top view frame based on the inner edge of the nail box. This region is then radially shrunk to eliminate the highly reflective border band. Subsequently, this region is used as a mask to crop a standardized top view. Simultaneously, the oblique view frames on both sides undergo the same preprocessing and shaping steps and are saved together with the top view frame. Uniform metadata is written to the finished frame for each viewpoint, including the acquisition timestamp, viewpoint identifier, resolution, pixel pitch, reference point screen coordinates, and cropped region polygon. The viewpoint frames obtained through the above processing, meeting the requirements for sharpness and geometric consistency, are organized in a fixed order to form a nail recognition image sequence. This nail recognition image sequence is the output of step S1, having undergone edge extraction, corner correction, and perspective correction. It possesses a unified scale, orientation, and brightness reference and can be directly used in the subsequent step S2 to extract the boundary information of the nail placement position without requiring further camera geometric clutter correction.

[0022] Furthermore, the acquisition of a multi-view image sequence of the bone screw box in its initial state, followed by edge extraction, corner correction, and perspective correction of the multi-view image sequence to obtain a bone screw recognition image sequence, includes:

[0023] A multi-view image sequence of the bone nail box in its initial state was acquired. During the acquisition process, the polarization direction of the incident light was adjusted by combining a polarizer and an adjustable light source to suppress specular reflection on the metal surface, thereby obtaining a raw multi-view image sequence with uniform illumination.

[0024] The original multi-view image sequence is input into the time synchronization module, the multi-view images at the same time are paired, and the median time filter is used to remove artifacts caused by jitter, and the time-consistent multi-view candidate image frames are output.

[0025] Geometric consistency correction is performed on the multi-view candidate image frames. A homography matrix is ​​established based on the corner detection results of the bone nail box edge. The multi-view images are mapped to a common reference plane. Distortion is eliminated by region overlap comparison to obtain the geometrically registered multi-view standard image.

[0026] Adaptive sharpness enhancement processing is performed on the multi-view standard images. Sharpness index is calculated using local frequency domain features. Multi-scale sharpening and contrast compensation are performed on regions below the threshold to generate a bone nail recognition image sequence with uniform sharpness distribution.

[0027] During implementation, the bone screw box should be placed at a fixed acquisition station equipped with multiple industrial cameras, including at least one top-view camera and two oblique-view cameras at different angles. The top-view camera should be perpendicular to the normal of the bone screw box surface, while the optical axis of the oblique-view cameras should form an angle of 25° to 35° with the normal to balance avoiding obstruction and maintaining sharpness. All cameras achieve synchronous acquisition through the same hardware trigger source, and a linear polarizer is fixedly installed in front of the lens. The light source consists of a set of adjustable ring LEDs and two symmetrical strip LEDs. Each light source is covered with an adjustable polarizer, and the polarization direction is adjusted by an electric rotation controller in 1° increments, ensuring that the polarization direction of the incident light intersects the lens polarizer at a 90° angle, thereby effectively suppressing specular reflection from the bone screw's metal surface. The color temperature of the light source is maintained between 5000K and 5500K, and the illuminance is maintained between 800 and 1200 lux through luminous flux adjustment to avoid local overexposure or underexposure. During acquisition, the camera exposure time and gain are locked during system initialization to ensure consistent grayscale distribution across different frames. The images obtained through the above configuration constitute the original multi-view image sequence, which includes timestamps, camera IDs, and exposure parameter metadata to ensure subsequent traceability.

[0028] After the original multi-view image sequence is acquired, it enters the timing synchronization module. The timing synchronization module first reads the hardware timestamp of each frame to establish multi-view pairing relationships within the same acquisition cycle. If a frame is lost at a certain viewpoint, it is filled in with the nearest neighbor frame to ensure that each pairing includes all expected views. To eliminate image jitter caused by mechanical vibration or manual operation, the system continuously acquires three frames within each viewpoint and performs temporal median filtering at the pixel level. Specifically, it takes the median of three grayscale values ​​at the same pixel location as the output value, thereby suppressing occasional noise and local artifacts. The output after this processing is a temporally consistent multi-view candidate image frame, each frame accompanied by its index in the multi-view pairing.

[0029] Candidate image frames from multiple viewpoints undergo geometric consistency correction to eliminate perspective distortion. First, the Canny operator is used to extract edges in each frame, with a low threshold set to 0.66 times the gradient median and a high threshold set to twice the low threshold. Then, Shi-Tomasi corner detection is performed on the edge map, with a quality factor of 0.01, a minimum spacing of 10 pixels, and a sub-pixel refinement algorithm to pinpoint corner positions to 0.1 pixels. A one-to-one correspondence is established between the four detected bone screw box edge corners and a standard reference rectangle. After eliminating mismatches using the RANSAC algorithm, the homography matrix is ​​calculated. The homography matrix is ​​then used to perform a projective transformation on the candidate frames, mapping the bone screw box surface to an orthographic rectangle. The corrected frames are projected onto a unified reference plane coordinate system, with the pixel resolution fixed at 0.05 mm / pixel (converted to millimeters). To further ensure geometric consistency between different viewpoints, the system calculates the region overlap ratio for adjacent corrected viewpoint frames. If the overlap ratio is below 95%, the corner set is readjusted and the calculation is repeated until the requirement is met. The output of geometric consistency correction is a multi-view standard image, with each image having a uniform scale and orientation reference.

[0030] After obtaining multi-view standard images, adaptive sharpness enhancement is required. The system divides each standard image into 32×32 local sub-blocks, calculates the Laplacian variance for each sub-block, and determines the sub-block to be blurry if the value is below a preset threshold of 200. Multi-scale sharpening is performed on blurry sub-blocks, first applying unsharpened mask filtering at scales of 1.0, 1.5, and 2.0, then weighted averaging the results; simultaneously, local histogram equalization is performed to adjust contrast and restore texture details. Unblurred sub-blocks undergo only mild contrast stretching. After processing, the sub-blocks are stitched together to restore the complete image, and Gaussian smoothing is used to reduce boundary artifacts. The final output sequence of bone nail recognition images not only has geometric consistency but also maintains uniform sharpness and brightness distribution, providing high-quality input for subsequent boundary extraction and bone nail structure atlas generation.

[0031] Step S2: Based on the bone screw recognition image sequence, extract the boundary information of the bone screw placement position and generate a bone screw structure map to represent the relative positional relationship of the bone screw placement area.

[0032] In step S2, processing is required based on the bone screw identification image sequence obtained in step S1. First, the usable placement area inside the bone screw box needs to be located in the image. Typically, the upper surface of the bone screw box has regular or semi-regular holes or grooves, each corresponding to a potential bone screw placement location. To extract these location boundaries, region segmentation is performed on the perspective-corrected image. Methods such as adaptive thresholding, edge enhancement with contour tracking, or deep learning semantic segmentation networks can be used to distinguish the hole areas from the background. In practice, the boundaries of empty slots that are close to the shape and size of the bone screw can be identified as reference regions, and a connected component labeling algorithm can be used to extract the contours of all potential placement slots.

[0033] After extracting the boundary contours, these contours need to be further standardized to accurately reflect spatial relationships. Specifically, the geometric center coordinates, major and minor axis directions, and bounding rectangle of each slot boundary need to be calculated to ensure consistency in the identified position of the same bone screw box in different images. To avoid the influence of noise, multi-view information fusion can be used, that is, spatial registration of the boundary features of the same slot in the top and side views to obtain a more robust slot representation. If the identified slots have defects or discontinuous edges, template matching can be used to compare with the standard bone screw box hole distribution, and the missing boundaries can be repaired by interpolation or fitting.

[0034] After extracting and standardizing the boundaries of all placement areas, a structured descriptive system needs to be established on the entire plane of the bone screw box. This system is typically stored in the form of a two-dimensional coordinate matrix or an ordered index table, meaning each placement area corresponds to a unique index number and its absolute coordinate position on the plane. Simultaneously, the adjacency relationships between each slot also need to be recorded, such as the indices of adjacent slots (top, bottom, left, and right), for subsequent spatial consistency judgment in the recognition results. In this way, a bone screw structure map is formed, which can fully reflect the number, spatial distribution, and relative relationships of each placement position in the bone screw box.

[0035] During the generation of the bone screw structure atlas, a global consistency check should be performed to ensure that the number of generated placement areas matches the standard bone screw box specifications and that the coordinate distribution of each slot meets the regularity requirements of the bone screw box's physical structure. If there are excessive deviations in quantity or abnormal local distributions, boundary extraction needs to be re-executed or a backup recognition algorithm needs to be called for supplementation. After the above processing, the obtained bone screw structure atlas has a stable indexing system and clear geometric boundaries, which can serve as the basic data input for feature comparison and collaborative reasoning in the subsequent step S3, ensuring sufficient accuracy and consistency in spatial positioning throughout the entire recognition and usage confirmation process.

[0036] Furthermore, the step of extracting boundary information of the placement position of the bone screws based on the bone screw recognition image sequence and generating a bone screw structure atlas to represent the relative positional relationship of the bone screw placement areas includes:

[0037] Multimodal boundary information extraction is performed on the bone nail recognition image sequence, and edge detection based on gray-level gradient and region segmentation based on convolutional features are combined to obtain a set of boundary information for the placement position of the candidate bone nail;

[0038] Spatial constraint optimization is applied to the boundary information set of the candidate bone screw placement positions. Each boundary information is matched with a preset bone screw box geometric template, and the geometric shape of the boundary information is corrected by least squares fitting. The result is a geometrically corrected boundary information set of bone screw placement positions.

[0039] Based on the set of boundary information of the bone screw placement positions after geometric correction, a topologically consistent network is constructed. The relative coordinate difference and row and column index positions between each boundary information and the adjacent boundary information are calculated to generate a boundary information mesh of bone screw placement positions with topological relationships.

[0040] In the boundary information grid of the bone screw placement position, a confidence index is marked for each boundary information. The confidence index is calculated based on the stability of boundary information extraction, the fitting error with the geometric template, and the residual after topological consistency correction. Finally, a bone screw structure map with confidence index is output.

[0041] For each frame of the bone screw recognition image sequence, the gradient magnitude and direction are first calculated in the grayscale domain. High and low thresholds are adaptively set based on the median of the overall grayscale histogram to generate edge detection results based on grayscale gradients. Simultaneously, region segmentation results based on convolutional features are calculated on the same frame. This convolutional feature-based region segmentation uses training samples labeled with bone screw placement positions as priors, outputting a probability map of each pixel belonging to a bone screw placement position. The edge detection results based on grayscale gradients and the region segmentation results based on convolutional features are then fused using a multimodal approach. The probability map of the region segmentation is used as a soft mask to constrain edge connectivity, and edge strength is used as the stopping condition for region growth. This yields a set of boundary information for candidate bone screw placement positions at the connected component level. The boundary information of each bone screw placement position is stored with the following essential fields: boundary information polygon, physical coordinates of the boundary information center point, boundary information principal axis direction angle, and boundary information circumscribed length and width. When there are multiple candidates for the same position in multiple view frames, the boundary information polygon is weighted point by point with view quality weight to eliminate single-view occasional noise, thereby ensuring that the set of boundary information of candidate bone screw placement positions is stable across view frames.

[0042] After obtaining the boundary information set of candidate bone screw placement positions, this set is placed in the spatial constraint optimization process. Spatial constraint optimization uses the bone screw box geometric template as the sole spatial prior. The bone screw box geometric template takes the reference edge of the inner surface of the bone screw box as its origin, and provides the nominal row and column spacing, nominal number of rows and columns, approximate shape parameters of the nominal boundary information polygon, and allowable deviation range for each standard bone screw placement position in millimeters. When performing a one-to-one matching between the boundary information set of candidate bone screw placement positions and the bone screw box geometric template, the objective function is the sum of the squares of the differences between the physical coordinates of the boundary information center point and the nominal coordinates of the bone screw box geometric template. Simultaneously, the angular deviation between the principal axis direction angle of the boundary information and the given direction of the bone screw box geometric template is introduced as a regularization term. The optimal global translation and rotation parameters are solved through least-squares fitting to align the entire boundary information set of candidate bone screw placement positions with the bone screw box geometric template. Based on this global alignment, a local shape correction using least-squares fitting is applied to the vertex coordinates of each boundary information polygon to constrain irregular perturbations within the allowable deviation range of the bone screw box geometry template. When the fitting residual of a boundary information with any nominal position exceeds the upper limit, it is marked as pending review and temporarily excluded from subsequent mesh construction. After completing the above fitting and correction, a set of boundary information for the geometrically corrected bone screw placement positions is output. Each boundary information in this set includes its nominal row and column candidate number that matches the bone screw box geometry template, as well as the residual value of the local shape correction.

[0043] Based on the boundary information set of geometrically corrected bone screw placement positions, a topologically consistent network is constructed. The topologically consistent network uses each geometrically corrected boundary information as a node and the spatial proximity between nodes as edges. First, the center point of each geometrically corrected boundary information is projected along the row and column directions of the bone screw box geometric template in the physical coordinate system. Density-based clustering is used to cluster the projected coordinates, resulting in row cluster sequences and column cluster sequences. These are then naturally numbered according to the order of the cluster centers, forming the initial allocation of row and column index positions. Subsequently, for each boundary information, a unique row and column index position is determined based on the membership degree of its center point in the row and column cluster sequences. The relative coordinate difference between the boundary information and the row and column cluster centers is calculated. Adjacency relationships are established based on the criterion that the difference between the row and column index positions of adjacent rows and columns is equal to one and the relative coordinate difference is within the allowable range of the bone screw box geometric template, resulting in a topologically consistent network with a set of adjacency edges. For cases where row and column index positions are missing, the corresponding row and column index positions are retained as placeholder nodes and the break intervals are recorded to ensure that the "should be but not" positions still have constraints during subsequent identification. The above topological consistency network is regularized and arranged on a two-dimensional plane to generate a boundary information grid of bone nail placement positions with topological relationships. Each cell in the grid corresponds one-to-one with each boundary information line after geometric correction or is an empty placeholder. The pointer relationship between the row and column index positions and adjacent cells is explicitly stored for subsequent steps to perform position consistency checks and context inference.

[0044] Within the boundary information grid of the bone screw placement location, a confidence index is calculated and labeled for each boundary information. The confidence index consists of three parts, normalized within the range of zero to one: boundary information extraction stability, the inverse score of the fitting error with the bone screw box geometric template, and the inverse score of the residual after topology consistency correction. Boundary information extraction stability is measured by the intersection-union ratio before and after multimodal fusion, the proportion of repeated detections of the boundary information polygon in multi-view frames, and the standard deviation of the principal axis direction angle of the boundary information across multiple views. The inverse score of the fitting error with the bone screw box geometric template is obtained by weighting the center point physical coordinate error after global least squares fitting with the residual of local shape correction and performing an inverse mapping. The inverse score of the residual after topology consistency correction is comprehensively evaluated based on the number of times the row and column index positions are reassigned, the average deviation of the relative coordinate difference from the nominal value of adjacent boundary information, and the interpolation bias introduced by placeholder nodes. The three scores are linearly combined according to preset weights to form the confidence index, which is then written into the corresponding cell of the boundary information grid of the bone screw placement location. At this point, a bone screw structure map with confidence level annotations is output. The bone screw structure map with confidence level annotations stores the boundary information, relative coordinate difference, adjacency relationship and confidence index of the geometrically corrected bone screw placement position using row and column index positions as keys. All coordinates are defined in a unified physical coordinate system within the bone screw recognition image sequence, and can be directly called by subsequent steps for feature extraction, comparison recognition and position consistency constraint reasoning, without the need to perform geometric alignment or topological reconstruction again.

[0045] Step S3: Combine the bone screw structure map with the bone screw recognition image sequence to extract the image features of the bone screw in each placement area, and perform comparison recognition based on the standard shape features of the bone screw to obtain the recognition result containing bone screw category information and recognition confidence.

[0046] In step S3, the bone nail recognition image sequence output from step S1 and the bone nail structure map generated in step S2 are used as inputs. Each placement region index in the structure map is traversed sequentially in the same coordinate system. For the placement region corresponding to the current index, the region of interest is first cropped in the top view frame based on the region's bounding polygon recorded in the structure map, and a safety boundary of at least 5 pixels is extended around it to include possible slight placement deviations. To avoid edge reflection interference, it is preferable to mask and suppress the bright saturated areas in HSV space, and use a brightness equalization method based on illuminance-reflection decomposition to normalize the illumination of this small region, so that bone nails of the same specification present comparable grayscale distributions under different shooting conditions. The cropped region is affinely rotated along the region's principal axis direction given in the structure map, so that the long axis of the suspected bone nail is approximately aligned with the horizontal axis of the image. Simultaneously, the region is resampled to a uniform resolution according to the pixel spacing calculated using the millimeter scale in step S1, ensuring the comparability of subsequent geometric measurements (length, diameter, etc.).

[0047] After geometric and lighting normalization, two complementary image features are extracted from the region. One type is quantitative features based on contour and geometry, including the main contour obtained through Canny and morphological processing, the major and minor axis lengths of the minimum bounding rectangle, the major and minor axis ratios, estimated end fillet radii, the intensity projection curve along the major axis, and the one-dimensional spectral peak position and amplitude based on the projection curve, used to characterize the presence or absence of threaded texture and its pitch. The other type is descriptor features based on local texture and keypoints. This can be achieved by using scale-invariant keypoint detection and calculating directional gradient histogram-type descriptors, or by calling a pre-trained lightweight classifier to perform forward inference on the small region without changing the method properties, obtaining preliminary confidence scores for each candidate category. To improve robustness, the geometric metrics and descriptor matching results obtained in the top-view frame are used to extract the corresponding regions with the same index in the side-view frame according to the spatial correspondence of the structural map, and the above processing is repeated to obtain independent scores and geometric metrics for the side-view. The results of top and side views are fused according to the view quality weight. The weight can be automatically calculated based on the sharpness evaluation (e.g., Laplacian variance), the overexposure area ratio, and the mask coverage ratio. The view with higher quality receives a higher weight.

[0048] The comparison and identification process uses a "standard bone screw shape feature library" as a reference. This feature library pre-stores the baseline parameters of various specifications and models of bone screws used in the hospital, including at least nominal length, nominal diameter, allowable tolerance range, typical head and tail shape features, and texture spectrum features under standard imaging conditions. The length and diameter extracted from the current region are matched with the standard parameters using pixel-to-millimeter conversion. If the geometric measurement falls within the tolerance range of a candidate specification, the candidate specification is assigned a geometric similarity score. The contour shape is compared with the baseline shape in the library using Fourier description or Hu invariant moments to obtain a shape similarity score. The position and amplitude of the main peak of the texture spectrum are compared with the nominal range of the specification in the library to obtain a texture similarity score. If a classifier is used, the classification probability of the specification can also be obtained. The above scores are combined into a weighted average to form the comprehensive similarity of the specification. The weights are set in descending order of geometric measurement, shape, texture, and classification probability by default, and can be parameterized according to the difficulty of distinguishing different types of bone screws in the hospital. To avoid false detections, when the overall similarity is below a preset threshold, the area is classified as a "no bone nail" or "cannot be determined" placeholder. When the overall similarity exceeds the threshold but there are two or more similar items, the one with the highest overall similarity is selected as the category information, and the second highest similarity and its similarity are retained as alternative items for behavior pattern comparison and anomaly correction in subsequent steps.

[0049] The confidence score calculation consists of three parts: first, a comprehensive similarity normalized value, reflecting the degree of proximity of regional features to a certain standard specification; second, a multi-view consistency measure, defined as the consistency ratio between top view and side view in terms of length, diameter, and category, with bonus points awarded for results with high consistency ratios; and third, a regional quality coefficient, calculated based on the region's sharpness, overexposure ratio, and effective texture area ratio, used to downweight low-quality imaging results. These three factors are linearly synthesized with configurable weights to form the final confidence score, with an output range of 0 to 1. The final recognition result uses the structural map index as the key, recording whether a bone screw exists in the placement area, the bone screw category information (including specifications, estimated length and diameter in millimeters, and the orientation angle of the bone screw within the area), the local scores of top view and side view, the fused recognition confidence score, and the coordinates of the cropped region used for backtracking. For regions deemed "undeterminable" or with confidence levels below a threshold, a complete record is still output, but the category information is marked as "unknown," along with the primary reason for the unknown status, such as geometric measurement exceeding limits, shape mismatch, or insufficient texture signal. This allows for targeted correction in subsequent steps during behavioral pattern comparison. Through this process, the bone screw structure atlas and bone screw recognition image sequences are organically combined, generating a set of recognition result data containing bone screw category information and recognition confidence for each placement area, which can be directly used for subsequent collaborative reasoning.

[0050] Furthermore, the process of combining the bone screw structure atlas with the bone screw recognition image sequence to extract image features of the bone screw in each placement area, and performing comparison and recognition based on the standard shape features of the bone screw to obtain a recognition result containing bone screw category information and recognition confidence, includes:

[0051] For each placement area in the bone screw structure atlas, the bone screw recognition image sequence is called to perform region slicing, and the corresponding region slice is output. The region slice has row and column index and relative position coordinates.

[0052] Geometric feature extraction is performed on the sliced ​​region, calculating the major axis length, minor axis length, aspect ratio, center point coordinates, and principal axis direction angle of the bounding rectangle, and outputting a geometric feature vector containing the geometric features;

[0053] The geometric feature vector is input into the texture feature extraction module, and a two-dimensional fast Fourier transform and gray-level co-occurrence matrix calculation are performed on the region slices to output a texture feature vector containing spectral peak features and texture energy distribution.

[0054] The geometric feature vector and the texture feature vector are jointly input into the standard shape matching module, and compared one by one with the preset bone nail standard shape feature library. Geometric similarity, texture similarity and spectral similarity are calculated, and candidate category information and corresponding category matching scores are output.

[0055] The candidate category information and the category matching score are input into the multi-view fusion module. The category matching scores of the top view slice and the side view slice are weighted and averaged, and the consistency of the row and column indexes is corrected. The final bone nail category information and recognition confidence are output, forming a recognition result that includes bone nail category information, row and column indexes and recognition confidence.

[0056] First, it is necessary to ensure that the bone screw structure atlas and the bone screw identification image sequence are aligned in the same physical coordinate system, and that each placement area has a clear row and column index and relative position coordinates in the atlas. The goal of this step is to call the corresponding areas in the image sequence based on the atlas information, extract the image features of the bone screw in each placement area one by one, and compare them with the standard shape feature library of bone screws for identification, thereby outputting bone screw category information and identification confidence.

[0057] In practice, the system first traverses each placement area in the bone screw structure atlas. For each area, based on its row and column indices and relative position coordinates, a corresponding region slice is extracted from the bone screw recognition image sequence. This region slice must include the complete slot area and be additionally extended by a certain boundary width (usually 5 to 10 pixels) to ensure that the bone screw edge is not lost due to truncation errors. The output region slice, along with its corresponding row and column indices and relative position coordinates, is saved together to provide an index for subsequent recognition and result association.

[0058] After the region slices are generated, geometric feature extraction is required. Geometric feature extraction includes calculating the major and minor axis lengths of the circumscribed rectangle of the bone screw within the region slice, and using the ratio of the major to minor axis as the aspect ratio; simultaneously, calculating the coordinates of the center point of the circumscribed rectangle to align with the region's position in the structural atlas; furthermore, calculating the principal axis angles and using ellipse fitting or principal component analysis to determine the main orientation of the bone screw within the slice. These geometric parameters are then combined into a geometric feature vector, which serves as a standardized numerical representation output.

[0059] Subsequently, the geometric feature vector is fed into the texture feature extraction module. In this module, a two-dimensional fast Fourier transform is performed on the region slice to obtain its spectral information, and the frequency position and amplitude of the main peaks are extracted from the spectrum as spectral peak features. At the same time, the gray-level co-occurrence matrix is ​​calculated on the original grayscale image to obtain texture statistics such as energy, contrast, and entropy. In this way, a texture feature vector that can characterize the surface texture and spatial distribution features of the bone nail is generated.

[0060] Next, the geometric feature vector and texture feature vector are jointly input into the standard shape matching module. This module has a pre-set standard shape feature library for bone screws, which stores the baseline geometric parameter range, typical texture distribution patterns, and spectral features for each bone screw specification. During matching, the system calculates the geometric similarity, texture similarity, and spectral similarity between the input vector and each specification in the feature library, and then weights and combines the three types of similarity into a comprehensive score. The output includes candidate category information and the corresponding category matching score.

[0061] To avoid recognition bias caused by a single viewpoint, the system further inputs candidate category information and category matching scores into a multi-view fusion module. This module utilizes the candidate category matching results of top-view and side-view slices under the same row and column indices, and performs a weighted average by applying a sharpness weight to the category matching scores of each viewpoint. If the categories output by the two views are inconsistent, the one with the higher overall score is prioritized, and correction is made based on the consistency of row and column index positions to eliminate single-view misjudgments. The final output includes bone nail category information and recognition confidence. The recognition confidence is numerically represented between 0 and 1 and is output along with the row and column indices, thus forming a recognition result that includes bone nail category information, row and column indices, and recognition confidence.

[0062] For example, in orthopedic surgery at a hospital, standard-model bone screw cases are used to manage surgical consumables. Before surgery begins, the bone screw cases are placed on a fixed acquisition table equipped with one top-view industrial camera and two side-view industrial cameras arranged at a 30° angle. The light source uses a ring-polarized LED, maintaining an illuminance of 1000 lux and a color temperature of 5200K, with polarizing filters suppressing reflections from metal surfaces. The camera resolution is 2048×2048 pixels, with a pixel calibration of 0.05 mm / pixel. The system acquires multi-view images of the bone screw cases and performs geometric correction and sharpness enhancement to form a sequence of bone screw recognition images.

[0063] During the processing phase, the system reads a bone screw structure map, which indicates the row and column indices and relative position coordinates of each bone screw placement area inside the bone screw box. The system then calls up the corresponding area one by one, extracting a region slice from the bone screw recognition image sequence. For example, at the position of row 3, column 5, the system extracts a region slice that includes the complete slot and extends outward by 10 pixels as a boundary buffer.

[0064] The system performs geometric feature extraction on the sliced ​​area. By fitting a bounding rectangle, the system obtains a major axis length of 12.0 mm, a minor axis length of 3.0 mm, an aspect ratio of 4.0, a center point coordinate of (15.5 mm, 23.2 mm), and a principal axis angle of 5°. These parameters are combined to form the geometric feature vector.

[0065] Next, the system performs a two-dimensional fast Fourier transform on the slice of this region, obtaining obvious spectral peaks. The main peak frequency corresponds to a texture pitch of approximately 0.5 mm. Subsequently, the gray-level co-occurrence matrix is ​​calculated, yielding an energy value of 0.75, a contrast value of 0.42, and an entropy value of 1.15. These values ​​form a texture feature vector, used to characterize the surface properties of the bone screw.

[0066] Both geometric and texture feature vectors are input into the standard shape matching module and compared with the standard shape feature library of bone screws. The standard parameters for this type of bone screw stored in the feature library are: length 12.0±0.2 mm, diameter 3.0±0.1 mm, and pitch 0.5±0.05 mm. The comparison results show a geometric similarity of 0.97, a texture similarity of 0.92, a spectral similarity of 0.95, and a weighted comprehensive score of 0.95. The system outputs the candidate category as "bone screw with a diameter of 3.0 mm and a length of 12 mm", with a matching score of 0.95.

[0067] To further improve robustness, the system inputs the matching scores of the top view slice and the two side view slices into the multi-view fusion module. The top view slice matching score is 0.95, the left view slice matching score is 0.94, and the right view slice matching score is 0.92. After sharpness-weighted averaging, the overall score is 0.94. Combined with row and column index consistency confirmation, the recognition result for this region is "bone nail with a diameter of 3.0 mm and a length of 12 mm", with a recognition confidence of 0.94.

[0068] Finally, the identification result, along with the row and column index (3,5), is written into the identification result table, outputting the identification result containing bone screw category information, row and column index, and identification confidence score. This process, completed across all locations in the bone screw box, forms a complete distribution and confidence score mapping of bone screw categories, providing high-quality data input for subsequent pre- and post-operative comparisons and dosage confirmation.

[0069] Step S4: Compare the identification results with the historical bone screw usage behavior pattern, identify potential abnormal identification items, and output the enhanced identification results after behavior pattern correction. The historical bone screw usage behavior pattern is constructed based on historical bone screw usage behavior data related to surgery type, doctor identification, and bone screw specifications.

[0070] When implementing step S4, the identification result output from step S3 is first used as input data. This result includes category information, geometric measurements, and identification confidence for each bone screw placement area. Subsequently, a pre-established historical usage behavior database is accessed. This database is constructed from long-term accumulated clinical surgical data, recording typical usage patterns for different surgical types, different surgeons' operating habits, and different bone screw sizes. This includes common distributions of screw usage quantities, common combinations of sizes, and statistical patterns of placement and usage order. This historical data is modeled to form historical bone screw usage behavior patterns. These patterns can be trained and stored using probability distribution-based statistical models, time-series pattern mining algorithms, or neural network-based predictive models. This allows the model to provide corresponding reasonable expected values ​​for screw usage and possible spatial distributions given the input surgical type, surgeon identification, and preset bone screw sizes.

[0071] After a new round of recognition results is input, the system needs to compare the category identified for each placement area with the behavioral pattern in that surgical scenario. If the recognition results show a bone screw category that is significantly inconsistent with historical patterns at a certain location—for example, a certain type of bone screw appearing in a corner area that is not usually used, or the number identified far exceeding the normal range—the system will mark this item as a potential anomaly. For potential anomalies, the system does not directly discard the results but performs secondary reasoning through a behavioral pattern correction mechanism. This mechanism comprehensively considers the recognition output of neighboring areas, the reasonableness of the overall quantity, the distribution of the same type of bone screw in other locations, and performs weighted fusion with the probability distribution of historical patterns. If a certain category achieves higher consistency between pattern prediction and image recognition after correction, the original recognition result is replaced with the corrected category, and the corresponding confidence value is updated. The corrected confidence value is not only based on visual recognition similarity but also takes into account the reasonableness weighting of historical patterns, thereby improving the stability of the overall results.

[0072] Throughout the comparison and correction process, detailed information on the anomaly identification items must be saved, including their initial identification category, original confidence level, anomaly cause label, and corrected category and new confidence level. This enhanced identification result replaces the original result from step S3 in the final output, becoming a valid input for subsequent steps comparing pre- and post-operative differences. In this way, the identification result not only relies on the direct calculation of visual features but also introduces historical behavioral patterns as semantic-level correction, enabling the system to maintain high robustness in the face of occlusion, noise, or abnormal lighting, ensuring the accuracy of bone screw identification and dosage confirmation.

[0073] Furthermore, the identification results are compared with historical bone screw usage behavior patterns to identify potential anomalies and output enhanced identification results corrected by the behavior patterns. The historical bone screw usage behavior patterns are constructed based on historical bone screw usage behavior data related to surgery type, doctor identification, and bone screw specifications, including:

[0074] The recognition results are analyzed one by one, and the bone nail category information, row and column index, spatial location and recognition confidence contained in the recognition results are extracted to generate corresponding recognition behavior features;

[0075] The identified behavioral features are matched with the historical usage data of the bone screws, and the corresponding historical behavioral features are extracted under the conditions of surgical type, doctor identification and bone screw specifications to generate a set of historical behavioral features.

[0076] The identified behavioral features are compared one by one with the set of historical behavioral features. The difference between the identified behavioral features and the historical behavioral features is calculated, and the potential anomaly identification items and their difference indices are output.

[0077] The potential anomaly identification items and their difference index are jointly analyzed with the identification results under the adjacent row and column indexes. Combined with the typical distribution patterns recorded in the bone nail historical usage behavior patterns, the potential anomaly identification items are corrected, and the corrected bone nail category information and the corrected identification confidence are output.

[0078] The corrected bone screw category information and the corrected recognition confidence are combined with the recognition result without correction to generate an enhanced recognition result after behavioral pattern correction. The enhanced recognition result includes bone screw category information, row and column index, spatial location and recognition confidence.

[0079] In practical implementation, the recognition result output in step S3 is used as input. The recognition result includes bone screw category information, row and column index, spatial location, and recognition confidence for each placement area. The system reads the recognition result line by line, extracting fields in the order of bone screw category information, row and column index, spatial location, and recognition confidence. Under the same physical coordinate system, the spatial location is represented as millimeter-level coordinate pairs based on the origin of the bone screw structure map. The above four items are encoded and concatenated at fixed positions to form recognition behavior features. The bone screw category information is encoded in the form of specification code and length / diameter numerical pairs; the row and column index uses integer pairs; the spatial location uses floating-point numerical pairs; and the recognition confidence uses a real number from zero to one. All values ​​are retained to at least three decimal places to ensure subsequent calculation accuracy. The recognition behavior features are written into a sequential container in memory with a timestamp to support subsequent matching and comparison.

[0080] To obtain a reference prior that is strictly aligned with the identified behavioral characteristics, the system retrieves historical usage data of bone screws within the same transaction and filters it using surgery type, doctor identifier, and bone screw specification as a joint key, retaining only records that meet the current surgery type, doctor identifier, and bone screw specification. On the filtered historical usage data, the system aggregates by row and column index to obtain a location conditional distribution and calculates the empirical probability of bone screw category information, the mean and covariance of common spatial location offset vectors for each row and column index, the distribution parameters of historical usage frequency, and the relative usage order quantiles on the surgical procedure timeline. These statistics are encoded as historical behavioral features using the same field order and data type as the identified behavioral characteristics, and all historical behavioral features corresponding to all row and column indices are aggregated into a historical behavioral feature set. For row and column indices without historical records, weighted interpolation of the nearest neighbor row and column indices in the same row or column is used to obtain the default parameters for the empirical probability and spatial location offset of bone screw category information, while marking the default identifier to avoid introducing uninterpretable extreme values ​​during difference calculation.

[0081] The system then compares the identified behavioral features with the historical behavioral feature set one by one. Difference calculations are performed under the same row and column index, avoiding cross-index mixing. The difference measure uses a weighted composite scalar measure, comprising four parts: First, a category difference term between the bone screw category information and the empirical probability of the bone screw category information in the historical behavioral features, defined as one minus the empirical probability of the bone screw category information under that row and column index; second, a deviation term between the spatial location and the mean spatial location offset in the historical behavioral features, defined as Mahalanobis distance and normalized to millimeters; third, a penalty term for recognition confidence, defined as one minus the recognition confidence, used to improve the difference sensitivity when recognition confidence is low; and fourth, a consistency term for the order of use, where if the difference between the time quantile of the identification occurring after the start of surgery and the relative order quantile in the historical behavioral features exceeds a threshold, the absolute value of this difference is used as a penalty. These four terms are linearly combined using weighted coefficients to form the difference measure, represented by a closed interval from zero to one. The weights are calibrated and fixed in the configuration file during deployment. For each identified behavioral feature, the system outputs a potential anomaly identification item and its difference index. When the difference index is greater than or equal to a set threshold, the identified behavioral feature is marked as a potential anomaly identification item, and the difference index is the corresponding difference index. When the difference index is lower than the set threshold, no anomaly is marked, but the difference index is still retained for subsequent joint analysis.

[0082] For identified potential anomalies, the system performs joint analysis on the potential anomaly and its dissimilarity index, along with the identification results under adjacent row and column indices. The adjacent row and column indices are fixedly specified in the configuration using either four-neighbor or eight-neighbor domains, and are used synchronously with the typical distribution patterns recorded in the historical usage behavior data of bone screws. Typical distribution patterns are represented by high-frequency category combinations of similar surgeries under the same doctor's identifier, the category association matrix of adjacent row and column indices, and the prior patterns of location occupancy. During joint analysis, without changing the row and column indices and spatial locations, the system enumerates the two candidate bone screw categories with the highest probabilities for each row and column index in the historical usage behavior data. It calculates the neighborhood consistency score of each candidate bone screw category under the adjacent row and column index category association matrix and the prior score under the empirical probability of that row and column index. A weighted average of these two scores, minus the dissimilarity score, is then introduced into the identification confidence level to obtain the posterior consistency score for each candidate bone screw category. If a posterior consistency score is significantly higher than the current nail category information, the candidate is replaced with the corrected nail category information, and the corrected recognition confidence is calculated using a weighted average of the posterior consistency score and the original recognition confidence. If no significantly better candidate exists, the original nail category information remains unchanged, and the recognition confidence is slightly reduced based on the magnitude of the difference index to reflect insufficient consistency between historical priors and the current recognition. The "significance" in the correction rule is defined with a fixed threshold, and the same threshold is used for all row and column indices to ensure consistency.

[0083] After processing all potential anomaly identification items, the system merges the corrected bone screw category information and the corrected identification confidence with the uncorrected identification results, maintaining the original row and column indices and spatial locations, to generate enhanced identification results corrected for behavioral patterns. The enhanced identification results output a record for each row and column index, containing bone screw category information, row and column index, spatial location, and identification confidence. For corrected records, the identification confidence uses the corrected confidence and includes archived values ​​of the difference index for subsequent auditing and traceability. For uncorrected records, the identification confidence remains the original confidence, while the corresponding difference index is retained for statistical analysis. The enhanced identification results are saved in the same serialization format as the identification results and are consistent with the indexing system of the bone screw structure atlas, allowing them to be directly consumed by subsequent steps without any additional format conversion or index reconstruction.

[0084] Step S5: Perform steps S1-S4 before and after the surgery to obtain enhanced recognition results before and after the use of bone screws. Calculate the difference information between the two results by comparing the corresponding positions to obtain bone screw usage difference information.

[0085] When implementing step S5, the same bone screw box needs to undergo the complete processing flow of steps S1 to S4 before and after the surgical procedure. Specifically, before the surgery, medical staff place the bone screw box in its initial state at the acquisition position. The acquisition device acquires multi-view images and obtains a bone screw recognition image sequence through edge extraction, corner correction, and perspective correction. Based on this sequence, a bone screw structure atlas is generated, and the images of each placement area are identified in combination with standard shape features. Finally, the results are compared and corrected by historical usage behavior patterns to obtain the enhanced recognition result before the surgery. After the surgery, the same bone screw box needs to be placed back at the acquisition position, and the same steps are repeated to complete image acquisition, structure atlas construction, feature comparison and recognition, and behavior pattern correction to obtain the enhanced recognition result after the surgery. Since the same acquisition parameters, image correction methods, and recognition strategies are used in the two processing steps, the consistency of the results in spatial reference and category judgment is ensured, making the comparative analysis comparable and accurate.

[0086] After obtaining enhanced identification results before and after surgery, the system compares all placement areas in the structural atlas one by one. During the comparison process, the system first confirms whether the positions correspond based on the area index, and then compares the category information and identification confidence level. If the preoperative result shows that a certain type of bone screw is present at the location with a high confidence level, but the postoperative result is empty or shows no bone screw, it is presumed that the bone screw at that location has been removed. If the same category and consistent confidence level are observed before and after surgery, it is determined that the status of the bone screw has not changed. If a new category or quantity appears after surgery that is inconsistent with the preoperative value, it is recorded as a difference, and further combined with historical usage behavior patterns to determine whether it is reasonable use or a potential abnormality. In this way, the system can automatically count the number of bone screws used, their specifications, and the locations of any suspicious points, forming bone screw usage difference information.

[0087] Throughout the process, discrepancies include not only increases or decreases in dosage but also potential anomaly markers resulting from inconsistencies in identification. All information is output in a structured manner for subsequent steps. By repeating the same identification and correction process before and after surgery and performing precise discrepancy comparisons, subjective errors from manual counting can be avoided, ensuring the objectivity and traceability of bone screw usage. Those skilled in the art can clearly reproduce the operation method of this step based on the above description.

[0088] Furthermore, steps S1-S4 are performed before and after the surgery to obtain enhanced identification results before and after the use of the bone screw, respectively. The difference between the two results is calculated by comparing corresponding locations to obtain bone screw usage difference information, including:

[0089] Before the surgery begins, a sequence of bone screw recognition images is acquired and steps S1 to S4 are executed sequentially to generate enhanced recognition results before bone screw use. The enhanced recognition results before bone screw use include bone screw category information, row and column index, spatial location, and recognition confidence.

[0090] After the surgery, bone screw recognition image sequence is acquired again and steps S1 to S4 are executed sequentially to generate bone screw enhanced recognition results after use. The bone screw enhanced recognition results after use include bone screw category information, row and column index, spatial location and recognition confidence.

[0091] The enhanced recognition results before and after the use of the bone screw are compared with each other under the same row and column index and spatial position. Position matching pairs are generated one by one, and a set of corresponding results containing category information before and after matching and recognition confidence before and after matching is output.

[0092] Based on the corresponding result set, the difference information is calculated, including whether the bone nail category information has changed, the difference in identification confidence, and the spatial position offset. A difference record is generated for each row and column index.

[0093] All discrepancy records are statistically summarized to output bone screw usage discrepancy information. This information includes changes in the number of screws used, distribution of changes in categories, abnormal distribution of spatial locations, and abnormal distribution of confidence levels, which are used to generate bone screw usage confirmation information later.

[0094] In this embodiment, the enhanced identification results of bone screws before surgery (obtained before the start of the operation) and after surgery (obtained after the operation) are used as inputs. Both are acquired, processed, and aligned using the same bone screw identification image sequence parameters to ensure that bone screw category information, row and column index, spatial location, and identification confidence are expressed in the same physical coordinate system and on the same scale. The enhanced identification results before bone screw use should be saved as an ordered set of records with row and column index as keys. Each record contains bone screw category information, row and column index, spatial location, and identification confidence. The spatial location is recorded as a two-dimensional coordinate pair in millimeters, and the identification confidence is recorded as a real number from zero to one and retained to at least three decimal places. The enhanced identification results after bone screw use are saved with completely consistent fields and precision requirements, and a timestamp is added under the same clock source to ensure traceability and consistency verification during subsequent comparisons.

[0095] To generate the data pairs required for corresponding comparison under the same row and column indices and spatial locations, the union of the row and column indices of the enhanced recognition results before and after the use of the bone screw is first taken as the traversal index set. During the traversal, if the same row and column index exists in both the pre-application and post-application enhanced recognition results of the bone screw, then the two records are directly used to form a position matching pair. If the same index exists only in the pre-application enhanced recognition result but not in the post-application enhanced recognition result, then a placeholder record is generated for the post-application enhanced recognition result side. The placeholder record maintains the same row and column index, sets its spatial position to the spatial position of the pre-application enhanced recognition result, sets the bone screw category information to a null value, and sets the recognition confidence to zero. Each pair of position matching results consisting of the enhanced recognition results before and after the use of the bone screw should be output to the corresponding result set. Each item in the corresponding result set should simultaneously store the category information before and after matching, the recognition confidence before and after matching, and the spatial position before and after matching, and maintain a one-to-one correspondence with the row and column indices to ensure the repeatability of subsequent difference calculations.

[0096] When calculating the difference information based on the corresponding result set, first determine whether the bone screw category information has changed for each positional matching pair. If the bone screw category information before and after matching are both valid categories and are different, it is recorded as a change; if one side is marked as null and the other side is a valid category, it is also determined as a change; if the bone screw category information before and after matching are both marked as null, it is determined as no change and excluded in subsequent statistics. The identification confidence difference is defined as the identification confidence after matching minus the identification confidence before matching, and positive and negative changes are recorded with consistent decimal precision; the spatial position offset is expressed as the Euclidean distance between the spatial position after matching and the spatial position before matching, and stored in millimeters. For the same positional matching pair, whether the bone screw category information has changed, the identification confidence difference, the spatial position offset, and the row and column index are written together into a difference record, and the category information before and after matching and the original identification confidence values ​​before and after matching are stored synchronously in the difference record for review and judgment during auditing.

[0097] After the difference records are generated, all difference records are statistically summarized to output the bone screw usage difference information. The change in the number of screws used is obtained by calculating the difference between the count of records that changed from valid categories to null values ​​and the count of records that changed from null values ​​to valid categories, and is saved in both total and category-specific formats. The category change distribution is generated by counting difference records that changed but remained valid before and after matching, according to the transition from "category information before matching → category information after matching," and aggregating them by row and column indexes to generate a detailed distribution. The spatial location anomaly distribution is generated by setting a spatial location offset threshold, listing difference records with spatial location offsets greater than or equal to the threshold by row and column indexes and spatial location, recording the offset in millimeters. The confidence level anomaly distribution is generated by setting a confidence level difference threshold, listing difference records with absolute confidence level differences greater than or equal to the threshold by row and column indexes, and simultaneously saving the original confidence level values ​​before and after matching, to trace suspicious identification fluctuations. The bone screw usage variation information is output in a structured form, strictly including four items: changes in the number of screws used, distribution of changes in categories, distribution of spatial location anomalies, and distribution of confidence anomalies. It also maintains a key-value relationship consistent with the row and column indices, so that it can be directly referenced when generating bone screw usage confirmation information without re-alignment or conversion.

[0098] To avoid misjudgments due to environmental fluctuations, the entire process is executed under a single coordinate system and uniform precision. The acquisition parameters, geometric alignment parameters, and scale conversion parameters of the bone screw recognition image sequence remain unchanged before and after the surgery. During the generation of position matching pairs, a floating tolerance of no more than one pixel is allowed for spatial positions to offset the discrete errors of sub-pixel corner regression without changing the row and column indices. During the formation of difference records, the statistical mean of the recognition confidence difference and spatial position offset is not calculated for null value marking cases to avoid lowering the overall statistical representativeness. Through the above specific rules, the enhanced recognition results before and after bone screw use are compared and matched under the same row and column indices and spatial positions to form corresponding result sets. The difference information calculated based on this is expressed as position-by-position difference records. Finally, the bone screw use difference information is obtained through statistical summarization, accurately reflecting the changes in the number of screws used, the distribution of changes in categories, the abnormal distribution of spatial positions, and the abnormal distribution of confidence, and providing direct, complete, and verifiable data input for the subsequent generation of bone screw use confirmation information.

[0099] Step S6: Based on the bone screw usage difference information, combined with surgical identification information, recognition confidence level and image acquisition time, generate bone screw usage confirmation information, and write the bone screw usage confirmation information into an immutable storage medium to achieve closed-loop traceability of bone screw usage records.

[0100] When implementing step S6, the bone screw difference information generated in step S5 is first used as the core input. This information clearly marks the changes in the position of each bone screw in the preoperative and postoperative identification results, including which bone screws were removed, which remained unchanged, and whether there were any abnormal identifications. Subsequently, this difference information needs to be combined with surgical identification information, which is usually provided by the surgical scheduling system and includes the unique surgical number, the performing department, the attending surgeon, the assistant surgeon, and the surgical time period. This information is used to bind the identification results to a specific surgical event. Simultaneously, the identification confidence parameter is integrated. This parameter originates from the quantification of the credibility of the identification results in steps S3 and S4 and is used to mark the reliability of each data point in the difference information. The image acquisition timestamp is also added to ensure that the time of each identification is clearly recorded, facilitating subsequent verification of the consistency of the timeline.

[0101] After integrating the above information, the system will generate a complete confirmation of bone screw usage. This confirmation not only lists the number and specifications of bone screws used during the operation, but also includes markers for suspicious items and corresponding confidence level descriptions, thus forming a traceable and verifiable record. The confirmation information should be stored in a structured format, such as a hierarchical structure based on JSON or XML, saving the index, category, status change, confidence level, collection time, and responsible personnel information for each bone screw location one by one to ensure consistency and operability during subsequent reading and comparison.

[0102] To ensure this information cannot be tampered with or deleted, it must be written to an immutable storage medium. Blockchain ledger technology can be used, recording each bone screw's confirmation information as a transaction on the chain, using a distributed consensus algorithm to guarantee data irreversibility and chronological order. Alternatively, tamper-proof storage hardware with write-once, read-many characteristics, such as WORM storage devices, can be used, where data is permanently stored after being written and cannot be modified through conventional methods. During the writing process, the system generates a hash checksum for each confirmation message and stores it along with the original content for subsequent auditing to verify integrity and authenticity. Once data storage is complete, a complete closed-loop traceability mechanism is formed. Any surgical screw record can be traced back to a specific surgical event through surgical identification information, and the storage medium ensures the long-term security and immutability of the data.

[0103] Furthermore, based on the bone screw usage difference information, combined with surgical identification information, recognition confidence level, and image acquisition time, bone screw usage confirmation information is generated and written into an immutable storage medium to achieve closed-loop traceability of bone screw usage records, including:

[0104] The bone screw is bound with the differential information and the surgical identification information to generate a set of differential information with a unique surgical identifier, which includes the surgical number, doctor's identity information and timestamp;

[0105] Read the difference records one by one in the difference information set, and combine the bone nail category information, row and column index, spatial location and recognition confidence in each difference record with the image acquisition time to generate a difference extended record with time sequence markers;

[0106] The consistency of the difference expansion records is checked. If the identification confidence is lower than the set threshold, it is marked as a low confidence record and a confidence correction label is attached. If the identification confidence is greater than or equal to the set threshold, the original confidence is maintained and the verified difference expansion records are output.

[0107] The verified difference extended records are sorted according to row and column index order and image acquisition time order. The cumulative changes in the number of nails used, the trajectory of category changes, and the trajectory of spatial position offset are calculated in the sorting results. A summary record of differences containing trajectory information is output.

[0108] The difference summary record is integrated with the surgical identification information to generate bone screw usage confirmation information. The bone screw usage confirmation information includes surgical number, doctor's identity information, changes in the number of screws used, category change trajectory, spatial position offset trajectory, and identification confidence distribution. The bone screw usage confirmation information is written into a storage medium with an immutable storage mechanism to ensure the uniqueness, integrity, and traceability of the information record.

[0109] In this embodiment, after generating the bone screw usage difference information, it is first necessary to bind it with the surgical identification information. The key to the binding operation is to ensure that the difference information uniquely corresponds to a specific surgical procedure. Therefore, a unique surgical number needs to be assigned to each surgery, and the doctor's identity information and collection timestamp need to be attached to the difference information. The surgical number can be automatically generated by the hospital management system and kept unique. The doctor's identity information is identified by the doctor's unique identification code or registration number within the hospital, and the timestamp should be recorded with the system time accurate to the second or even millisecond. Through this binding method, a set of difference information with unique surgical identifiers is generated, ensuring that data confusion or duplicate references will not occur in subsequent processing.

[0110] After binding is complete, the difference records in the difference information set need to be read one by one. The bone screw category information, row and column index, spatial location, and recognition confidence score in the difference records are combined with the image acquisition time to generate extended difference records with temporal markers. Temporal markers refer to explicitly marking the image acquisition time corresponding to each difference record, ensuring that each piece of difference information corresponds to an actual moment in the surgical process, thus enabling time-dimensional tracing and comparison. Therefore, the extended difference records not only contain spatial change information but also have complete temporal dimension markers, providing the necessary conditions for subsequent trajectory calculation.

[0111] After generating the difference expansion records, consistency verification is required. Specifically, the system determines whether the identification confidence level in each difference expansion record is greater than or equal to a set threshold. If the identification confidence level is lower than the set threshold, a confidence correction label needs to be added to the record to indicate that the record has potential uncertainty and requires special handling in statistical and decision-making processes. If the identification confidence level is greater than or equal to the threshold, the original confidence level is maintained, and the verified difference expansion record is output. This effectively distinguishes reliable data from low-confidence data, avoiding deviations in subsequent results due to excessively low identification confidence.

[0112] After completing the consistency check, the system sorts all verified difference extension records according to row and column index order and image acquisition time order. This ensures that all changes within the same row and column index are arranged chronologically, while also ensuring that records from different locations are aggregated according to the physical arrangement of the screw box. Based on the sorting results, the system calculates the cumulative change in the number of screws used, tracking the change from the presence to the absence of screws during the surgery. It also generates a category change trajectory, recording whether the category information of screws at the same location has been replaced or misused. Furthermore, it calculates the spatial position offset trajectory, determining whether the screws have been displaced or rearranged by comparing changes in position coordinates. Finally, it statistically analyzes the changes in confidence level, forming a confidence distribution to reflect the reliability level of the identification results. The difference summary records generated by the above processing can comprehensively reflect the use of screws during the surgery from four dimensions: quantity, category, spatial position, and confidence level.

[0113] After obtaining the summary records of discrepancies, the system integrates them with surgical identification information to form the final bone screw usage confirmation information. This information explicitly includes the surgical number, doctor's identity information, cumulative changes in the number of screws used, category change trajectory, spatial location offset trajectory, and identification confidence distribution. To ensure the uniqueness, integrity, and traceability of the information, the bone screw usage confirmation information is written to a storage medium employing an immutable storage mechanism. This storage medium can be a blockchain-based chain storage or a tamper-proof log file system within the hospital's information system. This immutable storage method ensures that once the bone screw usage confirmation information is written, it cannot be tampered with or deleted, thus achieving closed-loop traceability of surgical consumable usage records. This not only meets the hospital's stringent requirements for surgical compliance but also provides a solid evidentiary basis for assigning responsibility in cases of abnormal screw usage.

[0114] For example, in a hip replacement surgery numbered S2025-0315, the surgeon's identity information was D-4582, and the surgery time was March 15, 2025, from 09:00 to 11:00. During the surgery, the use of the bone screw box was fully monitored by the system.

[0115] Before the surgery begins, the system acquires images of the bone screw box based on steps S1 to S4 and performs recognition, generating enhanced recognition results before the use of the bone screws. These results include row and column indices, bone screw category information, spatial location, and recognition confidence. For example, a bone screw with a diameter of 3.0 mm and a length of 12 mm is recognized at row and column index (2,4), with spatial coordinates of (12.35 mm, 20.15 mm) and a recognition confidence of 0.96. A bone screw with a diameter of 4.0 mm and a length of 14 mm is recognized at row and column index (3,6), with spatial coordinates of (18.22 mm, 25.47 mm) and a recognition confidence of 0.93. The entire recognition result is accompanied by a timestamp "2025-03-15 08:59:45".

[0116] After the surgery, the system executes steps S1 to S4 again to obtain enhanced recognition results after the bone screw was used. For example, at row and column index (2,4), the spatial position remains consistent, but the bone screw category information is marked as null, with a recognition confidence of 0.00, indicating that the bone screw was used. At row and column index (3,6), a bone screw with a diameter of 4.0 mm and a length of 14 mm is still recognized, with a spatial position coordinate offset of (18.40 mm, 25.55 mm), and a recognition confidence of 0.91. The timestamp of the result is "2025-03-15 11:01:10".

[0117] The system compares the enhanced recognition results before and after the use of the bone screw one by one under the same row and column indices and spatial locations, generating a corresponding result set. In the corresponding record at row and column index (2,4), the category information before matching is "diameter 3.0 mm, length 12 mm", and the category information after matching is a null value. The recognition confidence before matching is 0.96, the recognition confidence after matching is 0.00, and the spatial location offset is 0.02 mm. In the corresponding record at row and column index (3,6), the category information before and after matching is consistent. The recognition confidence before matching is 0.93, the recognition confidence after matching is 0.91, and the spatial location offset is 0.18 mm.

[0118] Based on these corresponding results, the system generates difference records. For the record at row and column index (2,4), the difference information shows that the category information has changed, with a confidence difference of -0.96 and a spatial offset of 0.02 mm; for the record at row and column index (3,6), the category information has not changed, with a confidence difference of -0.02 and a spatial offset of 0.18 mm.

[0119] After all the discrepancies were statistically summarized, bone screw usage discrepancy information was generated. Among them, the change in the number of screws used was a reduction of 1 bone screw. The category change distribution showed that the value at row and column index (2,4) changed from "diameter 3.0 mm, length 12 mm" to null. The spatial location anomaly distribution showed an offset of 0.18 mm at row and column index (3,6). The confidence anomaly distribution showed a confidence difference anomaly of -0.96 at row and column index (2,4).

[0120] Based on the difference summary record, the system integrates it with the surgery number S2025-0315, doctor's identity information D-4582, and timestamp information to generate bone screw usage confirmation information. This information includes: surgery number S2025-0315, doctor's identity information D-4582, a decrease of 1 screw in the number of screws used, a category change trajectory showing that row and column index (2,4) changed from "diameter 3.0 mm, length 12 mm" to a null value, a spatial position offset trajectory showing that row and column index (3,6) had a displacement of 0.18 mm, and the recognition confidence distribution shows that most records are above 0.90, with only row and column index (2,4) showing an abnormal decrease.

[0121] Ultimately, the confirmation information regarding the use of the bone screw is written into an immutable storage medium using a blockchain structure. The generated hash value and the on-chain timestamp together ensure the uniqueness, integrity, and traceability of the record. In the event of a dispute or when an audit is required, the hospital can directly retrieve the corresponding confirmation information regarding the use of bone screws from this storage medium to determine whether the use of consumables during the surgical procedure complies with regulations, and can accurately pinpoint the specific doctor, the specific time, and the specific location of the bone screw.

[0122] Furthermore, the step involves reading each difference record in the difference information set, combining the bone nail category information, row and column index, spatial location, and recognition confidence score in each difference record with the image acquisition time to generate a time-series-marked extended difference record, including:

[0123] The difference records in the difference information set are parsed sequentially to extract the bone screw category information, row and column index, spatial location and recognition confidence. A unique difference record number is added to each difference record, and a difference record set containing the bone screw category information, row and column index, spatial location, recognition confidence and difference record number is output.

[0124] The set of difference records is combined with the image acquisition time. During the combination process, alignment is achieved through two methods: precise timestamp matching and missing time interpolation. If a difference record is missing a corresponding image acquisition time, a complete timestamp is generated by linear interpolation of adjacent image acquisition times, and the set of difference time records containing the image acquisition time is output.

[0125] The spatial location in the difference time record set is jointly modeled with the image acquisition time to form a continuous position sequence in which the spatial location changes over time. The trend value of the spatial location offset is calculated in the position sequence, and the difference position record set containing the position trend is output.

[0126] The recognition confidence in the difference location record set is jointly calculated with the image acquisition time to generate a confidence curve of recognition confidence changing over time. Abnormal fluctuation segments are detected in the confidence curve, and a confidence fluctuation mark is added to each abnormal fluctuation segment. The difference confidence record set containing the confidence fluctuation mark is output.

[0127] The set of difference confidence records is sorted according to the row and column index order and the image acquisition time order. The bone nail category information, row and column index, spatial location, recognition confidence, image acquisition time, difference record number and confidence fluctuation marker are integrated in the sorting result to generate the final difference extended record.

[0128] In practical implementation, the first step is to parse the set of difference information line by line. This set is generated in the previous step by comparing the enhanced recognition results before and after the use of the bone screw. Each difference record in the set contains bone screw category information, row and column indexes, spatial location, and recognition confidence. To ensure the uniqueness and traceability of subsequent processing, each difference record must be assigned a unique difference record number during parsing. This number can be generated using a combination of timestamps and row and column indices to ensure that no duplicate numbers appear within the same set. During this process, the output set of difference records not only contains the original bone screw category information, row and column indexes, spatial location, and recognition confidence, but also explicitly records the corresponding difference record number to ensure the integrity of the data structure.

[0129] After obtaining the set of difference records, it needs to be combined with the image acquisition time. Since different difference records may originate from images acquired at different times, a precise timestamp must be attached to each difference record. If, in some cases, an accurate image acquisition time cannot be obtained for a particular difference record, linear interpolation should be performed using the image acquisition times of adjacent time points to generate a reasonable supplementary timestamp. For example, when a difference record lacks time information, the intermediate interpolation time can be calculated using the difference between the acquisition times of the preceding and following records. After this processing, all difference records will include their image acquisition times, forming a set of difference time records.

[0130] Subsequently, further spatial and temporal modeling is required on the differential time record set. Specifically, the spatial location in each record is correlated with the image acquisition time, resulting in a continuous sequence of spatial locations changing over time. Within this sequence, the difference in spatial coordinates between adjacent time points is calculated to obtain the positional shift trend. Then, a moving average or regression fitting method is used to extract the overall trend value, enabling the identification of gradual displacement or sudden positional changes that may occur in the bone screw during surgery. The result of this process is a differential position record set that not only contains the original spatial locations but also includes the trend value of spatial location changes over time, allowing for the subsequent tracking of the spatial dynamics of the bone screw during use.

[0131] After obtaining the set of difference location records, time-series analysis of the recognition confidence is required. By jointly calculating the recognition confidence with the image acquisition time, a curve showing the change in confidence over time can be generated. Within this curve, those skilled in the art can detect abnormal fluctuations in recognition confidence by setting thresholds, such as significant drops or drastic fluctuations in confidence at certain times. Whenever such abnormal segments are detected, a confidence fluctuation marker needs to be attached to the corresponding difference record to clearly mark the abnormal segment of the confidence curve. In this way, stable recognition results can be distinguished from potentially problematic results, preventing short-term recognition errors from affecting the final confirmation. The final set of difference confidence records contains both the time-series change information of recognition confidence and the abnormal fluctuation markers.

[0132] After completing the above processing, the set of difference confidence records needs to be strictly sorted. The sorting criteria must include both row and column index order and image acquisition time order. The purpose of sorting is to ensure that the temporal changes of the same location throughout the entire surgical process can be completely recorded, while ensuring that the data from different locations maintains a consistent spatial arrangement logic. After sorting, the bone screw category information, row and column index, spatial location, identification confidence, image acquisition time, difference record number, and confidence fluctuation marker need to be uniformly integrated into each record to form the final difference extended record. The final output difference extended record will have a complete multi-dimensional information structure, preserving both static spatial and category attributes and introducing dynamic features such as time series, trends, and confidence fluctuations. This provides a complete, reliable, and traceable data foundation for the subsequent generation of difference summary records and bone screw usage confirmation information.

[0133] A second embodiment of this application provides an electronic device, the electronic device comprising:

[0134] processor;

[0135] The memory is used to store a program, which, when read and executed by the processor, executes the automatic identification and dosage confirmation method for bone nails based on multi-feature collaborative reasoning provided in the first embodiment of this application.

[0136] The third embodiment of this application provides a computer-readable storage medium storing a computer program thereon. When the program is executed by a processor, it executes a method for automatic identification and dosage confirmation of bone nails based on multi-feature collaborative reasoning provided in the first embodiment of this application.

[0137] Although this application discloses preferred embodiments as described above, it is not intended to limit this application. Any person skilled in the art can make possible changes and modifications without departing from the spirit and scope of this application. Therefore, the scope of protection of this application should be determined by the scope defined in the claims of this application.

Claims

1. A method for automatic identification and dosage confirmation of bone pins based on multi-feature collaborative reasoning, characterized in that, Comprise: Step S1: Collecting a multi-view image sequence of the bone nail box in the initial state, performing edge extraction, corner correction and perspective correction on the multi-view image sequence to obtain a bone nail recognition image sequence; Step S2: Based on the bone nail recognition image sequence, the boundary information of the bone nail placement position is extracted, and the bone nail structure atlas for representing the relative position relationship of the bone nail placement area is generated; Step S3: The bone nail structure atlas and the bone nail recognition image sequence are combined and processed, the image features of the bone nail in each placement area are extracted, and the comparison and recognition are performed based on the standard shape features of the bone nail to obtain the recognition result containing the bone nail category information and the recognition confidence, comprising: Call the bone nail recognition image sequence for each placement area in the bone nail structure atlas to perform area slicing, output the corresponding area slice, and the area slice has row and column indexes and relative position coordinates; Perform geometric feature extraction on the area slice, calculate the long axis length, short axis length, aspect ratio, center point coordinates and principal axis direction angle of the circumscribed rectangle, and output the geometric feature vector containing the geometric features; The geometric feature vector is input into the texture feature extraction module, and the two-dimensional fast Fourier transform and gray level co-occurrence matrix calculation are performed on the area slice to output the texture feature vector containing the frequency spectrum peak feature and the texture energy distribution; The geometric feature vector and the texture feature vector are jointly input into the standard shape matching module, and are compared with the preset bone nail standard shape feature library one by one to calculate the geometric similarity, texture similarity and frequency spectrum similarity, and output the candidate category information and the corresponding category matching score; The candidate category information and the category matching score are input into the multi-view fusion module, the category matching scores of the top view slice and the side view slice are weighted and averaged, and the consistency of the row and column indexes is corrected to output the final bone nail category information and the recognition confidence, forming the recognition result containing the bone nail category information, the row and column indexes and the recognition confidence; Step S4: Compare the recognition result with the bone nail historical use behavior mode, identify potential abnormal recognition items and output the enhanced recognition result after behavior mode correction, wherein the bone nail historical use behavior mode is constructed according to the historical bone nail use behavior data related to the operation type, the doctor identifier and the bone nail specification; Step S5: Steps S1-S4 are performed before and after the operation respectively to obtain the enhanced recognition result before the use of the bone nail and the enhanced recognition result after the use of the bone nail, and the difference information between the two is calculated through corresponding position comparison to obtain the bone nail use difference information; Step S6: Based on the bone nail use difference information, the operation identifier information, the recognition confidence and the image acquisition time, the bone nail use confirmation information is generated, and the bone nail use confirmation information is written into the non-tamperable storage medium to realize the closed-loop tracing of the bone nail use record.

2. The method of claim 1, wherein the method is based on multi-feature collaborative reasoning. The multi-view image sequence of the bone nail box in the initial state is collected, the multi-view image sequence is subjected to edge extraction, corner correction and perspective correction to obtain a bone nail recognition image sequence, comprising: A multi-view image sequence of the bone nail box in an initial state is collected, and during the collection, the polarization direction of incident light is adjusted by a combination of a polarizer and an adjustable light source to suppress specular reflection of the metal surface, thereby obtaining an original multi-view image sequence with balanced illumination; The original multi-view image sequence is input into a time synchronization module, multi-view images at the same time are paired, and time median filtering is used to remove artifacts caused by jitter, thereby outputting multi-view candidate image frames with consistent time sequences; Geometric consistency correction is performed on the multi-view candidate image frames, a homography matrix is established based on the corner point detection results of the edge of the bone nail box, the multi-view images are mapped to a common reference plane, and distortion is eliminated through region overlap comparison, thereby obtaining multi-view standard images after geometric registration; Adaptive sharpness enhancement processing is performed on the multi-view standard images, a sharpness index is calculated using local frequency domain features, multi-scale sharpening and contrast compensation are performed on regions below a threshold, and a bone nail recognition image sequence with uniform sharpness distribution is generated.

3. The method of claim 1, wherein the method is characterized by: Based on the bone nail recognition image sequence, boundary information of the bone nail placement position is extracted, and a bone nail structure atlas representing the relative positional relationship of the bone nail placement region is generated, including: Multi-modal boundary information extraction is performed on the bone nail recognition image sequence, edge detection based on grayscale gradients and region segmentation based on convolution features are combined, and a boundary information set of candidate bone nail placement positions is obtained; Spatial constraint optimization is applied to the boundary information set of the candidate bone nail placement positions, each boundary information is matched with a preset bone nail box geometric template, and the geometry of the boundary information is corrected using least squares fitting, thereby outputting a boundary information set of the bone nail placement position after geometric correction; A topological consistency network is constructed based on the boundary information set of the bone nail placement position after geometric correction, the relative coordinate difference and row-column index position between each boundary information and adjacent boundary information are calculated, and a boundary information grid of the bone nail placement position with topological relationship is generated; In the boundary information grid of the bone nail placement position, each boundary information is labeled with a confidence index, the confidence is calculated based on the stability of boundary information extraction, the fitting error with the geometric template, and the residual error after topological consistency correction, and finally the bone nail structure atlas with confidence annotation is output.

4. The method of claim 1, wherein the method is characterized by: The recognition result is compared with a bone nail historical use behavior pattern, potential abnormal recognition items are identified, and an enhanced recognition result after behavior pattern correction is output, wherein the bone nail historical use behavior pattern is constructed based on historical bone nail use behavior data related to surgical type, doctor identifier, and bone nail specification, including: The recognition result is parsed one by one, bone nail category information, row-column index, spatial position, and recognition confidence contained in the recognition result are extracted, and corresponding recognition behavior features are generated; The recognition behavior features are matched with the historical bone nail use behavior data, corresponding historical behavior features are extracted under the conditions of surgical type, doctor identifier, and bone nail specification, and a historical behavior feature set is generated; The identified behavior characteristics are compared with the historical behavior characteristics one by one, the difference between the identified behavior characteristics and the historical behavior characteristics is calculated, and a potential abnormal identification item and a difference index thereof are output; The potential abnormal identification item and the difference index thereof are combined with the identification result under the adjacent row and column indexes, the typical distribution law recorded in the historical use behavior mode of the bone nail is combined, the potential abnormal identification item is corrected, and the corrected bone nail category information and the corrected identification confidence are output; The corrected bone nail category information and the corrected identification confidence are combined with the identification result without correction to generate an enhanced identification result after behavior mode correction, and the enhanced identification result includes bone nail category information, row and column indexes, spatial position and identification confidence.

5. The multi-feature cooperative inference based automatic identification and quantity confirmation method of bone pegs according to claim 1, characterized in that, The steps S1-S4 are performed before and after the operation starts respectively, the enhanced identification result before the use of the bone nail and the enhanced identification result after the use of the bone nail are obtained, and the difference information between the two is calculated through corresponding position comparison to obtain bone nail use difference information, including: The bone nail identification image sequence is collected before the operation starts, and steps S1-S4 are sequentially performed to generate an enhanced identification result before the use of the bone nail, which includes bone nail category information, row and column indexes, spatial position and identification confidence; The bone nail identification image sequence is collected again after the operation ends, and steps S1-S4 are sequentially performed to generate an enhanced identification result after the use of the bone nail, which includes bone nail category information, row and column indexes, spatial position and identification confidence; The enhanced identification result before the use of the bone nail and the enhanced identification result after the use of the bone nail are compared under the same row and column indexes and spatial position to generate position matching pairs one by one, and a corresponding result set containing category information before and after matching and identification confidence before and after matching is output; Difference information is calculated based on the corresponding result set, including whether the bone nail category information changes, the identification confidence difference and the spatial position offset, and a difference record is generated for each row and column index; All difference records are statistically summarized, and bone nail use difference information is output, which includes the number of nails, category change distribution, spatial position abnormal distribution and confidence abnormal distribution, which are used for subsequent generation of bone nail use confirmation information.

6. The multi-feature cooperative inference based automatic identification and quantity confirmation method of bone pegs according to claim 1, characterized in that, Based on the bone nail use difference information, the operation identification information, the identification confidence and the image collection time are combined to generate bone nail use confirmation information, and the bone nail use confirmation information is written into a non-tamperable storage medium to realize closed-loop tracing of bone nail use records, including: The bone nail use difference information is bound with the operation identification information to generate a difference information set with a unique operation identification, and the operation identification information includes operation number, doctor identity information and timestamp; In the difference information set, the difference record is read one by one, the bone nail category information, row and column indexes, spatial position and identification confidence in each difference record are combined with the image collection time to generate a difference expansion record with a time sequence mark; In the difference information set, the difference record is read one by one, the bone nail category information, row and column indexes, spatial position and identification confidence in each difference record are combined with the image collection time to generate a difference expansion record with a time sequence mark; The difference expansion record is subjected to consistency check, if the recognition confidence is lower than the set threshold, it is marked as low confidence record and attached with confidence correction label, if the recognition confidence is greater than or equal to the set threshold, the original confidence is kept, and the checked difference expansion record is output; The checked difference expansion record is sorted according to the row-column index sequence and the image acquisition time sequence, and the cumulative nail quantity change, category change trajectory and spatial position offset trajectory are calculated in the sorting result, and the difference summary record containing the trajectory information is output; The difference summary record is integrated with the operation identification information to generate the bone nail use confirmation information, which includes operation number, doctor identity information, nail quantity change, category change trajectory, spatial position offset trajectory and recognition confidence distribution, and the bone nail use confirmation information is written into a storage medium with tamper-proof storage mechanism to ensure the uniqueness, integrity and traceability of the information record.

7. The method of claim 6, wherein the method further comprises: determining a number of the bone pins in the image; and determining a number of the bone pins in the surgical plan. The difference information set is read in the difference record, the bone nail category information, row-column index, spatial position and recognition confidence in each difference record are combined with the image acquisition time to generate the difference expansion record with time sequence mark, including: The difference record in the difference information set is parsed in sequence, the bone nail category information, row-column index, spatial position and recognition confidence are extracted, and a unique difference record number is attached to each difference record, and the difference record set containing the bone nail category information, row-column index, spatial position, recognition confidence and difference record number is output; The difference record set is combined with the image acquisition time, and alignment is realized by accurate time stamp matching and missing time interpolation in the combination process, if a difference record lacks corresponding image acquisition time, linear interpolation of adjacent image acquisition time is adopted to generate a complete time stamp, and the difference time record set containing the image acquisition time is output; The spatial position in the difference time record set is jointly modeled with the image acquisition time to form a continuous position sequence of spatial position changing with time, and the trend value of spatial position offset is calculated in the position sequence, and the difference position record set containing the position trend is output; The recognition confidence in the difference position record set is jointly calculated with the image acquisition time to generate a confidence curve of recognition confidence changing with time, and abnormal fluctuation sections are detected in the confidence curve, a confidence fluctuation label is attached to each abnormal fluctuation section, and the difference confidence record set containing the confidence fluctuation label is output; the difference confidence record set is sorted according to the row-column index sequence and the image acquisition time sequence, and the bone nail category information, row-column index, spatial position, recognition confidence, image acquisition time, difference record number and confidence fluctuation label are uniformly integrated in the sorting result to generate the final difference expansion record.

Images