Bone peg positioning and robotic control method, system, electronic device, and storage medium

Abstract

The present application relates to bone nail positioning and robot control method, system, electronic equipment and storage medium, relates to the field of information technology, including the following steps: obtaining 3D point cloud data containing bone nail through 3D camera; processing 3D point cloud data, identifying the three-dimensional space coordinates of each bone nail, and calculating the XYZ space coordinates of each group of bone nails with 6 to 9 bone nails as a group; the XYZ space coordinates of each group of bone nails are transmitted to the robot control device; the robot collaborative arm continuously moves according to the XYZ space coordinates, the industrial camera on the robot collaborative arm shoots 6 to 9 bone nails at a time, and all bone nail groups in the whole nail box are shot in turn through continuous movement; image recognition and OCR recognition are carried out on the image obtained by the industrial camera, and the digital information on each bone nail and the total number of bone nails are extracted. The present application solves the problems of low efficiency and easy error of traditional manual operation through automatic detection and operation process, and significantly improves the automation level, accuracy and efficiency of orthopedic surgery preparation.

Description

Technical Field

[0001] This invention relates to the field of information technology, and in particular to bone screw positioning and robot control methods, systems, electronic devices and storage media. Background Technology

[0002] In the field of modern medical technology, the precision and efficiency of orthopedic surgery are crucial to patient recovery. With the increasing complexity of surgery, the management and operation of orthopedic instruments have become key aspects of surgical success.

[0003] A common problem in the management of orthopedic devices is the lack of dynamic integration capabilities for the spatial location and labeling information of devices. Bone screws, as small objects, are highly susceptible to obstruction or viewing angle limitations, leading to missing or inaccurate data.

[0004] Therefore, how to achieve high-precision capture of the three-dimensional spatial position of bone nails in complex environments, and on this basis, accurately extract the identification information, has become a key issue in improving the efficiency and safety of orthopedic surgery. Summary of the Invention

[0005] This invention aims to provide a method, system, electronic device, and storage medium for bone screw positioning and robot control. Through the deep integration of 3D point cloud data and image recognition technology, it achieves high-precision identification of the spatial coordinates of bone screws within an orthopedic screw box and automated extraction of identification information. By automating the detection and operation process, it solves the problems of low efficiency and error-proneness in traditional manual operations, significantly improving the automation level, accuracy, and efficiency of orthopedic surgical preparation.

[0006] To achieve the above objectives, the present invention provides a method for bone screw positioning and robot control, comprising the following steps:

[0007] Acquire 3D point cloud data containing bone screws using a 3D camera;

[0008] Process 3D point cloud data, identify the three-dimensional spatial coordinates of each bone screw, and calculate the XYZ spatial coordinates of each group of 6 to 9 bone screws.

[0009] The XYZ spatial coordinates of each set of bone screws are transmitted to the robot control device;

[0010] The robotic collaborative arm moves continuously according to the XYZ spatial coordinates. The industrial camera on the robotic collaborative arm captures 6 to 9 bone screws at a time, and through continuous movement, it captures all bone screw groups in the entire screw box in sequence.

[0011] Image recognition and OCR are performed on images acquired by industrial cameras to extract digital information from each bone screw and the total number of bone screws.

[0012] Preferably, the step of acquiring 3D point cloud data containing bone screws using a 3D camera includes:

[0013] Orthopedic nail boxes were illuminated with light sources of different intensities and angles, and point cloud datasets were collected.

[0014] Based on the reflective properties of the metal material of bone nails, the predicted reflective intensity of each candidate three-dimensional coordinate point under different light source conditions is calculated using a metal bidirectional reflectance distribution function model. The fusion weight is assigned according to the predicted reflective intensity, and the lower the predicted reflective intensity, the higher the weight.

[0015] The point cloud dataset is weighted and fused according to the fusion weights to generate the final 3D point cloud data.

[0016] Using the nominal diameter of the bone nail as a typical physical dimension, the preset multiple range of physical length is dynamically calculated to filter out outlier point clusters that fall outside the range.

[0017] Preferably, the steps for processing 3D point cloud data to identify the three-dimensional spatial coordinates of each bone screw include:

[0018] Denoising and downsampling are performed on 3D cloud data;

[0019] The processed point cloud data is segmented based on the Euclidean distance clustering algorithm to obtain multiple independent bone nail point cloud clusters, each point cloud cluster corresponding to one bone nail;

[0020] Each bone screw point cloud cluster is iteratively matched with the nearest point in the pre-stored bone screw 3D CAD model library, and the bone screw type and spatial location are determined based on the matching error.

[0021] For each successfully identified bone screw point cloud cluster, calculate its geometric centroid three-dimensional coordinates as the spatial coordinates of the bone screw, or for special structural bone screws, use a random sampling consensus algorithm to extract the three-dimensional coordinates of the structural feature center point as the spatial coordinates.

[0022] Preferably, the steps for the robotic collaborative arm to perform continuous motion according to XYZ spatial coordinates include:

[0023] The shooting path is planned based on the XYZ spatial coordinates, and the shooting position sequence and dwell time of the industrial camera in each bone screw group are determined.

[0024] After acquiring initial image data with each shot, the image quality is used to determine whether there are blurry areas or shooting defects;

[0025] If a defect exists, a reshoot command is generated and the robot's collaborative arm trajectory is adjusted to the defect location for a second shot. The supplementary image data is then fused with the existing image data.

[0026] Preferably, the steps for performing image recognition and OCR recognition on images acquired by industrial cameras include:

[0027] The original image was converted to grayscale and denoised to obtain a clear image of the bone screw;

[0028] Edge detection was used to extract the contour features of the bone nails and segment the independent image region of each bone nail;

[0029] Perform OCR recognition on independent image regions, extract digital information and store it as structured data;

[0030] A counting tool is used to count the total number of bone nail image regions, and the digital content and total number information are integrated to generate the final recognition result.

[0031] The technical solution of this invention provides a bone screw positioning and robot control system, applied to an orthopedic screw box for setting multiple bone screws, comprising:

[0032] A 3D camera is used to acquire 3D point cloud data of the bone screws;

[0033] The processing unit is configured to process 3D point cloud data to identify the three-dimensional spatial coordinates of each bone screw, calculate the XYZ spatial coordinates of each group of bone screws, and transmit the coordinate data to the robot control device.

[0034] The robot control device includes a robot collaborative arm and an industrial camera mounted on the collaborative arm. The robot collaborative arm moves continuously according to the received XYZ spatial coordinates, and the industrial camera acquires images of the corresponding bone screw group at each shooting position.

[0035] The recognition unit is configured to perform image recognition and OCR recognition on images acquired by industrial cameras, extract digital information from each bone screw, and count the total number of bone screws.

[0036] Preferably, the processing unit calculates the XYZ spatial coordinates of each group of 6 to 9 bone screws.

[0037] Preferably, an industrial camera acquires digital information and counts the total number of bone screws until it covers all bone screw groups in the entire screw box.

[0038] The technical solution of the present invention provides an electronic device for bone screw positioning and robot control, comprising:

[0039] One or more processors;

[0040] A memory or storage medium used to store applications and data;

[0041] Instructions are stored in the memory or storage medium, and the processor invokes the instructions to cause the electronic device to perform the various steps of the above method.

[0042] The technical solution of the present invention provides a bone screw positioning and robot control storage medium, which stores instructions. When the instructions are executed by a processor or run on a computer, the various steps of the method described above are implemented. The storage medium includes a non-volatile or volatile computer-readable storage medium.

[0043] Compared with the prior art, the present invention has the following beneficial technical effects:

[0044] This invention discloses an automated detection and operation method for orthopedic screw boxes based on three-dimensional point cloud data and image recognition. It proposes an integrated solution for complex business scenarios such as the identification of bone screw positions, the counting of numbers and the extraction of information in screw boxes during orthopedic surgery.

[0045] This invention acquires point cloud data of bone screws within a screw box using a 3D camera, accurately identifies the spatial coordinates of each screw, calculates the coordinate data in groups, and transmits it to the robot control device to achieve precise and continuous movement of the collaborative arm. Simultaneously, the collaborative arm is equipped with an industrial camera, which, through group shooting and image recognition technology combined with optical character recognition, efficiently extracts digital information and the total number of screws.

[0046] This invention seamlessly integrates three-dimensional positioning and image information extraction, solving the problems of low efficiency and error-proneness in traditional manual operations. It significantly improves the automation level and accuracy of orthopedic surgical preparation, and ultimately achieves intelligent bone screw management and high efficiency in surgical support. Attached Figure Description

[0047] Figure 1 This is a flowchart of the bone nail positioning and robot control method, system, electronic device and storage medium of the present invention. Detailed Implementation

[0048] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0049] Example 1:

[0050] This invention discloses a method for bone screw positioning and robot control. This method, through the deep fusion of three-dimensional point cloud data and image recognition technology, achieves a complete automated process for bone screw detection and operation within an orthopedic screw box. Specifically, the method includes the following steps:

[0051] Step S101: Take a picture of the orthopedic nail box with a 3D camera to obtain 3D point cloud data containing the bone nails.

[0052] The orthopedic nail box was illuminated with light sources of different intensities and angles, and point cloud datasets were collected.

[0053] Based on the known reflective properties of the metal material of bone nails, weighted fusion is performed on the initial point cloud data acquired under different light source intensities, thereby effectively overcoming the data loss problem caused by metal reflection under single light source conditions;

[0054] After point cloud fusion, the nominal diameter of the bone nail is used as a typical physical size, and the preset multiple range of physical length is dynamically calculated based on the nominal diameter. Point cloud clusters falling within this range are identified as valid points, while isolated point cloud clusters outside this range are identified as outliers and filtered out, thereby ensuring data quality.

[0055] Based on the known reflective properties of the metal material of bone nails, weighted fusion is performed on initial point cloud data acquired under different light source intensities, specifically including:

[0056] For multiple candidate 3D coordinate points formed by the same spatial point under different light source intensities, calculate the angle between the normal vector of the local surface where each candidate 3D coordinate point is located and the direction of the incident light.

[0057] Based on the included angle and a preset bidirectional metal reflection distribution function model, a predicted value of reflectance intensity is calculated for each candidate three-dimensional coordinate point;

[0058] Each candidate 3D coordinate point is assigned a fusion weight based on the predicted reflectance intensity value. The lower the predicted reflectance intensity value, the higher the weight of the corresponding candidate 3D coordinate point in the weighted fusion.

[0059] Based on the fusion weight, all candidate 3D coordinate points are weighted and calculated to generate the final 3D coordinates of the spatial point.

[0060] Based on the included angle and a preset bidirectional metallic reflection distribution function model, a predicted reflection intensity value is calculated for each candidate 3D coordinate point, specifically including:

[0061] The included angle, as well as the preset basic reflectivity parameters and surface roughness parameters characterizing the optical properties of the bone nail metal surface, are substituted into the metal bidirectional reflectance distribution function model for calculation.

[0062] Among them, the metal bidirectional reflection distribution function model is a model based on micro-surface theory, and its function value is jointly determined by the Fresnel term, the normal distribution function term, and the geometric shading term.

[0063] The formula for the bidirectional reflection distribution function fr of metal is:

[0064]

[0065]

[0066]

[0067]

[0068]

[0069]

[0070] Where, ω i Indicates the direction of the incident light, ω r Let F represent the direction of reflected light, D represent the normal distribution function, G represent the geometric occlusion function, n represent the surface normal vector, v represent the viewing direction vector, l represent the incident light direction vector, h represent the half-path vector, and θ represent the direction of reflected light. h Let F0 be the angle between the half-range vector h and the macroscopic surface normal vector n; let F0 represent the base reflectivity; let α represent the surface roughness parameter, ranging from 0 to 1, with larger values ​​indicating rougher surfaces; let k represent the parameter calculated based on the surface roughness parameter α; and let G be the surface roughness parameter. schlickGGX Let θ represent the GGX geometric function of the Schlick approximation, and let cosθ represent the cosine of the angle between the observation direction v or the incident light direction l and the surface normal vector n.

[0071] This step involves using a 3D camera to photograph the orthopedic nail box to obtain 3D point cloud data containing the bone nails, laying the data foundation for subsequent spatial positioning.

[0072] Step S102: Process the 3D point cloud data, identify the three-dimensional spatial coordinates of each bone screw, and calculate the XYZ spatial coordinates of each group of 6 to 9 bone screws.

[0073] After obtaining the raw point cloud data, preprocessing is performed, specifically including:

[0074] Noise Reduction: A statistical outlier removal algorithm is used to filter out discrete noise points caused by ambient stray light or metallic reflection.

[0075] Downsampling: Voxel grid filters are used to uniformly downsample the point cloud, which reduces the amount of data and improves the speed of subsequent processing while maintaining the shape characteristics of the point cloud.

[0076] Individual bone screw point cloud segmentation and identification: This step aims to separate the point cloud clusters corresponding to each individual bone screw from the preprocessed overall point cloud and identify their type. Specifically, this includes:

[0077] Euclidean distance-based clustering segmentation: A Euclidean clustering algorithm (e.g., KD-tree accelerated nearest neighbor search) is used to segment the point cloud. The algorithm sets a distance threshold (e.g., 5 mm), grouping points in space with a distance less than this threshold into the same cluster. Due to the physical gaps between the screws, this algorithm effectively separates point clouds belonging to different screws, resulting in multiple independent point cloud clusters, each initially corresponding to one screw.

[0078] Bone screw model matching and recognition: Each segmented point cloud cluster is iteratively matched with a pre-stored library of 3D CAD models of bone screws. The transformation matrix and matching error between the point cloud cluster and each CAD model are calculated using the ICP algorithm. The CAD model with the smallest matching error is identified as the type of bone screw. Simultaneously, this process also preliminarily determines the approximate location of each bone screw in space.

[0079] The extraction of 3D coordinates of key points in bone screws involves extracting the coordinates of a feature point that represents the spatial location of each successfully identified bone screw point cloud. Specifically, this includes:

[0080] Centroid Calculation Method: Calculate the geometric center (centroid) of all points in the point cloud cluster of the bone screw, and use its three-dimensional coordinates as the spatial coordinates of the bone screw. This method is simple to calculate and is suitable for bone screws with symmetrical or regular shapes.

[0081] For bone screws with special structures, such as those with a central groove at the tail, feature extraction algorithms such as random sampling consensus are used to accurately locate the three-dimensional coordinates of the center point at the bottom of the groove as the coordinates of the bone screw.

[0082] Finally, the XYZ spatial coordinates of each group of 6 to 9 bone screws were calculated to provide grouped data for subsequent robot motion control.

[0083] Step S103: Transmit the XYZ spatial coordinates of each set of bone screws to the robot control device.

[0084] Using a pre-established coordinate transformation tool, the XYZ spatial coordinate data of each set of bone nails is converted from the original format to the standardized format required for robot control. The integrity of the data is checked during the conversion. If data is missing or abnormal, supplementary information is obtained from a backup data source to obtain a complete set of coordinates.

[0085] Based on the complete coordinate set, a data verification tool is used to check the accuracy of each set of XYZ spatial coordinates. If the deviation found during the verification exceeds the preset threshold, the coordinate data is adjusted by a calibration tool to finally determine the coordinate result that meets the accuracy requirements.

[0086] Once the coordinate results are obtained, the coordinate data is transmitted to the robot control module in batches using the data transmission interface. During the transmission, the transmission status is monitored using a log recording tool to determine whether the transmission is complete.

[0087] For the coordinate data that has been transmitted, a real-time comparison tool is used to verify the consistency of the data. If the consistency verification passes, the data is stored in the cache area of ​​the control module for subsequent operation instruction generation, ensuring that the coordinate data received by the robot device is accurate.

[0088] Step S104: The robot collaborative arm moves continuously according to the XYZ spatial coordinates. The robot collaborative arm is equipped with an industrial camera to capture images of 6 to 9 bone screws at a time, and captures images of all bone screw groups in the entire screw box in sequence through continuous movement.

[0089] Based on the movement trajectory of the robot collaborative arm, the coordinate information of the industrial camera at each shooting position is obtained. The coordinate information is sorted by a preset path planning tool to determine the order and dwell time of the shooting positions, and a shooting path covering the entire nail box is obtained.

[0090] For the shooting path, an image acquisition tool is used to control the industrial camera to stop at each position and acquire initial image data of at least one set of bone nails. The initial image data is then adjusted by an image preprocessing tool to obtain an optimized image set.

[0091] If there are blurry areas in the image set, the image comparison tool is used to compare them with the preset standard bone nail image to determine whether there are shooting defects, obtain the coordinate information of the defect location, and generate a reshoot instruction.

[0092] According to the reshooting instruction, the movement trajectory of the robot collaborative arm is adjusted and repositioned to the defect location. The designated bone nail group is then photographed a second time using an industrial camera to obtain supplementary image data. The fusion result of the supplementary image data and the existing image set is determined to ensure the integrity and clarity of the image acquisition.

[0093] Step S105: Using images acquired by an industrial camera, perform image recognition and OCR recognition to extract digital information from each bone screw and the total number of bone screws.

[0094] An industrial camera is used to capture images of the bone screw area, obtaining raw image data containing the bone screw. Image preprocessing tools are then used to perform grayscale conversion and noise reduction on the raw image data to obtain a clear image of the bone screw.

[0095] For clear bone nail images, edge detection tools are used to extract the contour features of the bone nails, and the independent regions of each bone nail are determined from the contour features. The independent regions are then separated by a region segmentation tool to obtain multiple bone nail image regions.

[0096] If the number of bone nail image regions exceeds a preset threshold, then the digital information of the bone nail image regions is extracted using an optical character recognition tool to obtain the digital content on each bone nail and store the digital content as structured data.

[0097] Based on structured data, a counting tool is used to count the total number of bone screws in the image region, obtaining the total number of bone screws. The numerical content and total number information are then integrated as the final recognition result, thereby realizing the automated extraction and statistics of bone screw identification information.

[0098] Example 2:

[0099] This embodiment discloses a bone screw positioning and robot control system, which can implement the method of Embodiment 1. The system is applied to an orthopedic screw box containing multiple bone screws, and includes:

[0100] A 3D camera is used to acquire 3D point cloud data containing bone screws;

[0101] The robot control device includes a robot collaborative arm, on which an industrial camera is mounted;

[0102] The processing unit is configured to process 3D point cloud data, identify the three-dimensional spatial coordinates of each bone screw, and calculate the XYZ spatial coordinates of each group of 6 to 9 bone screws; and transmit the XYZ spatial coordinates of each group of bone screws to the robot control device.

[0103] The robotic collaborative arm moves continuously according to the XYZ spatial coordinates, and the industrial camera is used to capture 6 to 9 bone screws at a time, and through continuous movement, it captures all bone screw groups in the entire screw box in sequence.

[0104] The recognition unit uses images acquired by an industrial camera to perform image recognition and OCR recognition, extracting digital information from each bone screw and the total number of bone screws.

[0105] The various units work together to automate the bone screw detection and operation process.

[0106] Example 3:

[0107] This embodiment discloses an electronic device for bone screw positioning and robot control. This electronic device can vary significantly depending on its configuration or performance, and may include one or more central processing units (CPUs) and one or more memories or storage media (e.g., one or more mass storage devices) for storing applications or data. The memories and storage media can be short-term or long-term storage.

[0108] Instructions are stored in the memory; at least one processor invokes the instructions in the memory to cause the electronic device to perform the steps of the method of Embodiment 1.

[0109] The program stored in the storage medium may include one or more modules, each module may include a series of instruction operations on the electronic device. Furthermore, the processor may be configured to communicate with the storage medium and execute the series of instruction operations stored in the storage medium on the electronic device.

[0110] Electronic devices may also include one or more power supplies, one or more wired or wireless network interfaces, one or more input / output interfaces, and / or one or more operating systems, such as Windows Server, MacOSX, Unix, Linux, FreeBSD, etc.

[0111] Those skilled in the art will understand that the electronic device structure in this embodiment does not constitute a limitation on electronic devices, and may include more or fewer components, or combine certain components, or have different component arrangements.

[0112] Example 4:

[0113] The present invention also provides a bone screw positioning and robot control storage medium, which stores instructions that, when executed by a processor or run on a computer, implement the various steps of the method of Embodiment 1. The storage medium can be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium.

[0114] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the system, device, or unit described above can be referred to the corresponding process in the method of Embodiment 1, and will not be repeated here.

[0115] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium.

[0116] Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method of Embodiment 1 of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a portable hard drive, read-only memory (ROM), random access memory (RAM), a magnetic disk, or an optical disk.

[0117] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for bone screw positioning and robot control, characterized in that, Includes the following steps: Acquire 3D point cloud data containing bone screws using a 3D camera; Process 3D point cloud data, identify the three-dimensional spatial coordinates of each bone screw, and calculate the XYZ spatial coordinates of each group of 6 to 9 bone screws. The XYZ spatial coordinates of each set of bone screws are transmitted to the robot control device; The robotic collaborative arm moves continuously according to the XYZ spatial coordinates. The industrial camera on the robotic collaborative arm captures 6 to 9 bone screws at a time, and through continuous movement, it captures all bone screw groups in the entire screw box in sequence. Image recognition and OCR recognition are performed on images acquired by industrial cameras to extract digital information on each bone screw and the total number of bone screws. The steps involved in acquiring 3D point cloud data containing bone screws using a 3D camera include: Orthopedic nail boxes were illuminated with light sources of different intensities and angles, and point cloud datasets were collected. Based on the reflective properties of the metal material of bone nails, the predicted reflective intensity of each candidate three-dimensional coordinate point under different light source conditions is calculated using a metal bidirectional reflectance distribution function model. The fusion weight is assigned according to the predicted reflective intensity, and the lower the predicted reflective intensity, the higher the weight. The point cloud dataset is weighted and fused according to the fusion weights to generate the final 3D point cloud data. Using the nominal diameter of the bone screw as a typical physical size, outlier point clusters that fall outside a preset multiple of the nominal diameter of the bone screw are filtered out.

2. The bone screw positioning and robot control method according to claim 1, characterized in that, The steps for processing 3D point cloud data to identify the three-dimensional spatial coordinates of each bone screw include: Denoising and downsampling are performed on 3D cloud data; The processed point cloud data is segmented based on the Euclidean distance clustering algorithm to obtain multiple independent bone nail point cloud clusters, each point cloud cluster corresponding to one bone nail; Each bone screw point cloud cluster is iteratively matched with the nearest point in the pre-stored bone screw 3D CAD model library, and the bone screw type and spatial location are determined based on the matching error. For each successfully identified bone screw point cloud cluster, calculate its geometric centroid three-dimensional coordinates as the spatial coordinates of the bone screw, or for bone screws with special structures, use a random sampling consensus algorithm to extract the three-dimensional coordinates of the structural feature center point as the spatial coordinates of the bone screw.

3. The bone screw positioning and robot control method according to claim 1, characterized in that, The steps for a robotic collaborative arm to perform continuous motion based on XYZ spatial coordinates include: The shooting path is planned based on the XYZ spatial coordinates, and the shooting position sequence and dwell time of the industrial camera in each bone screw group are determined. After acquiring initial image data with each shot, the image quality is used to determine whether there are blurry areas or shooting defects; If a defect exists, a reshoot command is generated and the robot's collaborative arm trajectory is adjusted to the defect location for a second shot. The supplementary image data is then fused with the existing image data.

4. The bone screw positioning and robot control method according to claim 1, characterized in that, The steps for image recognition and OCR recognition of images acquired by industrial cameras include: The original image was converted to grayscale and denoised to obtain a clear image of the bone screw; Edge detection was used to extract the contour features of the bone nails and segment the independent image region of each bone nail; Perform OCR recognition on independent image regions, extract digital information and store it as structured data; A counting tool is used to count the total number of bone nail image regions, and the numerical information and total number information are integrated to generate the final recognition result.

5. A bone screw positioning and robot control system, characterized in that, Orthopedic screw holders for setting multiple bone screws include: A 3D camera is used to acquire 3D point cloud data of the bone screws; the 3D camera is configured to illuminate the orthopedic screw box with light sources of different intensities and angles and collect point cloud datasets. The processing unit is configured to process 3D point cloud data to identify the three-dimensional spatial coordinates of each bone screw, calculate the XYZ spatial coordinates of each group of bone screws, and transmit the coordinate data to the robot control device. The processing unit is also configured to: calculate the predicted reflectance intensity of each candidate three-dimensional coordinate point under different light source conditions based on the reflective properties of the metal material of the bone screw, using a metal bidirectional reflectance distribution function model, and assign fusion weights according to the predicted reflectance intensity values, with higher weights for lower predicted reflectance intensity values; perform weighted fusion of the point cloud dataset according to the fusion weights to generate the final 3D point cloud data; and filter out outlier point cloud clusters that fall outside a preset multiple of the nominal diameter of the bone screw, using the nominal diameter of the bone screw as a typical physical size. The robot control device includes a robot collaborative arm and an industrial camera mounted on the collaborative arm. The robot collaborative arm moves continuously according to the received XYZ spatial coordinates, and the industrial camera acquires images of the corresponding bone screw group at each shooting position. The recognition unit is configured to perform image recognition and OCR recognition on images acquired by industrial cameras, extract digital information from each bone screw, and count the total number of bone screws.

6. The bone screw positioning and robot control system according to claim 5, characterized in that, The processing unit calculates the XYZ spatial coordinates of each group of 6 to 9 bone screws.

7. The bone screw positioning and robot control system according to claim 5, characterized in that, Industrial cameras acquire digital information and count the total number of bone screws until all bone screw groups in the entire screw box are covered.

8. An electronic device for bone screw positioning and robot control, characterized in that, include: One or more processors; A memory or storage medium used to store applications and data; Instructions are stored in the memory or storage medium, and the processor invokes the instructions to cause the electronic device to perform the steps of the method as described in any one of claims 1 to 4.

9. A bone screw positioning and robot control storage medium, characterized in that, The storage medium contains instructions that, when executed by a processor or run on a computer, implement the steps of the method as described in any one of claims 1 to 4, and the storage medium includes a non-volatile or volatile computer-readable storage medium.

Images