Bone nail classification method and device, computer equipment, readable storage medium and program product

Abstract

The invention relates to a bone nail classification method and device, computer equipment, a computer readable storage medium and a computer program product. The method comprises the steps of obtaining a to-be-classified bone nail image; processing the to-be-classified bone nail image through the bone nail recognition model to obtain a bone nail type of the to-be-classified bone nail and a region of interest, the region of interest being an image including the to-be-classified bone nail; performing edge extraction on the region of interest through an edge recognition network to obtain edge information of the to-be-classified bone nails, and determining size information of the to-be-classified bone nails by using the edge information; and a control instruction is generated according to the bone nail type of the to-be-classified bone nail and the size information of the to-be-classified bone nail, the control instruction is transmitted to the mechanical arm, and the control instruction is used for instructing the mechanical arm to execute the classification action of the to-be-classified bone nail according to the bone nail type of the to-be-classified bone nail and the size information of the to-be-classified bone nail. By adopting the method, the bone nail classification efficiency can be improved.

Description

Technical Field

[0001] This application relates to the field of medical device automation technology, and particularly to a bone nail classification method, device, computer device, computer-readable storage medium, and computer program product. Background Art

[0002] Bone nails are non-active surgical metal implants for bone fixation and are used in multiple fields, such as orthodontics, orthopedic surgeries, etc. And bone nails have multiple models to facilitate use in multiple scenarios.

[0003] Before using bone nails, it is necessary to extract and sort the bone nails according to the usage scenario for use.

[0004] In related technologies, the classification and assembly of bone nails mainly rely on manual operations. However, this method is not only inefficient but also prone to errors. Summary of the Invention

[0005] Based on this, it is necessary to provide a bone nail classification method, device, computer device, computer-readable storage medium, and computer program product that can improve efficiency for the above technical problems.

[0006] In a first aspect, this application provides a bone nail classification method, and the method includes:

[0007] Obtain an image of the bone nail to be classified;

[0008] Process the image of the bone nail to be classified through a bone nail recognition model to obtain the bone nail type and the region of interest of the bone nail to be classified, where the region of interest is an image including the bone nail to be classified;

[0009] Extract the edge of the region of interest through an edge recognition network to obtain the edge information of the bone nail to be separated, and use the edge information to determine the size information of the bone nail to be classified;

[0010] Generate a control instruction according to the bone nail type and the size information of the bone nail to be classified, and transmit the control instruction to the robotic arm. The control instruction is used to instruct the robotic arm to perform the classification action of the bone nail to be classified according to the bone nail type and the size information of the bone nail to be classified.

[0011] In one embodiment, generating a control instruction according to the type of bone nail to be classified and the size information of the bone nail to be classified includes: obtaining a target bone nail type and a target quantity corresponding to the target bone nail type; determining target bone nails from the bone nails to be classified according to the target bone nail type and the type of the bone nails to be classified, and determining the size information of the target bone nails; generating a control instruction according to the target bone nails, the target quantity, and the size information of the target bone nails, and sending the control instruction to a robotic arm. The control instruction is used to instruct the robotic arm to pick up the target quantity of target bone nails and place the picked-up target bone nails into a target bone nail box corresponding to the size information of the bone nails to be classified.

[0012] In one of the embodiments, generating a control instruction according to the target bone nails, the target quantity, and the size information of the target bone nails includes: determining the coordinate information of the target bone nails in the global coordinate system according to the size information of the target bone nails, and generating a picking coordinate according to the coordinate information; determining a target picking device according to the type of the target bone nails and the size information of the target bone nails; generating a control instruction according to the target bone nails, the target quantity, the size information of the target bone nails, the picking coordinate, and the target picking device. The control instruction is used to instruct the robotic arm to use the target picking device to pick up the target quantity of target bone nails according to the picking coordinate and place the picked-up target bone nails into the target bone nail box according to the size information of the target bone nails.

[0013] In an alternative embodiment, generating a control instruction according to the target bone nails, the target quantity, the size information of the target bone nails, the picking coordinate, and the target picking device includes: obtaining the vacant area of the bone nail box, and obtaining the type of bone nails that can be stored in the vacant area and the size information of the vacant area corresponding thereto; selecting a to-be-determined area from the vacant area of the bone nail box according to the type of bone nails that can be stored and the type of the bone nails that can be stored; screening out a target area from the to-be-determined area according to the size information of the control area and the size information of the target bone nails; generating a control instruction according to the picking coordinate, the target picking device, the target bone nails, the target quantity, and the target area. The control instruction is used to instruct the robotic arm to use the target picking device to pick up the target quantity of target bone nails and place the target bone nails into the target area.

[0014] In one embodiment, using an edge recognition network to recognize a region of interest to obtain the edge information of the bone nails to be classified includes: using the edge recognition network to determine the image gradient map and the gradient direction map of the region of interest; determining the target edge information of the bone nails to be separated according to the image gradient map and the gradient direction map; determining the size information of the bone nails to be separated according to the target edge information.

[0015] In one embodiment, the size information of the bone nail to be separated is determined using edge information, including: obtaining the coordinate information of each pixel corresponding to the edge information, fitting the coordinate information of each pixel using a line fitting algorithm to obtain a fitted line; and determining the size information of the bone nail to be separated according to the fitted line.

[0016] In a second aspect, the present application further provides a bone nail classification device, including:

[0017] An image acquisition module for acquiring an image of the bone nail to be classified;

[0018] An identification module for processing the image of the bone nail to be classified through a bone nail identification model to obtain the bone nail type and the region of interest of the bone nail to be classified, where the region of interest is an image including the bone nail to be classified;

[0019] A size information determination module for extracting the edge information of the bone nail to be separated from the region of interest through an edge recognition network, and determining the size information of the bone nail to be classified using the edge information;

[0020] A control instruction generation module for generating a control instruction according to the bone nail type and the size information of the bone nail to be classified, and transmitting the control instruction to the robotic arm, where the control instruction is used to instruct the robotic arm to perform the classification operation of the bone nail to be classified according to the bone nail type and the size information of the bone nail to be classified.

[0021] In a third aspect, the present application further provides a computer device, including a memory and a processor, where the memory stores a computer program, and when the processor executes the computer program, the steps of the above method embodiments are implemented.

[0022] In a fourth aspect, the present application further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the above method embodiments are implemented.

[0023] In a fifth aspect, the present application further provides a computer program product, including a computer program, and when the computer program is executed by a processor, the steps of the above method embodiments are implemented.

[0024] The above bone nail classification method, device, computer device, computer-readable storage medium, and computer program product use a bone nail recognition model to identify an image of a bone nail to be classified, obtain the region of interest and the fixation type of the bone nail to be classified, use an edge recognition network to identify the edge of the region of interest, determine the size information of the bone nail to be classified according to the recognized edge, and can generate a control instruction according to the bone nail type and size information of the bone nail to be classified. The robotic arm can execute a classification action on the bone nail to be classified according to the control instruction to achieve the classification of the bone nail to be classified. The above steps utilize an automated process of recognition, picking, and classification, effectively improving the bone nail classification efficiency and the accuracy of bone nail classification. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] To more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following briefly introduces the drawings required for use in the description of the embodiments of the present application or related technologies. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other related drawings can be obtained based on these drawings without creative efforts.

[0026] Figure 1 It is an application environment diagram of the bone nail classification method in an embodiment;

[0027] Figure 2 It is a flowchart of the bone nail classification method in an embodiment;

[0028] Figure 3 It is a flowchart of the bone nail classification method in another embodiment;

[0029] Figure 4 It is a structural block diagram of the bone nail classification device in an embodiment;

[0030] Figure 5 It is an internal structure diagram of a computer device in an embodiment. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0031] To make the objectives, technical solutions, and advantages of the present application clearer, the following further elaborates on the present application in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0032] The bone nail classification method provided by the embodiments of the present application can be applied to, for example Figure 1In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or can be placed on the cloud or other network servers. The server 104 obtains the bone nail image to be classified of the terminal 102; processes the bone nail image to be classified through the bone nail recognition model to obtain the bone nail type and the region of interest of the bone nail to be classified, and the region of interest is an image including the bone nail to be separated; performs edge extraction on the region of interest through the edge recognition network to obtain the edge information of the bone nail to be separated, and determines the size information of the bone nail to be classified by using the edge information; generates a control instruction according to the bone nail type of the bone nail to be classified and the size information of the bone nail to be classified, and transmits the control instruction to the robotic arm. The control instruction is used to instruct the robotic arm to perform the classification action of the bone nail to be classified according to the bone nail type of the bone nail to be classified and the size information of the bone nail to be classified. Among them, the terminal 102 can be but is not limited to various personal computers, laptop computers, smart phones, tablet computers, Internet of Things devices and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart in-vehicle devices, projection devices, etc. The portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The head-mounted device can be a virtual reality (VR) device, an augmented reality (AR) device, smart glasses, etc. The server 104 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services.

[0033] In an exemplary embodiment, as Figure 2 shown, a bone nail classification method is provided. Taking the server in Figure 1 as an example, the method includes the following steps 202 to step 208. Among them:

[0034] Step 202, obtain the bone nail image to be classified.

[0035] Among them, the bone nail image to be classified is an image including the bone nail to be classified.

[0036] In one embodiment, an image capturing device is used to capture the bone nail to be classified to obtain the bone nail image to be classified.

[0037] Optionally, the image capturing device is not limited, as long as it can capture the bone nail to be classified to obtain the bone nail image to be classified. Exemplarily, the image capturing device can be a high-speed scanner.

[0038] In one embodiment, the bone nail image to be classified can be directly sent to the server by the image capturing device.

[0039] In one embodiment, the bone nail image to be classified can also be sent by the terminal to the server. Exemplarily, an image capturing device can transmit the bone nail image to be classified to the terminal, and the terminal can then transmit it to the server.

[0040] In an alternative embodiment, the bone nail image to be classified can also be generated by a terminal with a photographing function, and the terminal can directly transmit the bone nail image to be classified to the server.

[0041] Step 204: Process the bone nail image to be classified through a bone nail recognition model to obtain the type of the bone nail to be classified and the region of interest.

[0042] Among them, the bone nail recognition model is pre-generated for recognizing the bone nails included in the image.

[0043] Among them, the region of interest is used to represent the image including the bone nail to be classified.

[0044] Optionally, the bone nail recognition model can recognize the type of the bone nail in the bone nail image to be classified.

[0045] In one embodiment, the bone nail recognition model can recognize the region of interest corresponding to the bone nail image to be classified to achieve preliminary segmentation of the bone nail image to be classified. Among them, the region of interest is the image including the bone nail to be separated.

[0046] In one of the embodiments, before processing the bone nail image to be classified using the bone nail recognition model, the bone nail recognition model can be pre-generated.

[0047] In one embodiment, various types of bone nail images can be obtained, the types of the bone nails in the images can be marked, and a deep learning algorithm can be used to learn and train the various types of bone nail images to obtain the bone nail recognition model.

[0048] In one of the embodiments, an image training set can be obtained, and the image training set can be trained using a deep learning algorithm to generate the bone nail recognition model. Optionally, each image in the image training set is marked with a bone nail category and a region of interest, so that subsequent training can be performed based on the marked image training set to generate the bone nail recognition model.

[0049] In an exemplary embodiment, the deep learning algorithm can be the FRCNN algorithm or other algorithms, which is not limited as long as model training can be achieved.

[0050] Step 206: Extract the edge of the region of interest through an edge recognition network to obtain the edge information of the bone nail to be separated, and use the edge information to determine the size information of the bone nail to be classified.

[0051] Among them, the edge recognition network is a pre-trained network for extracting the edges of target objects in images.

[0052] In one embodiment, a global coordinate system can be obtained, and the coordinate information of the bone screw in the global coordinate system can be determined as the position information.

[0053] Optionally, the pixel information of each pixel point in the region of interest can be obtained as the semantic information.

[0054] In one embodiment, after obtaining the position information and the semantic information, the edge recognition network is used to determine the edge information of the bone screw to be classified.

[0055] In one embodiment, the edge recognition network can be a U-NET network. Optionally, the U-NET network consists of an encoder and a decoder, and combines the position information and the semantic information through skip connections to achieve precise segmentation of the bone screw contour.

[0056] In one embodiment, the edge recognition network is used to determine the image gradient map and the gradient direction map of the region of interest; according to the image gradient map and the gradient direction map, the target edge information of the bone screw to be separated is determined; according to the target edge information, the size information of the bone screw to be separated is determined.

[0057] Among them, the image gradient map is an image generated by calculating the gradient value and direction of each pixel point in the image.

[0058] Among them, the gradient direction map is a feature descriptor widely used in the fields of computer vision and image processing and can be used for target detection.

[0059] In one embodiment, an edge detection algorithm can be used to identify the region of interest to determine the edge information of the bone screw to be classified in the region of interest. Optionally, the edge detection algorithm can be Canny, or other algorithms, as long as the edge information in the region of interest can be identified.

[0060] In one embodiment, after determining the edge information, the size information of the bone screw to be separated can be determined according to the edge information. Exemplarily, it can include: obtaining the coordinate information of each pixel point corresponding to the edge information, using a line fitting algorithm to fit the coordinate information of each pixel point to obtain a fitted line; determining the size information of the bone screw to be separated according to the fitted line.

[0061] Step 208, generate a control instruction according to the bone screw type of the bone screw to be classified and the size information of the bone screw to be classified, and transmit the control instruction to the robotic arm.

[0062] Among them, the control instruction is used to instruct the robotic arm to perform the classification action of the bone screw to be classified according to the bone screw type of the bone screw to be classified and the size information of the bone screw to be classified.

[0063] In one embodiment, a storage device may be provided for storing bone nails. Exemplarily, the storage device may be a bone nail box.

[0064] In one of the embodiments, there may be multiple storage devices. Optionally, there may also be one storage device.

[0065] In an alternative embodiment, the storage device may include multiple storage areas. Among them, the storage areas are used for storing bone nails.

[0066] In one embodiment, the storage areas may be of the same size, and the storage device may store bone nails of the same size or the same type.

[0067] In one embodiment, the storage areas may be of different sizes, and the storage device may store bone nails of different sizes, or bone nails of different types.

[0068] In one of the embodiments, a control instruction may be generated according to the fixation type of the bone nails to be separated and the size information of the bone nails to be separated, and the generated control instruction may be sent to the robotic arm. The robotic arm picks up the bone nails to be processed according to the control instruction and places the picked-up bone nails to be processed into the bone nail box.

[0069] In one of the embodiments, the generation of the control instruction may be controlled according to the bone nail requirements. Exemplarily, it may include: obtaining the target bone nail type and the corresponding target quantity; determining the target bone nails from the bone nails to be classified according to the target bone nail type and the bone nail type of the bone nails to be classified, and determining the size information of the target bone nails; generating a control instruction according to the target bone nails, the target quantity, and the size information of the target bone nails, and sending the control instruction to the robotic arm. The control instruction is used to instruct the robotic arm to pick up the target quantity of target bone nails and place the picked-up target bone nails into the target bone nail box corresponding to the size information of the bone nails to be classified.

[0070] Among them, the target bone nail type is used to represent the type of the required bone nails.

[0071] Optionally, the target bone nail type may be pre-determined and stored.

[0072] In one embodiment, the target bone nail type may be stored in the storage space corresponding to the robotic arm. When needed, it communicates with the robotic arm and reads the target bone nail type from the storage space of the robotic arm.

[0073] In one embodiment, the target bone nail type may also be stored in the local cache space. Exemplarily, a cache space is set on the server for storing the target bone nail type. Then, when needed, the target bone nail type is read from the cache space.

[0074] In one embodiment, the target bone screw type can be one or more, which are preset according to requirements.

[0075] In an alternative embodiment, the target quantity is used to represent the number of bone screws of the target bone screw type.

[0076] In one embodiment, the target quantity is set according to requirements. For example, in a certain operation, the number of the first type of bone screws required is the first quantity, and the number of the second type of bone screws required is the second quantity. Then the first quantity and the second quantity are the target quantities.

[0077] Optionally, the target quantity is set corresponding to the target bone screw type, so that after obtaining the target bone screw type, the target quantity can be determined through the corresponding relationship.

[0078] In one embodiment, the target quantity can be stored in the same storage space as the target bone screw type. Exemplarily, if the target bone screw type is stored on the robotic arm side, the target quantity is also stored on the robotic arm side.

[0079] In one embodiment, after obtaining the target bone screw type and the bone screw type of the bone screw to be classified, the target bone screw can be determined according to the target bone screw type and the bone screw type of the bone screw to be classified. Exemplarily, the target bone screw type and the bone screw type of the bone screw to be classified are compared. When the target bone screw type is the same as the bone screw type of the bone screw to be classified, the bone screw type corresponding to the bone screw to be classified is the target bone screw.

[0080] Optionally, after determining the target bone screw, the size information of the target bone screw can be determined in combination with the size information of the bone screw to be separated.

[0081] In an alternative embodiment, after determining the target bone screw, the target quantity, and the size information corresponding to the target bone screw, a control instruction can be generated according to the target bone screw, the target quantity, and the size information corresponding to the target bone screw, and the control instruction is sent to the robotic arm. After receiving the control instruction, the robotic arm classifies the target bone screw according to the indication of the control instruction. Exemplarily, after receiving the control instruction, the robotic arm picks up the target quantity of target bone screws according to the control instruction indication, and places the picked-up target bone screws into the corresponding storage area of the storage device, such as the storage area of the bone screw box.

[0082] In one embodiment, pick-up coordinates can also be generated so that control instructions can be generated in combination with the pick-up coordinates, enabling the robotic arm to directly pick up the target bone screw according to the pick-up coordinates included in the control instructions. Optionally, it may include: determining the coordinate information of the target bone screw in the global coordinate system based on the size information of the target bone screw, and generating pick-up coordinates based on the coordinate information; determining the target pick-up device according to the type of the target bone screw and the size information of the target bone screw; generating control instructions based on the target bone screw, the target quantity, the size information of the target bone screw, the pick-up coordinates, and the target pick-up device.

[0083] Optionally, the position where the robotic arm is located can be used as the origin to generate a global coordinate system. After obtaining the size information of the target bone screw, it can be converted into coordinate information in the global coordinate system so that pick-up coordinates can be generated based on the coordinate information.

[0084] In one embodiment, determine the position information between the workbench on which the bone screws to be classified are placed and the robotic arm, determine the coordinate information of the workbench in the global coordinate system based on the position information, and further determine the coordinate information of the target bone screw in the global coordinate system.

[0085] In one embodiment, the workbench on which the bone screws to be classified are placed can also be used as the origin to generate a global coordinate system. For example, the center point of the workbench can be used as the origin to generate a global coordinate system.

[0086] In one embodiment, after determining the coordinate information of the target bone screw, pick-up coordinates are generated based on the coordinate information of the target bone screw. Exemplarily, the coordinate information corresponding to the center point of the target bone screw can be determined as the pick-up coordinates. For example, assume that the coordinates of the target bone screw are ( , ) and ( , ), then the pick-up coordinates can be ( , ). Optionally, the pick-up coordinates can also be other coordinate information as long as it satisfies that the pick-up coordinates are in the middle of the coordinate information corresponding to the two end vertices of the target bone screw.

[0087] In one embodiment, control instructions can be generated based on the target bone screw, the target quantity, the size information of the target bone screw, and the pick-up coordinates. So that after the robotic arm receives the control instructions, it extracts the target bone screw, the target quantity, the size information of the target bone screw, and the pick-up coordinates from the control instructions to pick up the target bone screw, and places the picked-up target bone screw in the corresponding storage area of the bone screw box.

[0088] In one embodiment, the target pick-up device can also be determined according to the type of the target bone screw and the size information of the target bone screw.

[0089] Optionally, the target picking device can be a suction cup and / or a gripper, etc. Exemplarily, there can be multiple target picking devices.

[0090] It can be understood that different types of bone nails can adopt different target picking devices to improve the picking efficiency. For example, a suction cup is used for the first type of bone nail, and a gripper is used for the second type of bone nail to achieve picking.

[0091] In one embodiment, the picking device can also be determined according to the size information of the target bone nail. For example, when the size information of the target bone nail is less than or equal to the preset size information, a suction cup is used; if the size information is greater than the preset size information, a gripper is used.

[0092] In one of the embodiments, a control instruction can be generated according to the target bone nail, the target quantity, the size information of the target bone nail, the picking coordinates, and the target picking device, and the control instruction can be sent to the robotic arm. The robotic arm classifies the bone nails according to the indication of the control instruction. Exemplarily, the robotic arm uses the target picking device to pick the target quantity of target bone nails according to the picking coordinates, and places the picked target bone nails into the target bone nail box according to the size information of the target bone nail.

[0093] In an optional embodiment, the control instruction can also be generated in combination with the storage area of the bone nail box. Exemplarily, it can include: obtaining the vacant area of the bone nail box, and obtaining the bone nail type of the bone nails that can be stored corresponding to the vacant area and the size information of the vacant area; selecting the area to be determined from the vacant area of the bone nail box according to the bone nail type of the bone nails that can be stored and the bone nail type of the bone nails that can be stored; screening out the target area from the area to be determined according to the size information of the control area and the size information of the target bone nail; generating a control instruction according to the picking coordinates, the target picking device, the target bone nail, the target quantity, and the target area.

[0094] Among them, the vacant area is used to represent the storage area where bone nails can be stored.

[0095] In one embodiment, the vacant area can be obtained in real time each time a bone nail is placed. For example, identifying the vacant area included in the picture corresponding to the bone nail box.

[0096] Optionally, the storage area of the bone nail box can also be obtained and stored before bone nail classification. Each time a bone nail is stored, the stored area where the bone nail is stored is deleted from the storage area, and the remaining is the vacant area.

[0097] In one of the embodiments, after obtaining the vacant area, the bone nail type of the bone nails that can be stored corresponding to the vacant area and the size information of the control area can be obtained. Optionally, the target area can be determined in combination with the bone nail type of the target bone nail and the size information of the target bone nail.

[0098] In one embodiment, the vacant area can be screened to obtain the area to be determined according to the type of bone nail that can be stored and the type of the target bone nail. Exemplarily, the control area where the type of bone nail that can be stored is the same as the type of the target bone nail is determined as the area to be determined. Optionally, after determining the area to be determined, the type of bone nail of the bone nail that can be stored corresponding to the area to be determined and the size information can be determined.

[0099] In one of the embodiments, the target area can be further screened from the area to be determined in combination with the size information of the target bone nail. Exemplarily, the area where the size information in the area to be determined is greater than or equal to the size information of the target bone nail is determined as the target area.

[0100] In an alternative embodiment, a control instruction can be generated according to the picking coordinates, the target picking device, the target bone nail, the target quantity, and the target area. After receiving the control instruction, the robotic arm can be used according to the indication of the control instruction to pick up the target quantity of target bone nails by using the target picking device and place the target bone nails in the target area.

[0101] In the above bone nail classification method, the bone nail recognition model is used to recognize the image of the bone nail to be classified, obtaining the region of interest and the fixing type of the bone nail to be classified. The edge recognition network is used to recognize the edge of the region of interest, so as to determine the size information of the bone nail to be classified according to the recognized edge, and a control instruction can be generated according to the type and size information of the bone nail to be classified. The robotic arm can execute the classification action on the bone nail to be classified according to the control instruction, so as to realize the classification of the bone nail to be classified. The above steps utilize an automated process of recognition, picking, and classification, effectively improving the bone nail classification efficiency and the accuracy of bone nail classification.

[0102] In an exemplary embodiment, as Figure 3 shown, a bone nail classification method is provided. Taking the case where the method is applied to the Figure 1 server as an example, it includes the following steps 302 to step 328. Among them:

[0103] Step 302, obtain the image of the bone nail to be classified.

[0104] Step 304, process the image of the bone nail to be classified through the bone nail recognition model to obtain the type of the bone nail to be classified and the region of interest.

[0105] Among them, the region of interest is the image including the bone nail to be classified.

[0106] Step 306, use the edge recognition network to determine the image gradient map and the gradient direction map of the region of interest.

[0107] Step 308: Determine the target edge information of the bone screw to be separated according to the image gradient map and the gradient direction map.

[0108] Step 310: Obtain the coordinate information of each pixel point corresponding to the target edge information, and use the line fitting algorithm to fit the coordinate information of each pixel point to obtain a fitted line.

[0109] Step 312: Determine the size information of the bone screw to be separated according to the fitted line.

[0110] Step 314: Obtain the target bone screw type and the target quantity corresponding to the target bone screw type.

[0111] Step 316: Determine the target bone screws from the bone screws to be classified according to the target bone screw type and the bone screw type of the bone screws to be classified, and determine the size information of the target bone screws.

[0112] Step 318: Determine the coordinate information of the target bone screws in the global coordinate system according to the size information of the target bone screws, and generate pick-up coordinates according to the coordinate information.

[0113] Step 320: Determine the target pick-up device according to the bone screw type of the target bone screws and the size information of the target bone screws.

[0114] Step 322: Obtain the vacant area of the bone screw box, and obtain the bone screw type of the bone screws that can be stored corresponding to the vacant area and the size information of the vacant area.

[0115] Step 324: Select the area to be determined from the vacant area of the bone screw box according to the bone screw type of the bone screws that can be stored and the bone screw type of the bone screws that can be stored.

[0116] Step 326: Screen out the target area from the area to be determined according to the size information of the control area and the size information of the target bone screws.

[0117] Step 328: Generate a control command according to the pick-up coordinates, the target pick-up device, the target bone screws, the target quantity and the target area, and send the control command to the robotic arm.

[0118] Among them, the control command is used to instruct the robotic arm to pick up the target quantity of target bone screws at the pick-up coordinates by using the target pick-up device, and place the target bone screws into the target area.

[0119] In this embodiment, a bone nail recognition model is used to recognize the bone nail image to be classified, obtain the region of interest and the fixation type of the bone nail to be classified, and an edge recognition network is used to recognize the edge of the region of interest, so as to determine the size information of the bone nail to be classified according to the recognized edge. And a control instruction can be generated according to the bone nail type and size information of the bone nail to be classified. The robotic arm can execute the classification action on the bone nail to be classified according to the control instruction, so as to realize the classification of the bone nail to be classified. In the above steps, the automated recognition, picking, and classification process is used to effectively improve the bone nail classification efficiency and the accuracy of bone nail classification.

[0120] It should be understood that although the steps in the flowcharts involved in the above embodiments are displayed in sequence according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless there is a clear description in this article, the execution of these steps has no strict order limit, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be executed alternately or alternately with at least a part of other steps or steps or stages in other steps.

[0121] Based on the same inventive concept, an embodiment of the present application also provides a bone nail classification device for implementing the bone nail classification method described above. The implementation solution provided by this device to solve the problem is similar to the implementation solution described in the above method. Therefore, the specific limitations in one or more embodiments of the bone nail classification device provided below can refer to the limitations on the bone nail classification method in the above text, and will not be repeated here.

[0122] In an exemplary embodiment, as Figure 4 shown, a bone nail classification device 400 is provided, including: an image acquisition module 402, a recognition module 404, a size information determination module 406, and a control instruction generation module 408, where:

[0123] The image acquisition module 402 is used to acquire the bone nail image to be classified.

[0124] The recognition module 404 is used to process the bone nail image to be classified through a bone nail recognition model, and obtain the bone nail type and the region of interest of the bone nail to be classified. The region of interest is an image including the bone nail to be classified.

[0125] The size information determination module 406 is used to extract the edge of the region of interest through an edge recognition network, obtain the edge information of the bone nail to be classified, and determine the size information of the bone nail to be classified by using the edge information.

[0126] The control instruction generation module 408 is configured to generate a control instruction according to the bone nail type of the bone nail to be classified and the size information of the bone nail to be classified, and transmit the control instruction to the robotic arm. The control instruction is used to instruct the robotic arm to perform a classification action on the bone nail to be classified according to the bone nail type of the bone nail to be classified and the size information of the bone nail to be classified.

[0127] In one embodiment, the control instruction generation module is further configured to obtain a target bone nail type and a target quantity corresponding to the target bone nail type; determine target bone nails from the bone nails to be classified according to the target bone nail type and the bone nail type of the bone nails to be classified, and determine the size information of the target bone nails; generate a control instruction according to the target bone nails, the target quantity, and the size information of the target bone nails, and send the control instruction to the robotic arm. The control instruction is used to instruct the robotic arm to pick up the target quantity of target bone nails and place the picked-up target bone nails into a target bone nail box corresponding to the size information of the bone nails to be classified.

[0128] In one of the embodiments, the control instruction generation module is further configured to determine the coordinate information of the target bone nails in the global coordinate system according to the size information of the target bone nails, and generate a pick-up coordinate according to the coordinate information; determine a target pick-up device according to the bone nail type of the target bone nails and the size information of the target bone nails; generate a control instruction according to the target bone nails, the target quantity, the size information of the target bone nails, the pick-up coordinate, and the target pick-up device. The control instruction is used to instruct the robotic arm to use the target pick-up device to pick up the target quantity of target bone nails according to the pick-up coordinate, and place the picked-up target bone nails into the target bone nail box according to the size information of the target bone nails.

[0129] In one embodiment, the control instruction generation module is further configured to obtain the vacant area of the bone nail box, and obtain the bone nail type of the bone nails that can be stored corresponding to the vacant area and the size information of the vacant area; select a to-be-determined area from the vacant area of the bone nail box according to the bone nail type of the bone nails that can be stored and the bone nail type of the bone nails that can be stored; screen out a target area from the to-be-determined area according to the size information of the control area and the size information of the target bone nails; generate a control instruction according to the pick-up coordinate, the target pick-up device, the target bone nails, the target quantity, and the target area. The control instruction is used to instruct the robotic arm to use the target pick-up device to pick up the target quantity of target bone nails at the pick-up coordinate and place the target bone nails into the target area.

[0130] In one of the embodiments, the size information determination module is further configured to use an edge recognition network to determine an image gradient map and a gradient direction map of the region of interest; determine the target edge information of the bone nails to be separated according to the image gradient map and the gradient direction map; determine the size information of the bone nails to be separated according to the target edge information.

[0131] In one embodiment, the size information determination module is further configured to obtain the coordinate information of each pixel point corresponding to the edge information, fit the coordinate information of each pixel point using a line fitting algorithm to obtain a fitted line; and determine the size information of the bone nail to be separated according to the fitted line.

[0132] Each module in the above bone nail classification device can be implemented in whole or in part by software, hardware, and their combination. Each of the above modules can be embedded in or independent of a processor in a computer device in the form of hardware, or stored in a memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0133] In an exemplary embodiment, a computer device is provided. The computer device can be a server, and its internal structure diagram can be as Figure 5 shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O), and a communication interface. Among them, the processor, the memory, and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with an external terminal through a network connection. The computer program, when executed by the processor, implements a bone nail classification method.

[0134] Those skilled in the art can understand that Figure 5 the structure shown in

[0135] is only a block diagram of some structures related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have a different component layout.

[0136] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above method embodiments are implemented.

[0137] In one embodiment, a computer program product is provided, including a computer program which, when executed by a processor, implements the steps of the above-mentioned method embodiments.

[0138] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data need to comply with relevant regulations.

[0139] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned method embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the above-mentioned method embodiments. Among them, any reference to a memory, database, or other medium used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The databases involved in the embodiments provided in this application can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., without limitation. The processors involved in the embodiments provided in this application can be general-purpose processors, central processors, graphics processors, digital signal processors, programmable logic devices, data processing logics based on quantum computing, artificial intelligence (AI) processors, etc., without limitation.

[0140] The technical features of the above embodiments can be combined arbitrarily. For the sake of concise description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope recorded in this application.

[0141] The above embodiments only express several implementation manners of this application, and their descriptions are relatively specific and detailed. However, it should not be construed as a limitation on the patent scope of this application. It should be noted that for those of ordinary skill in the art, without departing from the concept of this application, several modifications and improvements can still be made, and these all belong to the protection scope of this application. Therefore, the protection scope of this application shall be subject to the appended claims.

Claims

1. A bone screw classification method, characterized in that: The method comprises: Acquire a bone nail image to be classified; Processing the bone screw image to be classified by using a bone screw recognition model to obtain the bone screw type and a region of interest of the bone screw to be classified, wherein the region of interest is the image including the bone screw to be classified; Extracting edges of the region of interest through an edge recognition network to obtain edge information of the bone screws to be classified, and determining size information of the bone screws to be classified using the edge information; A control instruction is generated according to the bone screw type of the bone screw to be classified and the size information of the bone screw to be classified, and the control instruction is transmitted to the robotic arm, wherein the control instruction is used to instruct the robotic arm to perform a classification action of the bone screw to be classified according to the bone screw type of the bone screw to be classified and the size information of the bone screw to be classified.

2. The method according to claim 1, characterized in that The step of generating a control instruction according to the bone screw type of the bone screw to be classified and the size information of the bone screw to be classified comprises: Acquire a target bone screw type and a target quantity corresponding to the target bone screw type; According to the target bone screw type and the bone screw types of the bone screws to be classified, determining a target bone screw from the bone screws to be classified, and determining size information of the target bone screw; A control instruction is generated according to the target bone screws, the target quantity and the size information of the target bone screws, and the control instruction is sent to the robotic arm. The control instruction is used to instruct the robotic arm to pick up the target quantity of the target bone screws and place the picked-up target bone screws into a target bone screw box corresponding to the size information of the bone screws to be classified.

3. The method according to claim 2, characterized in that The generating of the control instruction according to the target bone screw, the target quantity and the size information of the target bone screw comprises: Determining coordinate information of the target bone screw in the global coordinate system according to the size information of the target bone screw, and generating picking coordinates according to the coordinate information; Determining a target picking device according to the bone screw type of the target bone screw and the size information of the target bone screw; The control instruction is generated according to the target bone screws, the target quantity, the size information of the target bone screws, the picking coordinates and the target picking device. The control instruction is used to instruct the robot arm to use the target picking device to pick up the target quantity of target bone screws according to the picking coordinates, and place the picked up target bone screws into the target bone screw box according to the size information of the target bone screws.

4. The method according to claim 3, characterized in that The generating the control instruction according to the target bone screw, the target quantity, the size information of the target bone screw, the picking coordinates and the target picking device includes: Acquire a vacant area of ​​the bone screw box, and acquire the bone screw type that can store the bone screws corresponding to the vacant area and the size information of the vacant area; According to the bone screw types that can store bone screws and the bone screw types that can store bone screws, a region to be determined is selected from the vacant regions of the bone screw box; Filtering a target area from the to-be-determined area according to the size information of the vacant area and the size information of the target bone screw; A control instruction is generated according to the picking coordinates, the target picking device, the target bone screws, the target quantity and the target area, wherein the control instruction is used to instruct the robot arm to use the target picking device to pick up the target quantity of the target bone screws at the picking coordinates and place the target bone screws into the target area.

5. The method according to claim 1, characterized in that The step of extracting edges of the region of interest by using an edge recognition network to obtain edge information of the bone screw to be classified includes: Determine the image gradient map and gradient direction map of the region of interest using the edge recognition network; Determining target edge information of the bone screw to be classified according to the image gradient map and the gradient direction map; The size information of the bone screw to be separated is determined according to the target edge information.

6. The method according to claim 5, characterized in that The step of determining the size information of the bone screw to be separated by using the target edge information includes: Obtaining coordinate information of each pixel point corresponding to the edge information, and fitting the coordinate information of each pixel point using a straight line fitting algorithm to obtain a fitting straight line; The size information of the bone screw to be separated is determined according to the fitting straight line.

7. A bone screw sorting device, characterized in that: The device comprises: An image acquisition module, used for acquiring bone nail images to be classified; A recognition module, used for processing the bone nail image to be classified by using a bone nail recognition model to obtain the bone nail type and a region of interest of the bone nail to be classified, wherein the region of interest is the image including the bone nail to be classified; A size information determination module, configured to extract edges of the region of interest through an edge recognition network to obtain edge information of the bone screws to be classified, and determine size information of the bone screws to be classified using the edge information; A control instruction generating module is used to generate a control instruction according to the bone screw type of the bone screw to be classified and the size information of the bone screw to be classified, and transmit the control instruction to the robotic arm, wherein the control instruction is used to instruct the robotic arm to perform a classification action of the bone screw to be classified according to the bone screw type of the bone screw to be classified and the size information of the bone screw to be classified.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images