Medical waste sorting method and system based on depth camera

Abstract

This application relates to the field of image recognition technology and provides a method and system for sorting medical waste based on a depth camera. The method's implementation principle is as follows: a terminal device acquires an image to be detected using a depth camera; then, based on a target detection model, it generates object category information and three-dimensional object information; finally, based on the object's three-dimensional and category information, it accurately generates and executes robotic arm sorting instructions. This application can ensure high accuracy in medical waste identification and compliant classification, which is beneficial for handling large-scale medical waste flows, significantly reducing reliance on manual operation, lowering the possibility of human error, reducing the workload of operators, improving overall work efficiency and operational safety, promoting the standardization, intelligentization, and environmental protection of medical waste management, and contributing to a clean and safe medical environment.

Description

Technical Field

[0001] This application relates to the technical field of image recognition, and more specifically, to a method and system for sorting medical waste based on a depth camera. Background Technology

[0002] The hazards of medical waste mainly lie in its infectiousness, chemical and physical risks. If medical waste is not properly disposed of, it poses a risk of spreading disease and polluting the environment. Therefore, sorting medical waste is particularly important.

[0003] Currently, the sorting of medical waste mainly relies on manual operation. However, manual sorting is easily affected by fatigue. After long hours of high-intensity work, the efficiency of workers will decrease significantly, resulting in low sorting efficiency, which needs further improvement. Summary of the Invention

[0004] Based on this, embodiments of this application provide a medical waste sorting method and system based on a depth camera to solve the problem of low sorting efficiency in the prior art.

[0005] In a first aspect, embodiments of this application provide a medical waste sorting method based on a depth camera, the method comprising: Based on a preset depth camera, acquire the image to be detected; Based on the preset target detection model and the image to be detected, generate object category information and object 3D information; Based on the object's three-dimensional information and object category information, a robotic arm sorting instruction is generated and executed, wherein the robotic arm sorting instruction is used to drive a preset sorting robotic arm to perform sorting processing.

[0006] Compared with existing technologies, the beneficial effects are as follows: The medical waste sorting method based on a depth camera provided in this application allows the terminal device to acquire the image to be detected based on the depth camera, and then quickly generate object category information and object 3D information according to the preset target detection model and the image to be detected. Finally, based on the object 3D information and object category information, the robotic arm sorting instructions are accurately generated and executed, thereby ensuring high accuracy in medical waste identification and compliance with regulations for classification and processing. This is beneficial for handling large-scale medical waste flows, significantly reducing reliance on manual operation, alleviating the labor intensity of operators, improving the overall sorting efficiency and operational safety, and to a certain extent solving the problem of low sorting efficiency at present.

[0007] Secondly, embodiments of this application provide a medical waste sorting system based on a depth camera, the system comprising: Image acquisition module: used to acquire the image to be detected based on a preset depth camera; Object category information generation module: used to generate object category information and object 3D information based on the preset target detection model and the image to be detected; Robotic arm sorting instruction generation module: used to generate and execute robotic arm sorting instructions based on the object's three-dimensional information and object category information, wherein the robotic arm sorting instructions are used to drive a preset sorting robotic arm to perform sorting processing.

[0008] Thirdly, embodiments of this application provide a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method described in the first aspect above.

[0009] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the method described in the first aspect above.

[0010] It is understood that the beneficial effects of the second to fourth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here. Attached Figure Description

[0011] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below.

[0012] Figure 1 This is a schematic flowchart of a medical waste sorting method provided in an embodiment of this application; Figure 2 This is a flowchart illustrating the process after step S100 in a medical waste sorting method provided in an embodiment of this application. Figure 3 This is a flowchart illustrating step S200 in a medical waste sorting method provided in an embodiment of this application; Figure 4 This is a flowchart illustrating step S300 in a medical waste sorting method provided in an embodiment of this application; Figure 5 This is a flowchart illustrating the process after step S300 in a medical waste sorting method provided in an embodiment of this application. Figure 6 This is a block diagram of a medical waste sorting system based on a depth camera, provided in one embodiment of this application. Figure 7 This is a schematic diagram of a terminal device provided in an embodiment of this application. Detailed Implementation

[0013] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0014] In the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0015] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0016] To illustrate the technical solution described in this application, specific embodiments are provided below.

[0017] Please see Figure 1 , Figure 1 This is a schematic flowchart of a medical waste sorting method based on a depth camera provided in this application embodiment. In this embodiment, the executing entity of the medical waste sorting method is a terminal device. It is understood that the types of terminal devices include, but are not limited to, tablet computers, laptops, Ultra-Mobile Personal Computers (UMPCs), netbooks, Personal Digital Assistants (PDAs), etc., and this application embodiment does not impose any restrictions on the specific type of terminal device.

[0018] Please see Figure 1 The medical waste sorting method provided in this application includes, but is not limited to, the following steps: In S100, the image to be detected is acquired based on a preset depth camera.

[0019] Specifically, the terminal device can first take pictures of the place where a large amount of medical waste is placed based on a preset depth camera to obtain the image to be detected. The medical waste can be needles, glass containers, plastic packaging, masks and syringes, etc.

[0020] For some possible implementations, please refer to [link / reference needed] to improve recognition accuracy. Figure 2 After step S100, the method further includes, but is not limited to, the following steps: In step S101, the image to be detected is subjected to grayscale conversion processing to generate grayscale image information.

[0021] Specifically, the terminal device can perform grayscale conversion processing on the image to be detected to quickly generate grayscale image information. The grayscale image information is used to describe the image to be detected after grayscale conversion processing. For example, the terminal device can take the average value of the corresponding values ​​of the three RGB channels of the image to be detected.

[0022] In S102, grayscale mean information is generated based on grayscale image information.

[0023] Specifically, after the terminal device generates grayscale image information, the terminal device can generate grayscale mean information based on the grayscale image information. The grayscale mean information is used to describe the average grayscale value of the grayscale image information.

[0024] For example, the terminal device can first add up the gray values ​​of all pixels in the grayscale image information to obtain the sum of gray values, and then divide the sum of gray values ​​by the total number of pixels to obtain the gray average information.

[0025] In S103, it is determined whether the grayscale mean value is greater than the preset compliance mean value.

[0026] Specifically, after the terminal device generates grayscale average information, the terminal device can determine whether the grayscale average information is greater than the preset compliance average information, thereby determining whether the light brightness meets the requirements. The value of the compliance average information can be customized, for example, 80.

[0027] In S104, if the grayscale mean information is less than the compliance mean information, then supplementary lighting instruction information is generated.

[0028] Specifically, if the grayscale average information is less than the compliant average information, the terminal device can generate supplementary lighting instruction information, which is used to instruct the depth camera to activate the supplementary lighting.

[0029] In S105, in response to the supplementary lighting instruction information, the system again executes the acquisition of the image to be detected based on the preset depth camera until it determines whether the grayscale mean information is greater than the preset compliance mean information, until the grayscale mean information is greater than or equal to the compliance mean information.

[0030] Specifically, after the terminal device generates the supplementary light instruction information, the terminal device can respond to the supplementary light instruction information and execute the above steps S100 to S103 again until the grayscale average information is greater than or equal to the compliant average information.

[0031] In S200, object category information and object 3D information are generated based on the preset target detection model and the image to be detected.

[0032] Specifically, after the terminal device acquires the image to be detected, it can accurately generate object category information and object 3D information based on the preset target detection model and the image to be detected. The object 3D information includes object shape information, object size information, and object position information. The object shape information describes the shape of the object to be detected, the object size information describes the size of the object to be detected, and the object position information describes the position of the object to be detected.

[0033] In some possible implementations, to accurately generate object category information and object 3D information, please refer to [link / reference needed]. Figure 3 Step S200 includes, but is not limited to, the following steps: In S210, edge contour information is generated based on the image to be detected using a preset edge detection algorithm.

[0034] Specifically, the terminal device can generate edge contour information based on the image to be detected using a preset edge detection algorithm. The edge detection algorithm can be an edge detection algorithm based on the Roberts operator or an edge detection algorithm based on the Prewitt operator. The edge contour information is used to describe the edge contours corresponding to objects in the image to be detected.

[0035] In S220, based on the object detection model, object category information and object shape information are generated according to the edge contour information.

[0036] Specifically, after the terminal device generates edge contour information, it can perform target detection processing on the area enclosed by the edge contour information based on the target detection model to generate object category information and object shape information. The target detection model can be a target detection model based on the YOLOv8 algorithm; the object category information can be bandages, gloves, masks, blades, or medicine packaging boxes.

[0037] In S230, the camera parameter set information of the depth camera is obtained.

[0038] Specifically, after the terminal device generates object category information and object shape information, the terminal device can obtain the camera parameter set information of the depth camera, which includes focal length information, pixel size information, pixel width information and relative distance information.

[0039] In S240, based on object category information, the object size information is determined according to focal length information, pixel size information, pixel width information, and relative distance information.

[0040] Specifically, after the terminal device obtains the camera parameter set information, it can quickly determine the object size information based on the object category information, focal length information, pixel size information, pixel width information, and relative distance information.

[0041] For example, the terminal device can input focal length information, pixel size information, pixel width information, and relative distance information into a preset size calculation function to determine the object size information, wherein the size calculation function is: , In the formula, For object size information, This is pixel width information. For pixel size information, This refers to relative distance information, specifically the distance between the object to be detected and the depth camera. This is focal length information.

[0042] In S250, the object's position information is determined based on the focal length information.

[0043] Specifically, after the terminal device determines the object's size information, it can accurately determine the object's position information based on the focal length information.

[0044] For example, the terminal device can determine the object's position information based on focal length information and a position calculation function, wherein the position calculation function can be: , In the formula, The x-coordinate represents the object's position information. The vertical coordinate represents the object's position information. The height coordinates are the object's position information. The x-coordinate of the object region in pixels. The vertical coordinate of the object region in pixels. This represents the depth value of the object region in pixels. The x-coordinate corresponding to the principal point coordinates calibrated by the camera. The ordinate is the coordinate of the principal point calibrated by the camera. This refers to the focal length parameter of the depth camera on the horizontal axis. Focal length parameters of the depth camera on the vertical axis In S300, robotic arm sorting instructions are generated and executed based on the object's 3D information and object category information.

[0045] Specifically, after the terminal device generates object category information and object 3D information, the terminal device can generate and execute robotic arm sorting instructions based on the object 3D information and object category information, thereby realizing automatic sorting of medical waste. Among them, the robotic arm sorting instructions are used to drive the preset sorting robotic arm to perform sorting processing.

[0046] In some possible implementations, for generating and executing robotic arm sorting instructions, please refer to [link / reference]. Figure 4 Step S300 includes, but is not limited to, the following steps: In S310, the position information of the robotic arm end effector is acquired, and the position information of the object is continuously acquired.

[0047] Specifically, after the terminal device generates object category information and object 3D information, the terminal device can obtain the end position information of the robotic arm and continuously obtain object position information. The end position information of the robotic arm is used to describe the 3D position corresponding to the end of the sorting robotic arm.

[0048] In S320, it is determined whether the latest object position information is equal to the original object position information.

[0049] Specifically, after the terminal device obtains the position information of the robotic arm's end effector, it can determine whether the latest object position information is equal to the original object position information.

[0050] In S330, if the latest object position information is equal to the original object position information, then the robot arm motion trajectory information is generated based on the robot arm end position information and the original object position information; otherwise, the robot arm motion trajectory information is generated based on the robot arm end position information and the latest object position information.

[0051] Specifically, if the latest object position information is equal to the original object position information, the terminal device can generate the robotic arm movement trajectory information based on the robotic arm end position information and the original object position information. Otherwise, it indicates that the position of the medical waste has changed. Therefore, the terminal device can generate the robotic arm movement trajectory information based on the robotic arm end position information and the latest object position information, so that even if the position and state of the object to be detected changes during the waste sorting process, the sorting robotic arm can still adjust its movement trajectory and actions in a timely manner.

[0052] For guidance on designing and optimizing the sorting robot arm, please refer to the following descriptions of some possible implementations. Figure 5 After step S300, the method further includes, but is not limited to, the following steps: In the S400, in response to the sorting completion command, information on the joint friction force and load change of the sorting robot arm is obtained.

[0053] Specifically, the terminal device can respond to the sorting completion command and obtain the joint friction force information and load change information of the sorting robot arm. The joint friction force information is used to describe the joint friction force of the sorting robot arm, and the load change information is used to describe the load change of the sorting robot arm.

[0054] In S410, joint friction information and load change information are uploaded to a preset cloud database.

[0055] Specifically, after the terminal device obtains the joint friction force information and load change information, the terminal device can upload the joint friction force information and load change information to a preset cloud database.

[0056] The implementation principle of the medical waste sorting method based on a depth camera in this application embodiment is as follows: The terminal device can acquire the image to be detected based on the depth camera, and then quickly generate object category information and object 3D information according to the preset target detection model and the image to be detected. Finally, based on the object 3D information and object category information, the robotic arm sorting instructions are accurately generated and executed, thereby ensuring high accuracy in medical waste identification and compliance with regulations for classification and processing. This is beneficial for handling large-scale medical waste flows, significantly reducing reliance on manual operation, reducing the possibility of human error, reducing the labor intensity of operators, and improving overall work efficiency and operational safety.

[0057] It should be noted that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0058] Embodiments of this application also provide a medical waste sorting system based on a depth camera. For ease of explanation, only the parts relevant to this application are shown, such as... Figure 6 As shown, the system 60 includes: Image acquisition module 61: used to acquire the image to be detected based on a preset depth camera; Object category information generation module 62: used to generate object category information and object 3D information based on the preset target detection model and the image to be detected; Robotic arm sorting instruction generation module 63: used to generate and execute robotic arm sorting instructions based on the object's three-dimensional information and object category information, wherein the robotic arm sorting instructions are used to drive the preset sorting robotic arm to perform sorting processing.

[0059] Optionally, the three-dimensional information of the object includes object shape information, object size information, and object position information; the object category information generation module 62 mentioned above includes: Edge contour information generation submodule: used to generate edge contour information based on the image to be detected, according to a preset edge detection algorithm; Object category information generation submodule: Used to generate object category information and object shape information based on the object detection model and edge contour information; Camera parameter set information acquisition submodule: used to acquire camera parameter set information of depth camera, including focal length information, pixel size information, pixel width information and relative distance information; Object size information determination submodule: used to determine object size information based on object category information, focal length information, pixel size information, pixel width information, and relative distance information; Object position information determination submodule: used to determine the object position information based on focal length information.

[0060] Optionally, the system 60 also includes: Grayscale image information generation module: used to perform grayscale conversion processing on the image to be detected and generate grayscale image information; Gray-scale mean information generation module: used to generate gray-scale mean information based on gray-scale image information; Gray-scale mean information judgment module: used to determine whether the gray-scale mean information is greater than the preset compliance mean information; The fill light instruction information generation module is used to generate fill light instruction information if the grayscale average information is less than the compliance average information. The fill light instruction information is used to instruct the activation of the fill light of the depth camera. Re-execution module: In response to the supplementary lighting instruction, it re-executes the image to be detected based on the preset depth camera until it determines whether the grayscale mean information is greater than the preset compliance mean information, until the grayscale mean information is greater than or equal to the compliance mean information.

[0061] Optionally, the robotic arm sorting instruction generation module 63 includes: Robotic arm end-effector position information acquisition submodule: used to acquire the position information of the robotic arm end-effector and continuously acquire the position information of the object; Object position information determination submodule: used to determine whether the latest object position information is equal to the original object position information; The robotic arm motion trajectory information generation submodule is used to generate robotic arm motion trajectory information based on the robotic arm end-effector position information and the original object position information if the latest object position information is equal to the original object position information; otherwise, it generates robotic arm motion trajectory information based on the robotic arm end-effector position information and the latest object position information.

[0062] Optionally, the system 60 also includes: Joint friction information acquisition module: used to acquire joint friction information and load change information of the sorting robot arm in response to the sorting completion command; Joint friction information upload module: used to upload joint friction information and load change information to a preset cloud database.

[0063] It should be noted that the information interaction and execution process between the above modules are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, which will not be repeated here.

[0064] This application also provides a terminal device, such as... Figure 7 As shown, the terminal device 70 of this embodiment includes: a processor 71, a memory 72, and a computer program 73 stored in the memory 72 and executable on the processor 71. When the processor 71 executes the computer program 73, it implements the steps described in the above-described medical waste sorting method embodiment, for example... Figure 1 Steps S100 to S300 are shown; or, when processor 71 executes computer program 73, it implements the functions of each module in the above-described device, for example... Figure 6 The functions of modules 61 to 63 are shown.

[0065] The terminal device 70 can be a desktop computer, laptop, handheld computer, cloud server, or other computing device, and includes, but is not limited to, a processor 71 and a memory 72. Those skilled in the art will understand that... Figure 7 This is merely an example of terminal device 70 and does not constitute a limitation on terminal device 70. It may include more or fewer components than shown, or combine certain components, or different components. For example, terminal device 70 may also include input / output devices, network access devices, buses, etc.

[0066] The processor 71 can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.; the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0067] The memory 72 can be an internal storage unit of the terminal device 70, such as the hard disk or memory of the terminal device 70. The memory 72 can also be an external storage device of the terminal device 70, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the terminal device 70. Furthermore, the memory 72 can include both internal storage units and external storage devices of the terminal device 70. The memory 72 can also store computer program 73 and other programs and data required by the terminal device 70. The memory 72 can also be used to temporarily store data that has been output or will be output.

[0068] One embodiment of this application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable file, or some intermediate form. The computer-readable medium can include any entity or device capable of carrying computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.

[0069] The above are all preferred embodiments of this application, and are not intended to limit the scope of protection of this application. Therefore, all equivalent changes made in accordance with the methods, principles and structures of this application should be covered within the scope of protection of this application.

Claims

1. A method for sorting medical waste based on a depth camera, characterized in that, The method includes: Based on a preset depth camera, acquire the image to be detected; Based on the preset target detection model and the image to be detected, generate object category information and object 3D information; Based on the object's three-dimensional information and object category information, a robotic arm sorting instruction is generated and executed, wherein the robotic arm sorting instruction is used to drive a preset sorting robotic arm to perform sorting processing.

2. The method according to claim 1, characterized in that, The three-dimensional information of the object includes object shape information, object size information, and object position information; The step of generating object category information and object 3D information based on a preset target detection model and the image to be detected includes: Based on a preset edge detection algorithm, edge contour information is generated according to the image to be detected; Based on the target detection model, the object category information and object shape information are generated according to the edge contour information; Obtain the camera parameter set information of the depth camera, wherein the camera parameter set information includes focal length information, pixel size information, pixel width information and relative distance information; Based on the object category information, the object size information is determined according to the focal length information, pixel size information, pixel width information, and relative distance information; Based on the focal length information, the position information of the object is determined.

3. The method according to claim 1, characterized in that, After acquiring the image to be detected based on the preset depth camera, the method further includes: The image to be detected is subjected to grayscale conversion processing to generate grayscale image information; Based on the grayscale image information, grayscale mean information is generated; Determine whether the grayscale mean value is greater than the preset compliance mean value; If the grayscale mean information is less than the compliance mean information, then fill light instruction information is generated, wherein the fill light instruction information is used to instruct the fill light of the depth camera to be activated; In response to the supplementary lighting instruction, the preset depth camera is executed again to acquire the image to be detected until the grayscale mean information is greater than the preset compliance mean information, until the grayscale mean information is greater than or equal to the compliance mean information.

4. The method according to claim 2, characterized in that, The process of generating and executing robotic arm sorting instructions based on the object's 3D information and object category information includes: Acquire the position information of the robotic arm's end effector and continuously acquire the object's position information; Determine whether the latest object position information is equal to the original object position information; If the latest object position information is equal to the original object position information, then the robot arm motion trajectory information is generated based on the robot arm end position information and the original object position information; otherwise, the robot arm motion trajectory information is generated based on the robot arm end position information and the latest object position information.

5. The method according to claim 1, characterized in that, After generating and executing robotic arm sorting instructions based on the object's three-dimensional information and object category information, the method further includes: In response to the sorting completion command, the joint friction force information and load change information of the sorting robot arm are obtained; The joint friction information and load change information are uploaded to a preset cloud database.

6. A medical waste sorting system based on a depth camera, characterized in that, The system includes: Image acquisition module: used to acquire the image to be detected based on a preset depth camera; Object category information generation module: used to generate object category information and object 3D information based on the preset target detection model and the image to be detected; Robotic arm sorting instruction generation module: used to generate and execute robotic arm sorting instructions based on the object's three-dimensional information and object category information, wherein the robotic arm sorting instructions are used to drive a preset sorting robotic arm to perform sorting processing.

7. The system according to claim 6, characterized in that, The system also includes: Joint friction information acquisition module: used to acquire joint friction information and load change information of the sorting robot arm in response to the sorting completion command; Joint friction information upload module: used to upload the joint friction information and load change information to a preset cloud database.

8. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1 to 5.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 5.

Images