A medical waste garbage can residue detection method based on multi-dimensional machine vision

Abstract

This invention discloses a method for detecting residues in medical waste bins based on multidimensional machine vision. First, a database of the overall three-dimensional feature information of the bin is established. Depth, light intensity, and color images of the detection scene are collected and calculated, and a point cloud map is calculated after preprocessing. Three-dimensional features of the point cloud plane at the bin opening are extracted, and the overall three-dimensional feature information is matched from the database to segment the bottom area to be detected. Based on the depth and color images of the bottom detection area, an initial residue result contour sequence is extracted, and the contour sequence is further filtered using the scene's light intensity image to obtain the final detection result. This invention integrates depth, light intensity, and color images, resulting in high detection accuracy, fast speed, and strong adaptability. It effectively avoids the adverse effects of manual inspection, providing a guarantee for the safe disposal of medical waste bins.

Description

Technical Field

[0001] This invention relates to the fields of medical waste treatment and machine vision technology, specifically to a method for detecting residues in medical waste bins based on multidimensional machine vision. Background Technology

[0002] Medical waste is characterized by acute infectiousness, latent potential, and spatial contamination. The pathogens it carries pose a significant threat, directly endangering human health and potentially causing even more serious consequences by polluting soil, water, and the atmosphere. Currently, medical waste disposal involves several steps: bin handling, waste dumping, waste incineration, residue detection inside bins, and bin cleaning. Existing residue detection methods include manual visual inspection and comparison detection based on specific signals. The disadvantages of manual visual inspection include: prolonged exposure to medical waste can harm worker health; it can cause visual fatigue; it is labor-intensive; and it can easily overlook tiny residues that are similar in color to the waste bin. Comparison detection based on specific signals involves emitting infrared light or sound waves into the waste bin, collecting the reflected light and sound signals, and comparing them with a pre-set residue signal template to determine the presence of residue. However, waste bins are prone to aging and soiling, and the size and type of residues inside vary, leading to lower accuracy, efficiency, and robustness. This method may also fail if the waste bin is misaligned. Machine vision has been applied in many inspection fields. Combining machine vision technology with unmanned medical waste disposal can more intelligently, safely and accurately determine the presence of residues in the trash can, thus ensuring the safe disposal of medical waste. Summary of the Invention

[0003] The purpose of this invention is to acquire, process, and analyze depth, light intensity, and color image data of the inside of a trash can collected by a TOF (Time of Flight) depth camera and a color camera, use image processing algorithms to extract trash can features, perform feature information matching in a database, combine the matching results to accurately segment the bottom of the trash can, and determine the presence of residue inside the trash can based on the depth, color, and light intensity images of the bottom of the can.

[0004] To achieve the above objectives, the present invention provides a method for detecting residues in medical waste bins based on multidimensional machine vision as follows:

[0005] This machine vision-based method for detecting residues in medical waste bins is characterized by three main steps: matching the overall three-dimensional feature information of the bin, accurately segmenting the bottom of the bin, and determining the presence of residues. Specifically, it includes the following steps:

[0006] S1: Establish a database of overall three-dimensional feature information of trash cans, and add the overall three-dimensional feature information of trash cans of different specifications to the database;

[0007] S2: Image acquisition inside the trash can. A TOF depth camera and a color camera are fixed directly above the trash can to acquire depth images, light intensity images, and color images inside the trash can. The 3D point cloud data of the trash can is calculated based on the depth images.

[0008] S3: Extract the three-dimensional feature information of the trash can opening plane and compare it with the overall three-dimensional feature information stored in the database to obtain the overall three-dimensional feature information of the trash can. Based on this information, the bottom of the trash can is segmented.

[0009] S4: Based on the depth image, light intensity image, and color image data of the bottom of the trash can, determine the presence of residue at the bottom of the trash can.

[0010] Preferably, in step S1, the overall three-dimensional feature information of the trash can includes: the feature spacing of the trash can opening plane, the trash can color information, the three-dimensional offset information of the bottom of the trash can relative to the straight line of the edge of the opening plane, and the length and width information of the bottom area; the feature spacing of the trash can opening plane and the trash can color information are used to match the trash can to be detected, and the three-dimensional offset information and the length and width information of the bottom area are used to segment the bottom of the trash can to be detected.

[0011] Preferably, in step S2, the image acquisition inside the trash can involves fixing a TOF depth camera and a color camera directly above the trash can. The TOF camera and the color camera are installed at the same horizontal height, and the height is such that the field of view of the TOF and color cameras completely covers the trash can to be detected. The optical axes of the two cameras are perpendicular to the ground plane.

[0012] Preferably, scene depth information and light intensity data are calculated based on the phase difference between emitted and reflected light waves. A multi-integral time dynamic fusion method is used to remove scene light intensity overexposure and invalid depth. A lookup table based on scene light intensity data is used to correct depth data errors. A guided filtering method using a color image as a guide image is used to recover missing depth data.

[0013] Preferably, based on the depth camera lens parameters, the scene depth data is transformed from the polar coordinate system to the camera point cloud coordinate system with the camera lens center point as the origin, and point cloud filtering is performed to remove discrete points: extract the bounded plane set of the scene 3D point cloud, for 3D points not in the plane set, retain the points whose projection onto the plane within the set falls within the plane boundary and whose distance from the plane meets the requirements, and for the remaining points, a radius-based clustering method is used to retain in-cluster points;

[0014] Preferably, step S3 extracts the three-dimensional feature information of the trash can opening plane and compares it with the overall three-dimensional feature information stored in the database to obtain the overall three-dimensional feature information of the trash can. Based on this information, the bottom of the trash can is segmented, specifically including the following steps:

[0015] S31: Detect scene point cloud planes within the specified ROI (RegionOfInterest) range and distance, and obtain the point cloud plane sequence [plane1, plane2, ..., plane...]. N Generally, a subplane n = [A,B,C,D,{(x1,y1),(x2,y2),…},Count], where A,B,C,D are the coefficients of the point cloud plane equation Ax+By+Cz+D=0, {(x1,y1),(x2,y2),…} represent the pixel coordinates of all points in the point cloud plane, and Count represents the total number of points in the point cloud plane;

[0016] S32: If the number of point cloud planes in the point cloud plane sequence obtained in step S31 is 0, that is, no plane that meets the requirements is detected, then the detection process ends; otherwise, plane filtering is performed. If the number of sub-planes in the point cloud plane sequence after filtering is 0, then the detection process ends.

[0017] S33: For the planar sequence after plane filtering in step S32, calculate the width of the outer left and right edges and the distance between the inner and outer bottom edges of each sub-plane, extract the average color value of all pixels in the point cloud plane, and obtain the planar feature information sequence [param1, param2, ..., param...]. n ], n = 1...P, where P represents the P sub-planes retained after filtering, where param i Each sub-feature is compared with the overall three-dimensional feature information in the overall three-dimensional feature information database of the trash can, and the matching degree is calculated. The sub-plane and the overall three-dimensional feature information with the highest matching degree score are the point cloud plane and the overall three-dimensional feature information of the trash can opening to be detected.

[0018] S34: Based on the point cloud plane of the opening of the trash can to be detected and the overall three-dimensional feature information described in step S33, calculate the three-dimensional vertex coordinates of the quadrilateral region at the bottom of the trash can to be detected [{X LT Y LT Z LT};{X LD Y LD Z LD};{X RT Y RT Z RT};{X RD YRD Z RD}], transforming the 3D coordinates to the camera pixel plane, yields the 2D pixel coordinates of the vertices of the quadrilateral region at the bottom of the trash can [{PX LT PY LT};{PX LD PY LD};{PX RT PY RT};{PX RD PY RD}).

[0019] Preferably, step S4 determines the presence of residue at the bottom of the trash can based on the depth image, light intensity image, and color image data of the bottom of the trash can, including the following steps:

[0020] S41: Based on the two-dimensional pixel coordinates of the quadrilateral region at the bottom of the trash can described in step S34, generate a binary mask image MaskImg. The pixel value in the mask image located within the quadrilateral region at the bottom is 255, otherwise it is 0.

[0021] S42: Calculate the distance between all three-dimensional points within the quadrilateral region at the bottom of the trash can and the point cloud plane at the mouth of the trash can to be detected as described in step S33, using the following formula:

[0022]

[0023] In the formula, i and j represent the x and y coordinates of the mask image, respectively, and A, B, C, and D are the coefficients of the plane equation of the point cloud of the trash can opening to be detected. ij Y ij Z ij} represents the 3D coordinates of pixel (i, j);

[0024] S43: The distance D mentioned in step S42 ij The comparison result image CompareImg is obtained by comparing the image with the actual height of the trash can. The pixel value of this image is determined by the following formula:

[0025]

[0026] Where height is the actual height of the trash can to be detected, σ is the allowable height deviation, the outer contour is extracted from the comparison result image, contour filtering is performed, and all contours that meet the requirements of area, perimeter and area-perimeter ratio are retained to obtain the depth image result contour sequence.

[0027] S44: Enhance the color image ColorImg of the scene to be detected to obtain the enhanced image EnhanceImg, which effectively avoids the influence of changes in external lighting environment. The enhancement formula is as follows:

[0028]

[0029] In the formula, avg is the grayscale mean value of the corresponding bottom area of ​​the trash can in the color image. ContrastImg is obtained by enhancing the contrast of EnhanceImg, as shown in the following formula:

[0030]

[0031]

[0032] Where b is the brightness coefficient, with a value range of [-1, 1], k is the contrast enhancement coefficient, and c is the contrast ratio, with a value range of [-1, 1].

[0033] S45: Perform color clustering on the bottom area of ​​the trash can corresponding to the contrast enhancement image ContrastImg described in step S44. First, perform two-class clustering, comparing the color of each class with the color in the overall three-dimensional feature information of the trash can to be detected described in step S33. For categories with large color differences, perform multi-class color clustering, comparing the color of each subclass with the color of the trash can to be detected. Extract the outer contour of the categories with large color differences, perform contour filtering, and retain all contours that meet the requirements for area, perimeter, and area-to-perimeter ratio to obtain the color image result contour sequence. The color comparison formula is as follows:

[0034] max(|R n -R|,|G n -G|,|B n -B|)>threshold

[0035] In the above formula, R n G n B n R, G, and B are the color values ​​of the subclasses after clustering, respectively; and threshold is the color contrast threshold.

[0036] S46: Based on the depth and color image result contour sequences described in steps S43 and S45, if the number of contours in the contour sequence is zero, then there is no residue at the bottom of the trash can to be detected; otherwise, a secondary judgment is made in conjunction with the light intensity image: For the depth and color image result contour sequences, the overlap degree of the sub-contours between the two contour sequences is calculated. For the sub-contours of the depth image result contour sequence and the sub-contours of the color image result contour sequence that meet the overlap degree requirement, the intersection is taken, and the contour intersection is added to the residue detection result contour sequence. For the sub-contours that do not meet the overlap degree requirement, the average light intensity within the contour area is calculated using the following formula:

[0037]

[0038] Where i and j are the pixel coordinates within the contour, k is the weighting factor, and u0 and v0 are lens parameters. The centroid coordinates of the contour region are extracted, and a circle is extracted with a radius d equal to the pixel distance d from the centroid to u0 and v0. The average light intensity Avg of all pixels on the circle is calculated. threshold ,like or The contour is then added to the residue detection result contour sequence, where β is the offset factor; all contours are marked on the scene color map and the detection results are displayed through the detection software interface.

[0039] This invention proposes a method for detecting residues in medical waste bins based on multidimensional machine vision. Its advantages include: an expandable database of bin specifications, facilitating future functional expansion; rapid localization and accurate segmentation of the bottom detection area for bins with different orientations; fusion of depth, color, and light intensity images to detect residues inside the bin; high accuracy, speed, and adaptability; and a visual interface for real-time display of detection results. This invention is a crucial component of unmanned medical waste disposal, enabling automatic and efficient detection of residues inside bins, effectively avoiding the negative impacts of manual visual inspection and overcoming the shortcomings of existing methods. Attached Figure Description

[0040] Figure 1 This is a flowchart of the method for detecting residues in medical waste bins based on multidimensional machine vision, according to the present invention.

[0041] Figure 2 This is a structural information diagram of the trash can to be tested according to an embodiment of the present invention.

[0042] Figure 3 This is a schematic diagram of the installation of the depth camera and color camera of the present invention. Detailed Implementation

[0043] To more clearly describe the technical content of the present invention, further detailed description is provided below in conjunction with embodiments.

[0044] This invention provides a method for detecting residues in medical waste bins based on multidimensional machine vision, such as... Figure 1 As shown, the specific steps are as follows:

[0045] S1: Establish a database of overall three-dimensional feature information for trash cans, adding the overall three-dimensional feature information of trash cans of different specifications to the database. The overall three-dimensional feature information includes the feature spacing of the trash can opening plane, the trash can color information, the three-dimensional offset information of the bottom relative to the straight line of the opening plane edge, and the length and width information of the bottom area. The feature spacing of the trash can opening includes: the straight line distance between the left and right edges of the outer side of the opening plane... Figure 2The distance between straight lines 1 and 2, and the distance between the lower edges of the inner and outer sides of the barrel opening plane are... Figure 2 The spacing between lines 3 and 4 in the center; the color information of the trash can is the average color value of the can opening plane; the three-dimensional offset information of the bottom of the can relative to the edge line of the can opening plane includes: the relative offset between the left edge line of the outer side of the can opening plane and the left edge line of the can bottom plane. Figure 2 The offset between lines 1 and 5, and the relative offset between the right side of the outer edge of the bucket opening plane and the right side of the bucket bottom plane. Figure 2 The offset between lines 2 and 6, and the relative offset between the lower edge of the outer side of the barrel opening plane and the lower edge of the barrel bottom plane are... Figure 2 The offset between lines 3 and 7 in the diagram; the length and width information of the bin bottom region are the length and width of the quadrilateral area of ​​the bin bottom. The three-dimensional offset information and the length and width information of the bin bottom region are used to segment the bottom of the trash can to be detected. The overall three-dimensional feature information is described as follows:

[0046] template k =[fw k fh k R k G k B k leftOffset k ,rightOffset k downOffset k L k W k ]

[0047] k = 1...m

[0048] Where k represents the k-th 3D specification information in the database, and there are m 3D specification information in the database; fw k fh k R is the feature spacing of the barrel opening plane. k G k B k For the color information of the trash can, leftOffset k ,rightOffset k downOffset k L represents the three-dimensional offset information of the bottom of the bucket relative to the straight line of the edge of the bucket opening plane. k W k This refers to the length and width information of the bottom area of ​​the bucket.

[0049] S2: Image acquisition inside the trash can. A TOF depth camera and a color camera are fixed directly above the trash can, such as... Figure 3As shown, the TOF camera and the color camera are mounted at the same horizontal height, ensuring that the field of view of both cameras completely covers the trash can being inspected. The camera's optical axis is perpendicular to the ground plane. The scene depth information and reflected light intensity are calculated based on the phase difference between the emitted and reflected light waves, using the following formula:

[0050]

[0051]

[0052] Where DCS0, DCS1, DCS2, and DCS3 are the signal values ​​acquired sequentially by the depth camera sensors at 90° phase delays, c is the constant speed of light, and f is the modulation frequency of the emitted light wave. A multi-integral time dynamic fusion method is used to remove overexposure of scene reflected light intensity and invalid depth. Depth data error correction is based on a lookup table of scene light intensity data, as shown in the following formula:

[0053]

[0054]

[0055] In the formula, Depth ij The values ​​represent the corrected depth values. DLut is a two-dimensional depth correction table created by changing the light intensity and the actual distance. Let (p, Amp) be a point in the table. ij The depth value at ) DReal p To obtain the actual depth value corresponding to this point, a color-map-guided bilateral filtering method is used to recover the missing depth data. Based on the depth camera lens parameters, the scene depth data is transformed from polar coordinates to a camera point cloud coordinate system with the camera lens center point as the origin, as shown below. Figure 3 As shown in the coordinate system, and the coordinate system conforms to the right-hand rule, the formula is as follows:

[0056]

[0057]

[0058]

[0059] In the formula, u0, v0, f x f y For the intrinsic parameters of the TOF depth camera lens, {X ij Y ij Z ij} represents the coordinate values ​​in the point cloud coordinate system. Point cloud filtering is performed to remove discrete points: Extract the set of boundary planes of the scene's 3D point cloud. For 3D points that are not in the plane set, retain the points whose projection onto the planes within the set falls within the plane boundary and whose distance from the plane meets the requirements. For the remaining points, a radius-based clustering method is used to retain the points within the cluster.

[0060] S3: Extract the 3D feature information of the trash can opening plane and compare it with the overall 3D feature information stored in the database to obtain the overall 3D feature information of the trash can. Based on this information, the bottom of the trash can is segmented. This includes the following steps:

[0061] S31: Detect scene point cloud planes within a specified ROI range and distance, and obtain the point cloud plane sequence [plane1, plane2, ..., plane...]. N Generally, a subplane n = [A, B, C, D, {(x1, y1), (x2, y2), ...}, Count], where A, B, C, and D are the coefficients of the point cloud plane equation Ax + By + Cz + D = 0, {(x1, y1), (x2, y2), ...} represent the pixel coordinates of all points in the point cloud plane, and Count represents the total number of points in the point cloud plane;

[0062] S32: If the number of point cloud planes in the point cloud plane sequence obtained in step S31 is 0, that is, no plane meeting the requirements has been detected, then the detection process ends; otherwise, plane filtering is performed: for each sub-plane in the plane sequence... n If the plane parameters of the point cloud meet the following requirements, the plane will be retained:

[0063]

[0064] Where θ TH The threshold for the angle between the subplane normal vector and the Z-axis of the camera point cloud coordinate system, Count TH This represents the threshold for the number of points in the point cloud plane. If the number of sub-planes in the filtered point cloud plane sequence is 0, then the detection process ends.

[0065] S33: For the planar sequence after plane filtering in step S32, calculate the width of the outer left and right edges, the distance between the inner and outer bottom edges, and the average color value of all pixels in the point cloud plane for each sub-plane, to obtain the planar feature information sequence [param1, param2, ..., param...]. n ...]n=1...P, where P represents the P sub-planes retained after filtering, and the sub-feature information param n =[fw n fh nR n G n B n Each sub-feature information is compared with the overall three-dimensional feature information in the overall three-dimensional feature information database of the trash can, and the matching degree is calculated. The matching degree measurement formula is as follows:

[0066]

[0067] In the formula, S nk The matching score is the subplane and the overall 3D feature information with the highest matching score, which are the point cloud plane and the overall 3D feature information of the garbage can opening to be detected.

[0068] S34: Based on the point cloud plane of the opening of the trash can to be detected and the overall three-dimensional feature information described in step S33, calculate the three-dimensional vertex coordinates of the quadrilateral region at the bottom of the trash can to be detected [{X LT Y LT Z LT};{X LD Y LD Z LD};{X RT Y RT Z RT};

[0069] {X RD Y RD Z RD}], transforming the three-dimensional coordinates to the camera pixel plane, yields the two-dimensional pixel coordinates of the quadrilateral region at the bottom of the trash can [{PX LT PY LT};{PX LD PY LD};{PX RT PY RT};{PX RD PY RD The conversion formula is as follows:

[0070]

[0071] In the formula, u, v, f x f y Here are the intrinsic parameters of the TOF depth camera lens, {X, Y, Z} is a point in the point cloud coordinate system, and {PX, PY} are the pixel coordinates in the corresponding pixel coordinate system.

[0072] S4: Based on the depth image, light intensity image, and color image data of the bottom of the trash can, determine the presence of residue at the bottom of the trash can, including the following steps:

[0073] S41: Based on the two-dimensional pixel coordinates of the quadrilateral region at the bottom of the trash can described in step S34, generate a binary mask image MaskImg. The pixel value in the mask image located within the quadrilateral region at the bottom is 255, otherwise it is 0.

[0074] S42: Calculate the distance between all three-dimensional points within the quadrilateral region at the bottom of the trash can and the point cloud plane at the mouth of the trash can to be detected as described in step S33, using the following formula:

[0075]

[0076] In the formula, i and j represent the x and y coordinates of the mask image, respectively, and A, B, C, and D are the coefficients of the plane equation of the point cloud of the trash can opening to be detected. ij Y ij Z ij} represents the 3D coordinates of pixel (i, j);

[0077] S43: The distance D mentioned in step S42 ij The comparison result image CompareImg is obtained by comparing the image with the actual height of the trash can. The pixel value of this image is determined by the following formula:

[0078]

[0079] Where height is the actual height of the trash can to be detected, such as... Figure 2 The vertical distance between the bucket mouth plane and the bucket bottom plane is 8, and σ is the allowable height deviation. The outer contour is extracted from the comparison result map, and contour filtering is performed to retain all contours that meet the requirements in terms of area, perimeter, and area-to-perimeter ratio, thus obtaining the depth map result contour sequence.

[0080] S44: Enhance the color image ColorImg of the scene to be detected to obtain the enhanced image EnhanceImg, which effectively avoids the influence of changes in external lighting environment. The enhancement formula is as follows:

[0081]

[0082] In the formula, avg is the grayscale mean value of the corresponding bottom area of ​​the trash can in the color image. ContrastImg is obtained by enhancing the contrast of EnhanceImg, as shown in the following formula:

[0083]

[0084]

[0085] Where b is the brightness coefficient, with a value range of [-1, 1], k is the contrast enhancement coefficient, and c is the contrast ratio, with a value range of [-1, 1].

[0086] S45: Perform color clustering on the bottom area of ​​the trash can corresponding to the contrast enhancement image ContrastImg described in step S44. First, perform two-class clustering, comparing the color of each class with the color in the trash can specification information described in step S33. For categories with large color differences, perform multi-class color clustering, comparing the color of each subclass with the color of the trash can to be detected. Extract the outer contour of the categories with large color differences, perform contour filtering, and retain all contours that meet the requirements for area, perimeter, and area-to-perimeter ratio to obtain the color image result contour sequence. The color comparison formula is as follows:

[0087] max(|R n -R|,|G n -G|,|B n -B|)>threshold

[0088] In the above formula, R n G n B n R, G, and B are the color values ​​of the subclasses after clustering, R, G, and B are the color values ​​of the trash cans to be detected, and threshold is the color contrast threshold.

[0089] S46: Based on the depth and color image result contour sequences described in steps S43 and S45, if the number of contours in the contour sequence is zero, then there is no residue at the bottom of the trash can to be detected; otherwise, a secondary judgment is made in conjunction with the light intensity image: For the depth and color image result contour sequences, the overlap degree of the sub-contours between the two contour sequences is calculated. For the sub-contours of the depth image result contour sequence and the sub-contours of the color image result contour sequence that meet the overlap degree requirement, the intersection is taken, and the contour intersection is added to the residue detection result contour sequence. For the sub-contours that do not meet the overlap degree requirement, the average light intensity within the contour area is calculated using the following formula:

[0090]

[0091] Where i and j are the pixel coordinates within the contour, k is the weighting factor, and u0 and v0 are lens parameters. The centroid coordinates of the contour region are extracted, and a circle is extracted with a radius d equal to the pixel distance d from the centroid to u0 and v0. The average light intensity Avg of all pixels on the circle is calculated. threskold ,like or The contour is then added to the residue detection result contour sequence, where β is the offset factor; all contours are marked on the scene color map and the detection results are displayed through the detection software interface.

[0092] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the scope of the invention. It should be noted that various modifications and improvements made to the technical solutions of the present invention by those skilled in the art without departing from the spirit of the invention should be covered within the protection scope of the present invention.

Claims

1. A method for detecting residual waste in a medical waste bin based on multi-dimensional machine vision, characterized in that, Includes the following steps: S1: Establish a database of overall three-dimensional feature information of trash cans, and add the overall three-dimensional feature information of trash cans of different specifications to the database; S2: Image data acquisition inside the trash can. A TOF depth camera and a color camera are fixed directly above the trash can to collect depth images, light intensity images, and color images inside the trash can. The 3D point cloud data of the trash can is calculated based on the depth images. S3: Extract the three-dimensional feature information of the trash can opening plane and compare it with the overall three-dimensional feature information of the trash can stored in the database to obtain the point cloud plane of the trash can opening and the overall three-dimensional feature information of the trash can. Based on this information, the vertex two-dimensional pixel coordinates of the quadrilateral region at the bottom of the trash can are segmented. S4: Based on the depth image, light intensity image, and color image data of the bottom of the trash can, determine the presence of residue at the bottom of the trash can; Step S4 specifically includes the following steps: S41: Generate a binarized mask image MaskImg based on the two-dimensional pixel coordinates of the quadrilateral region at the bottom of the trash can as described in step S3; S42: Calculate the distance between all three-dimensional points in the quadrilateral area at the bottom of the trash can and the point cloud plane at the mouth of the trash can to be detected as described in step S3; S43: Compare the distance described in step S42 with the actual height of the trash can to obtain a comparison result image CompareImg; S44: Enhance the color image ColorImg of the scene to be detected to obtain the enhancement result image EnhanceImg, and then enhance the contrast of EnhanceImg to obtain the contrast enhancement image ContrastImg. S45: Perform color clustering on the bottom area of ​​the trash can corresponding to the contrast enhancement image ContrastImg described in step S44, compare the color of each subclass with the color of the trash can to be detected, extract the outer contour of the category area with large color difference, perform contour filtering, retain all contours that meet the requirements of area, perimeter, and area-perimeter ratio, and obtain the color image result contour sequence. S46: Based on the depth and color image result contour sequences described in steps S43 and S45, if the number of contours in the contour sequence is zero, then there is no residue at the bottom of the trash can to be detected; otherwise, a secondary judgment is made in conjunction with the light intensity image: For the depth map and color image result contour sequences, the overlap degree of the sub-contours between the two contour sequences is calculated. For the sub-contours of the depth map result contour sequence and the sub-contours of the color image result contour sequence that meet the overlap degree requirement, the intersection is taken, and the contour intersection is added to the residue detection result contour sequence. For the sub-contours that do not meet the overlap degree requirement, the average light intensity within the contour area is calculated. All contours are marked on the scene color image and the detection results are displayed through the detection software interface.

2. The method for detecting residues in medical waste bins based on multidimensional machine vision according to claim 1, characterized in that, In step S1, the overall three-dimensional feature information database of the trash can includes the feature spacing of the trash can opening plane, the color information of the trash can, the three-dimensional offset information of the bottom of the trash can relative to the straight line of the edge of the opening plane, and the length and width information of the bottom area. The database stores the overall three-dimensional feature information of trash cans of different specifications and types.

3. The method for detecting residues in medical waste bins based on multidimensional machine vision according to claim 1, characterized in that, In step S2, the TOF depth camera and the color camera are fixedly installed at the same horizontal height directly above the trash can, with the optical axes of the two cameras perpendicular to the ground and ensuring that the camera's field of view covers the trash can to be detected.

4. The method for detecting residues in medical waste bins based on multidimensional machine vision according to claim 1, characterized in that, In step S2, the depth information of the detected scene is calculated based on the phase difference between the emitted and reflected light waves. The multi-integral time dynamic fusion method is used to remove overexposure of the scene reflected light intensity and invalid depth. The depth data error is corrected based on the lookup table of the scene light intensity data. The missing depth data is restored using a guided filtering method with a color image as the guide image.

5. The method for detecting residues in medical waste bins based on multidimensional machine vision according to claim 1, characterized in that, In step S2, the scene depth data is transformed from the polar coordinate system to the camera point cloud coordinate system with the camera lens center point as the origin, and point cloud filtering is performed to remove discrete points: extract the boundary plane set of the scene 3D point cloud, for 3D points not in the plane set, retain the points whose projection points onto the plane within the set fall within the plane boundary and whose distance from the plane meets the requirements, and perform a radius-based clustering method to retain in-cluster points for the remaining points.

6. The method for detecting residues in medical waste bins based on multidimensional machine vision according to claim 1, characterized in that, Step S3 specifically includes the following steps: S31: Detect the scene point cloud planes within the specified ROI range and distance, and obtain the point cloud plane sequence. Generally, subplane ,in The plane equation of this point cloud coefficient, This represents the pixel coordinates of all points within the point cloud plane. This represents the total number of points within the point cloud plane. S32: If the number of point cloud planes in the point cloud plane sequence obtained in step S31 is 0, that is, no plane that meets the requirements is detected, then the detection process ends; otherwise, plane filtering is performed. If the number of sub-planes in the point cloud plane sequence after filtering is 0, then the detection process ends. S33: For the planar sequence after plane filtering in step S32, calculate the width of the outer left and right edges and the distance between the inner and outer bottom edges of each sub-plane, extract the average color value of all pixels in the point cloud plane, and obtain the planar feature information sequence. , This represents the P sub-planes retained after filtering, where, Each sub-feature is compared with the overall three-dimensional feature information in the overall three-dimensional feature information database of the trash can, and the matching degree is calculated. The sub-plane and the overall three-dimensional feature information with the highest matching degree score are the point cloud plane and the overall three-dimensional feature information of the trash can opening to be detected. S34: Based on the point cloud plane of the opening of the trash can to be detected and the overall three-dimensional feature information described in step S33, calculate the three-dimensional vertex coordinates of the quadrilateral region at the bottom of the trash can to be detected. ; The 3D coordinates are transformed to the camera pixel plane to obtain the 2D pixel coordinates of the vertices of the quadrilateral region at the bottom of the trash can. .

7. The method for detecting residues in medical waste bins based on multidimensional machine vision according to claim 1, characterized in that, Step S4 specifically includes the following steps: S41: Based on the two-dimensional pixel coordinates of the quadrilateral region at the bottom of the trash can described in step S34, generate a binary mask image MaskImg. The pixel value in the mask image located within the quadrilateral region at the bottom is 255, otherwise it is 0. S42: Calculate the distance between all three-dimensional points within the quadrilateral region at the bottom of the trash can and the point cloud plane at the mouth of the trash can to be detected as described in step S33, using the following formula: ; In the formula, These represent the x and y coordinates of the mask image, respectively. The coefficients of the plane equation for the point cloud at the mouth of the trash can to be tested are: For pixels The corresponding 3D point cloud coordinates; S43: The distance described in step S42 The comparison result image CompareImg is obtained by comparing the image with the actual height of the trash can. The pixel value of this image is determined by the following formula: ; in, The actual height of the trash can to be tested. To allow for height deviation, the outer contour is extracted from the comparison result map, contour filtering is performed, and all contours that meet the requirements in terms of area, perimeter, and area-to-perimeter ratio are retained to obtain the depth map result contour sequence. S44: Enhance the color image ColorImg of the scene to be detected to obtain the enhanced image EnhanceImg, which effectively avoids the influence of changes in external lighting environment. The enhancement formula is as follows: ; In the formula, Given the grayscale mean of the area at the bottom of the trash can in the color image, EnhanceImg is used to enhance contrast, resulting in the contrast-enhanced image ContrastImg, as shown in the following formula: ; ; in, b This is the brightness coefficient, with a value range of [value range missing]. , k This is the contrast enhancement factor. c For contrast, the value range is... ; S45: Perform color clustering on the bottom area of ​​the trash can corresponding to the contrast enhancement image ContrastImg described in step S44. First, perform two-class clustering, comparing the color of each class with the color in the trash can specification information described in step S33. For categories with large color differences, perform multi-class color clustering, comparing the color of each subclass with the color of the trash can to be detected. Extract the outer contour of the categories with large color differences, perform contour filtering, and retain all contours that meet the requirements for area, perimeter, and area-to-perimeter ratio to obtain the color image result contour sequence. The color comparison formula is as follows: ； In the above formula, The color values ​​of the subclasses after clustering. The color value of the trash can to be tested. This is the color contrast threshold; S46: Based on the depth and color image result contour sequences described in steps S43 and S45, if the number of contours in the contour sequence is zero, then there is no residue at the bottom of the trash can to be detected; otherwise, a secondary judgment is made in conjunction with the light intensity image: For the depth and color image result contour sequences, the overlap degree of the sub-contours between the two contour sequences is calculated. For the sub-contours of the depth image result contour sequence and the sub-contours of the color image result contour sequence that meet the overlap degree requirement, the intersection is taken, and the contour intersection is added to the residue detection result contour sequence. For the sub-contours that do not meet the overlap degree requirement, the average light intensity within the contour area is calculated using the following formula: ； in These are the pixel coordinates within the contour. k As a weighting factor, For lens parameters, extract the centroid coordinates of the contour region, and use the distance from the centroid to... pixel distance d Extract a circle with a radius and calculate the average light intensity of all pixels on the circle. ,like or Then, the contour is added to the contour sequence of the residue detection results, where β The offset factor is used; all contours are marked on the scene color map and the detection results are displayed through the detection software interface.

Images