A method for detecting width of reconstituted stem based on monocular vision
Through monocular visual detection method and perspective transformation technology, the problem of unstable width measurement under irregular shapes in the existing stalk width detection is solved, and the accurate measurement and reasonable mapping of stalk width are achieved.
Patent Information
- Application Number
- CN202110269906.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-03-12
- Publication Date
- 2024-11-29
- Estimated Expiration
- 2041-03-12
Smart Images

Figure CN112862815B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of reconstituted shredded stems.
[0002] In particular, the present invention relates to a method for detecting width of reconstituted stems based on monocular vision. Background Art
[0003] Reconstructed cut stems are made by extracting and separating the chemical components of tobacco stems, and then replacing them with the chemical components of tobacco leaves. This allows the cut stems to have the smoke characteristics of tobacco leaves while maintaining their original filling properties, thereby achieving the denaturation, color change, taste change, and deformation of the cut stems, improving the intrinsic quality of the cut stems, and broadening the addition ratio and application range in the cigarette product formula.
[0004] By scientifically processing the tobacco stems and improving the utilization rate of reconstituted stem shreds in cigarette formula, it is possible to reduce tar and harm, reduce costs, and at the same time help to broaden the scope of raw material use and highlight the brand style characteristics. There is a certain "tobacco stem deformation" in the pressing and cutting process of reconstituted tobacco stems. The tobacco stem deformation processing parameters mainly refer to the thickness of the pressed stems and the width of the cut shreds.
[0005] Now, the accurate measurement method of the width of the shredded tobacco is also receiving more and more attention; for example, the Chinese patent invention patent CN107798672A provides a method, system and server with the system for measuring the width of shredded tobacco or shredded tobacco, and the method for measuring the width of shredded tobacco or shredded tobacco includes: preprocessing the collected original image of shredded tobacco or shredded tobacco to generate a preprocessed image of shredded tobacco or shredded tobacco; the original image of shredded tobacco or shredded tobacco includes at least one shredded tobacco or shredded tobacco; morphological segmentation is performed on each shredded tobacco or shredded tobacco or shredded tobacco in the preprocessed image of shredded tobacco or shredded tobacco, and the morphologically segmented shredded tobacco or shredded tobacco image is subjected to straight line detection to detect the pixel width of each shredded tobacco or shredded tobacco; the pixel width of each shredded tobacco or shredded tobacco is converted into the predicted width of the shredded tobacco or shredded tobacco. The above invention avoids the problem of unstable centerline solution caused by local noise, provides a guarantee for the accuracy of the width of shredded tobacco or shredded tobacco, can quickly detect the actual average width of each shredded tobacco or shredded tobacco, and satisfies the requirement that the sample is fully representative.
[0006] However, the above-mentioned method for measuring the width of shredded stems still has the following disadvantages: 1. The preprocessing step in the shredded stem image processing is unreasonable, and the obtained image has large interference, resulting in low detection accuracy; 2. The shredded stems are curved and deformed, so the shape of the reconstructed shredded stems is irregular, and the pixel width is highly unstable when being obtained, and the error of using a straight line detection method is large; 3. The mapping between the pixel width and the actual width is different due to the different positions of the shredded stems, and the direct conversion between the shapes of the reconstructed shredded stems is unreasonable.
[0007] Therefore, in order to solve the above problems, it is necessary for us to design a reasonable method for detecting the width of reconstructed stems based on monocular vision. Summary of the invention
[0008] The object of the present invention is to provide a method for detecting the width of reconstructed stems based on monocular vision, which can extract the contour of reconstructed stems clearly and accurately, so that there is no interference when obtaining the pixel width, and can measure the pixel width of irregular stems. Moreover, through monocular vision calibration, a mapping between a reasonable pixel width and an actual width is obtained through perspective transformation, and the calculated stem width value is more accurate.
[0009] In order to achieve the above object, the present invention adopts the following technical solutions:
[0010] A method for detecting width of reconstructed stems based on monocular vision comprises the following steps:
[0011] S1: placing a number of reconstructed stems on white paper, taking a photo with a camera using the white paper as the background, graying and adaptively binarizing the obtained image, adjusting the background to black, and highlighting the area and shape of the reconstructed stems with white;
[0012] S2: extracting the contour of the area where each reconstructed stem is located, and saving the pixel points composed of each contour into a two-dimensional array;
[0013] S3: selecting any point in the obtained reconstructed stem region as the center of a circle, gradually increasing the radius, and obtaining the diameter of the largest inscribed circle as the pixel width of the reconstructed stem, and obtaining the average value of the pixel widths of all reconstructed stems as the actual pixel width of the reconstructed stem;
[0014] S4: placing a calibration plate at the location where the regenerated stem silk is photographed, photographing the calibration plate and converting it into grayscale, performing corner point detection, and saving the pixel coordinates of each corner point;
[0015] S5: Obtain the pixel distance between each adjacent corner point, read the specification data of the calibration plate, and obtain the mapping relationship between the pixel distance in the image and the actual distance;
[0016] S6: Calculate the actual width of the reconstructed shredded stem according to the mapping relationship and the actual pixel width of the reconstructed shredded stem.
[0017] As a preferred embodiment of the present invention, after executing step S1, the photographic data of the reconstructed shredded stems are preprocessed, and the preprocessing includes filtering processing and expansion processing.
[0018] As a preferred embodiment of the present invention, when executing step S2, the contour of the area where each reconstructed stem is located is extracted using the Find Contours function in OpenCV.
[0019] As a preferred embodiment of the present invention, when executing step S3, the following steps are specifically included:
[0020] S31: Select any point in the obtained regenerated stem region as the center and draw a circle;
[0021] S32: Increase the radius of the circle;
[0022] S33: determining whether a part of the circle is located in the blank area outside the reconstituted shredded stem area; if so, moving the center of the circle in a direction away from the outside of the reconstituted shredded stem area until all parts of the circle are located in the reconstituted shredded stem area, and executing step S34; otherwise, no operation is executed;
[0023] S34: Determine whether both sides of the circle are tangent to the contour line. If so, the diameter of the circle is the pixel width of the reconstructed stem; otherwise, return to step S32.
[0024] As a preferred embodiment of the present invention, before executing step S3, the minimum value of the gradually increasing radius of the circle and the minimum moving distance of the circle when moving are calibrated.
[0025] As a preferred embodiment of the present invention, before executing step S3, it is necessary to determine the circumscribed rectangle of the area of the reconstructed stems, and the drawn circle must be located within the circumscribed rectangle, thereby avoiding infinite expansion of the inscribed circle, improving measurement efficiency, and saving time costs.
[0026] As a preferred embodiment of the present invention, when executing step S3, a circle is drawn within the outline of each reconstructed stem to obtain the pixel width of each reconstructed stem, and the average value is taken as the actual pixel width of the reconstructed stem.
[0027] As a preferred embodiment of the present invention, when executing step S4, the calibration plate is a square with 8*8 black and white grids and 7*7 corner points, and the actual corner point distance between adjacent corner points is 20 mm.
[0028] As a preferred embodiment of the present invention, when executing step S5, the following steps are specifically included:
[0029] S51: taking a number of vertices of the calibration plate, and calculating the mapping transformation matrix M of the two vertices based on the pixel distance between any two vertices;
[0030] S52: Perform perspective transformation on the image of the calibration plate using the matrix M;
[0031] S53: performing corner point detection on the transformed image again, and storing the measured corner points in an array;
[0032] S54: Calculate the average pixel distance between new corner points;
[0033] S55: The actual corner point distance is divided by the average pixel distance to obtain the actual distance Q represented by the unit pixel.
[0034] As a preferred embodiment of the present invention, when executing step S6, the actual pixel width of the reconstructed shredded stem is obtained by multiplying the actual distance Q value represented by the unit pixel to calculate the actual width of the reconstructed shredded stem.
[0035] As a preferred embodiment of the present invention, when executing step S51, the four corner points of the upper left corner, lower left corner, upper right corner and lower right corner of the calibration plate are taken as vertices to form a quadrilateral, and the pixel distance of the connecting line between any two vertices is calculated.
[0036] The invention discloses a method for detecting the width of reconstructed stems based on monocular vision, which has the beneficial effects of: extracting the outline of reconstructed stems clearly and accurately, so that there is no interference when acquiring the pixel width, and the pixel width of irregular stems can be measured. Moreover, through monocular vision calibration, a mapping between a reasonable pixel width and an actual width is obtained through perspective transformation, and the calculated stem width value is more accurate. BRIEF DESCRIPTION OF THE DRAWINGS
[0037] Figure 1 The present invention is a schematic flow chart of a method for detecting width of reconstituted stems based on monocular vision. DETAILED DESCRIPTION
[0038] The following are specific embodiments of the present invention, which further describe the technical solutions of the present invention, but the present invention is not limited to these embodiments.
[0039] Various exemplary embodiments of the present invention will now be described in detail with reference to the accompanying drawings. It should be noted that unless otherwise specifically stated, the relative arrangement and steps of modules and steps set forth in these embodiments do not limit the scope of the present invention.
[0040] At the same time, it should be understood that for the convenience of description, the processes in the drawings are not just performed separately, but multiple steps are performed in an interlaced manner.
[0041] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the invention, its application, or uses.
[0042] Technologies and methods known to ordinary technicians in the relevant field may not be discussed in detail, but in appropriate circumstances, technologies and methods should be considered as part of the authorization specification.
[0043] Example: Figure 1 As shown, a method for detecting the width of reconstructed stems based on monocular vision comprises the following steps:
[0044] S1: placing a number of reconstructed stems on white paper, taking a photo with a camera using the white paper as the background, graying and adaptively binarizing the obtained image, adjusting the background to black, and highlighting the area and shape of the reconstructed stems with white;
[0045] S2: extracting the contour of the area where each reconstructed stem is located, and saving the pixel points composed of each contour into a two-dimensional array;
[0046] S3: Select any point in the obtained reconstructed stem area as the center of the circle, gradually increase the radius, and obtain the diameter of the largest inscribed circle, which is the pixel width of the reconstructed stem. Obtain the average of the pixel widths of all reconstructed stems and take it as the actual pixel width of the reconstructed stem;
[0047] S4: placing a calibration plate at the location where the regenerated stem silk is photographed, photographing the calibration plate and converting it into grayscale, performing corner point detection, and saving the pixel coordinates of each corner point;
[0048] S5: Obtain the pixel distance between each adjacent corner point, read the specification data of the calibration plate, and obtain the mapping relationship between the pixel distance in the image and the actual distance;
[0049] S6: Calculate the actual width of the reconstructed shredded stem according to the mapping relationship and the actual pixel width of the reconstructed shredded stem.
[0050] Steps S1 and S2 are for extracting the contour of the reconstructed stem; step S3 is for measuring the pixel width of the reconstructed stem; steps S4 and S5 are for mapping the pixel width to the actual width. The actual width of the reconstructed stem is calculated by combining the pixel width of the reconstructed stem and the mapping relationship from pixel to actual through the final step S6.
[0051] In step S1, a plurality of reconstructed stem shreds are placed on white paper using an image processing algorithm. The plurality of reconstructed stem shreds are of the same type, and photographed with the white paper as the background. The obtained image is subjected to grayscale processing and adaptive binarization processing, and the background is blackened, and the area and shape of the reconstructed stem shreds are highlighted in white. After executing step S1, the photographed data of the reconstructed stem shreds are preprocessed, and the preprocessing includes filtering processing and expansion processing, and the order of filtering and expansion can be selected according to the environment of the photographing and the specific noise situation. The filtering algorithm is used to remove the salt and pepper noise, small white spots and holes on the image during the shooting, so as to reduce the interference in the subsequent detection of the contour of the reconstructed stem shreds. The expansion algorithm has two functions: first, it enhances the connectivity of each reconstructed stem shred, and ensures as much as possible that the same reconstructed stem shred is detected as one contour when the contour of the reconstructed stem shreds is detected later, so as to reduce the situation where one reconstructed stem shred is distinguished as multiple contours. Second, it is to prevent the pixels belonging to the reconstructed stems from being filtered out as noise when filtering, because some of the pixels are misjudged as background, resulting in a high degree of distribution of points, leading to missed detection or only partial detection. These two steps can be selected according to specific environmental factors or shooting conditions. If the filtering process is a single point-shaped salt and pepper noise, it can be simply completed by using only median filtering.
[0052] When executing step S2, after the preprocessing is completed, the obtained image has basically separated the background, noise and reconstructed stems. At this time, the Find Contours function in OpenCV is used to extract the contour of the area where each reconstructed stem is located, and the pixel points composed of each contour are saved in a two-dimensional array for subsequent width measurement.
[0053] In addition, when executing step S3, the following steps are specifically included:
[0054] S31: Select any point in the obtained regenerated stem region as the center and draw a circle;
[0055] S32: Increase the radius of the circle;
[0056] S33: determining whether a part of the circle is located in the blank area outside the reconstituted shredded stem area; if so, moving the center of the circle in a direction away from the outside of the reconstituted shredded stem area until all parts of the circle are located in the reconstituted shredded stem area, and executing step S34; otherwise, no operation is executed;
[0057] S34: Determine whether both sides of the circle are tangent to the contour line. If so, the diameter of the circle is the width of the reconstructed stems; otherwise, return to step S32.
[0058] Of course, before executing step S3, the minimum value of the gradually increasing radius of the circle and the minimum moving distance of the circle when moving are calibrated. Moreover, when executing step S3, a circle is drawn once within the outline of each reconstructed stem, and the pixel width of each reconstructed stem is obtained, and the average value is taken as the actual pixel width of the reconstructed stem.
[0059] Also, before executing step S3, the circumscribed rectangle of the area of the regenerated stems needs to be determined, and the drawn circle must be located within the circumscribed rectangle, thereby avoiding the infinite expansion of the inscribed circle, improving the measurement efficiency, saving time cost, and the processing speed is 0.8 seconds / frame.
[0060] In fact, the shape of the reconstituted stems is irregular, so the irregular shape width measurement method is used as the basis. Select any point in the obtained reconstituted stem area (a set of contour pixel points saved in step S2), draw a circle with the point, and gradually increase the radius (the step size can be set to 0.1 pixel for accuracy). Ensure that the range in the circle is all within the reconstituted stem area. If it does not meet the requirements, move the center of the circle away from the outside of the reconstituted stem area until the range in the circle is all within the reconstituted stem area (set the minimum moving range); in this way, the maximum inscribed circle is obtained in the width direction, and the tobacco width is the diameter of the circle; the same method is used to draw a circle once in the area of multiple reconstituted stems (and move to the maximum inscribed circle), respectively detect the pixel widths of multiple groups of tobacco stems, and finally calculate the average value as the actual pixel distance of this batch of reconstituted stems.
[0061] In addition, when executing step S4, while ensuring that the camera position for taking the reconstructed stem silk picture remains fixed, take a picture of a calibration plate, the calibration plate is square, has 8*8 black and white grids, has 7*7 corner points, and the actual corner point distance between adjacent corner points is 20 mm.
[0062] Furthermore, when executing step S5, the following steps are specifically included:
[0063] S51: taking a number of vertices of the calibration plate, and calculating the mapping transformation matrix M of the two vertices based on the pixel distance between any two vertices;
[0064] S52: Perform perspective transformation on the image of the calibration plate using the matrix M;
[0065] S53: performing corner point detection on the transformed image again, and storing the measured corner points in an array;
[0066] S54: Calculate the average pixel distance between new corner points;
[0067] S55: The actual corner point distance is divided by the average pixel distance to obtain the actual distance Q represented by the unit pixel.
[0068] Here, when executing step S51, the four corner points of the upper left corner, lower left corner, upper right corner and lower right corner of the calibration plate are taken as vertices to form a quadrilateral, and the pixel distance of the connecting line between any two vertices is calculated.
[0069] In fact, the calibration plate is grayed out before corner detection, and the pixel coordinates of each corner are saved. The pixel distances of adjacent corners are calculated respectively, and the average value is taken as the final side length, that is, the actual pixel distance of adjacent corners.
[0070] Using perspective transformation, select the upper left corner, lower left corner, upper right corner, and lower right corner of the calibration plate as the base points before transformation. Take the upper left corner as the starting point, calculate the total side length between the upper left corner vertex and the upper right corner vertex, and the total side length between the upper left corner vertex and the lower left corner vertex, and obtain the mapping relationship between two sets of adjacent vertices to generate the mapping transformation matrix M. Then use the matrix M as a parameter to perform perspective transformation on the image of the calibration plate to adjust and compensate for the image distortion caused by shooting. Try to ensure that the original square calibration plate is also a square in the image, so as to ensure that the pixel distance to the mapping distance in the horizontal and vertical directions can be consistent. Then perform corner point detection on the transformed image again, and calculate the average pixel distance of adjacent corner points again. Then divide the actual distance (20mm) by the pixel distance to get the mapping relationship of the actual distance length represented by each pixel. A large number of samples are collected by simulating various types of distortion, and the calibration plate is used to determine the mapping relationship between the pixel distance in the image and the actual distance.
[0071] Finally, when executing step S6, the actual pixel width of the reconstructed shredded stem is obtained by multiplying the actual distance Q value represented by the unit pixel to calculate the actual width of the reconstructed shredded stem.
[0072] It should be noted that, in the present invention, the shredded stem width detection can also be performed in the order of steps S4, S5, S1, S2, S3, and S6, that is, before extracting the reconstructed shredded stem contour, a mapping from pixel width to actual width is first obtained, and then the pixel width measurement of the reconstructed shredded stem contour is obtained, and the actual width of the reconstructed shredded stem is calculated based on the mapping from pixel width to actual width obtained earlier.
[0073] At this time, when detecting the width of the reconstructed stems, the image must also be perspective transformed according to the matrix M, and then image processing and width measurement are performed.
[0074] The invention discloses a reconstructed stem width detection method based on monocular vision, in which the contour of the reconstructed stem is extracted clearly and accurately, so that there is no interference when the pixel width is obtained, and the pixel width of irregular stems can be measured. Moreover, through the monocular vision calibration method, a mapping between a reasonable pixel width and an actual width is obtained through perspective transformation, and the calculated stem width value is more accurate.
[0075] The present invention is not limited to the above specific implementations, and the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made to the above implementations based on the technical essence of the present invention shall be included in the protection scope of the present invention.
Claims
1. A method for detecting the width of reconstructed stems based on monocular vision, characterized in that: The following steps are involved: S1: placing a number of reconstructed stems on white paper, taking a photo with a camera using the white paper as the background, graying and adaptively binarizing the obtained image, adjusting the background to black, and highlighting the area and shape of the reconstructed stems with white; After executing step S1, the photographic data of the reconstructed shredded stems are preprocessed, the preprocessing includes filtering processing and expansion processing, and the order of filtering processing and expansion processing is selected according to the photographing environment and the specific noise situation; S2: extracting the contour of the area where each reconstructed stem is located, and saving the pixel points composed of each contour into a two-dimensional array; S3: Select any point in the obtained reconstructed stem region as the center of the circle, gradually increase the radius, and obtain the diameter of the largest inscribed circle, which is the pixel width of the reconstructed stem. Obtain the average value of the pixel widths of all reconstructed stems and use it as the actual pixel width of the reconstructed stem; S4: while ensuring that the position of the camera for taking the reconstructed stem shreds picture remains fixed, a calibration plate is placed at the position where the reconstructed stem shreds are taken, the calibration plate is photographed and grayscaled, corner point detection is performed, and the pixel coordinates of each corner point are saved; S5: Obtain the pixel distance between each adjacent corner point, read the specification data of the calibration plate, and obtain the mapping relationship between the pixel distance in the image and the actual distance; When executing step S5, the following steps are specifically included: S51: Take the four corner points of the upper left corner, lower left corner, upper right corner and lower right corner of the calibration plate as vertices to form a quadrilateral, and calculate the total side length between the vertex of the upper left corner and the vertex of the upper right corner, and the total side length between the vertex of the upper left corner and the vertex of the lower left corner according to the pixel distance between any two vertices, obtain the mapping relationship between two groups of adjacent vertices, and generate the mapping transformation matrix M; S52: Performing perspective transformation on the image of the calibration plate using the matrix M to adjust and compensate for image distortion caused by shooting; S53: performing corner point detection on the transformed image again, and storing the measured corner points in an array; S54: Calculate the average pixel distance between new corner points; S55: dividing the actual corner point distance by the average pixel distance to obtain the actual distance Q represented by the unit pixel; S6: Calculate the actual width of the reconstructed shredded stem according to the mapping relationship and the actual pixel width of the reconstructed shredded stem.
2. The method for detecting width of reconstituted stems based on monocular vision according to claim 1, characterized in that: When executing step S2, the contour of the area where each reconstructed stem strand is located is extracted using the Find Contours function in OpenCV.
3. The method for detecting width of reconstituted stems based on monocular vision according to claim 1, characterized in that: When executing step S3, the following steps are specifically included: S31: Select any point in the obtained regenerated stem region as the center and draw a circle; S32: Increase the radius of the circle; S33: determining whether a part of the circle is located in the blank area outside the reconstituted shredded stem area; if so, moving the center of the circle in a direction away from the outside of the reconstituted shredded stem area until all parts of the circle are located in the reconstituted shredded stem area, and executing step S34; otherwise, no operation is executed; S34: Determine whether both sides of the circle are tangent to the contour line. If so, the diameter of the circle is the pixel width of the reconstructed stem; otherwise, return to step S32.
4. The method for detecting width of reconstituted stems based on monocular vision according to claim 1, characterized in that: Before executing step S3, the minimum value of the gradually increasing radius of the circle and the minimum moving distance of the circle when moving are calibrated.
5. The method for detecting width of reconstituted stems based on monocular vision according to claim 1, characterized in that: When executing step S3, a circle is drawn within the outline of each reconstructed shredded stem to obtain the pixel width of each reconstructed shredded stem, and the average value is taken as the actual pixel width of the reconstructed shredded stem.
6. The method for detecting width of reconstituted stems based on monocular vision according to claim 1, characterized in that: When executing step S4, the calibration plate is a square with 8*8 black and white grids and 7*7 corner points, and the actual corner point distance between adjacent corner points is 20 mm.
7. The method for detecting width of reconstituted stems based on monocular vision according to claim 6, characterized in that: When executing step S6, the actual pixel width of the reconstructed shredded stem is obtained by multiplying the actual distance Q value represented by the unit pixel to calculate the actual width of the reconstructed shredded stem.
Citation Information
Patent Citations
Tobacco or stem width measuring method, system and server with same
CN107798672A
Online self-adaptive cut tobacco width measuring method based on machine vision
CN111060014A
Computer vision tobacco shred width detection method based on reducing circle
CN112330663A
System and method of measuring distances related to an object
US20140314276A1
