Machine vision-based head rice rate online detection method and system

Abstract

The invention relates to the technical field of rice quality detection, in particular to a head rice rate online detection method and system based on machine vision, and the method comprises the steps: obtaining an initial binary image of a rice image; intelligently screening a rice grain adhesion region in the initial binarized image by using a BP neural network to obtain a to-be-segmented binarized image; segmenting each rice grain adhesion region in the to-be-segmented binary image by using a multi-distance transformation watershed segmentation method to obtain a first segmentation result image; segmenting a rice grain adhesion area in the first segmentation result image by using an angle limitation concave point matching segmentation method to obtain a second segmentation result image as a final segmentation result image; and obtaining the length of each rice grain in the rice image on the basis of the obtained segmentation result image by using a minimum bounding rectangle method so as to calculate the head rice rate of the rice image. According to the method, the segmentation accuracy is improved, the segmentation method is high in applicability, and subsequent measurement of characteristic parameters of single-grain rice is facilitated.

Description

Technical Field

[0001] The present invention relates to the technical field of rice quality detection, and in particular to an online detection method and system for whole polished rice rate based on machine vision. Background Art

[0002] How to reduce broken rice in the rice milling process is the key to saving grain and reducing losses for grain enterprises. The polished rice rate is an indicator of the quality of rice processing, the most concerned issue for merchants in rice trade, and is closely related to the economic benefits of rice mills.

[0003] At present, the detection of the whole polished rice rate in rice processing is done offline, sampling from the processing line and then manually testing; this traditional whole rice and broken rice detection method is a combination of mechanical screening and manual selection to sort out the whole rice and broken rice in the sample, and weigh the mass to calculate the content of whole rice and broken rice. However, there are a series of limitations in the traditional manual visual inspection and mechanical screening methods for the detection of the whole polished rice rate of rice; although manual visual inspection is not affected by light and adhesion, human eye recognition is subjective and requires a lot of human resources; mechanical screening is to grade and screen rice through machines equipped with screens of different specifications. This method will not only cause the screen to wear on the rice during the screening process, but also lead to high cost consumption of the screen for a long time; therefore, the method of relying on manpower and traditional machines to detect rice quality is far from meeting the existing needs, and there are problems of low efficiency, high cost, and serious waste of rice.

[0004] With the rapid development of information technology and the widespread application of computers, image analysis technology has been applied to multiple disciplines and has made many breakthroughs in the field of crops. The use of computer image processing technology to detect and analyze whole polished rice and broken rice has the advantages of being fast, convenient, objective, and accurate. It not only saves time and effort, but also can accurately identify the appearance characteristics of rice, avoid the subjectivity of manual methods, and persist the monitoring data for future query and retrieval. It has great practical significance for replacing traditional detection methods. However, in the process of detecting batch-collected rice images, the randomness of the rice grain position will cause the rice grains to overlap or stick together, making it difficult to identify a single whole rice grain, resulting in errors in the subsequent calculation of the number of rice grains and the whole polished rice rate index.

[0005] In order to solve this problem, some detection devices have adopted the method of manually spreading the rice on the placement table in advance, or only conveying a single grain of rice for image scanning of the rice appearance characteristics; although this method can quickly extract the characteristic parameters of each grain of rice, thereby obtaining high-precision detection, this method is not applicable to the whole polished rice rate detection of large quantities of rice samples, and it is difficult to meet the development needs of the rice processing industry; in order to meet the development needs, some researchers have adopted the image segmentation method to segment the adhesion area and then conduct subsequent detection and analysis, but the existing adhesion rice grain segmentation method can only deal with the segmentation problem of particles of a certain specific shape, and when the degree of adhesion is large, it is easy to have under-segmentation problems, that is, the existing adhesion rice grain segmentation algorithm is difficult to adapt to the irregular rice grain size and shape, and often produces incorrect segmentation, resulting in large errors in the whole polished rice rate, which in turn causes the problem of increased losses for grain enterprises.

[0006] In addition, the existing methods for detecting the whole polished rice rate are all through sampling offline detection, which has problems such as delayed detection feedback, low efficiency, time-consuming and labor-intensive, and insufficient accuracy. Therefore, there is an urgent need for a method that can realize online real-time and accurate detection of the whole polished rice rate. Summary of the invention

[0007] To this end, the technical problem to be solved by the present invention is to overcome the problems in the prior art of offline detection through sampling, such as delayed detection feedback, low efficiency, time and labor consumption, and insufficient accuracy; the existing algorithm for segmenting sticky rice grains is difficult to adapt to the irregular size and shape of rice grains, and the phenomenon of erroneous segmentation occurs, resulting in large errors in the detection of the whole polished rice rate, which in turn causes the problem of increased losses for grain enterprises.

[0008] In order to solve the above technical problems, the present invention provides an online detection method for whole polished rice rate based on machine vision, comprising:

[0009] Obtaining an initial binary image of a rice image; after obtaining various feature parameters in the initial binary image according to the shape features of rice in the initial binary image, inputting the parameters into a BP neural network, screening the rice grain adhesion area in the initial binary image, and obtaining a binary image to be segmented;

[0010] The multiple distance transform watershed segmentation method is used to segment each rice grain adhesion area in the binary image to be segmented, and a first segmentation result image is obtained, including:

[0011] Performing distance transformation, normalization, and binarization processing on the binary image to be segmented to obtain a distance transformation binary image;

[0012] If there are rice grain adhesion areas in the distance transform binary image, then each rice grain adhesion area in the distance transform binary image is screened to obtain a primary screening image; if there are rice grain adhesion areas in the primary screening image after the distance transform, then each rice grain adhesion area in the primary screening image after the distance transform is screened to obtain a secondary screening image; and so on, until there are no rice grain adhesion areas in the obtained screening image after the distance transform, and the screening image obtained at this time is used as the target image; based on the distance transform binary image and the target image, the foreground area mark of the distance transform binary image is determined to obtain a foreground area map;

[0013] If there is no rice grain adhesion area in the distance transform binary image, then based on the distance transform binary image, determine the foreground area mark of the distance transform binary image to obtain a foreground area map;

[0014] The foreground region map is combined with the background region mark of the binary image to be segmented to determine the foreground region boundary and obtain a complete marked image; based on the complete marked image, a first segmentation result map is obtained by a watershed transformation method;

[0015] The minimum circumscribed rectangle method is used to obtain the length of each rice grain in the rice image based on the segmentation result image, thereby calculating the whole polished rice rate of the rice image.

[0016] Preferably, after obtaining the first segmentation result map, the method further includes:

[0017] If there is still a rice grain adhesion area in the first segmentation result image, the rice grain adhesion area in the first segmentation result image is segmented by using an angle-limited concave point matching segmentation method to obtain a second segmentation result image as a final segmentation result image, including:

[0018] Screen the rice grain adhesion areas in the first segmentation result image to obtain each adhesion area image; use the ellipse structure element to perform closing operation on the current adhesion area image to obtain the adhesion closed area map corresponding to the current adhesion area image; use the Canny operator to perform edge detection on the adhesion closed area in the current adhesion closed area map to obtain the contour curve of the current adhesion closed area; perform difference operation on the current adhesion area image and its corresponding adhesion closed area map to obtain the concave area map corresponding to the current adhesion area image; perform skeletonization operation on the current concave area map to obtain the concave area skeletonization map corresponding to the current adhesion area image; find all the endpoints in the concave area skeletonization map corresponding to the current adhesion area image to obtain the skeleton endpoint map corresponding to the current adhesion area image; calculate the shortest distance from each endpoint in the current skeleton endpoint map to the contour curve of the current adhesion closed area, take the endpoint farthest from the contour curve of the current adhesion closed area as the concave point to be segmented, and obtain the concave point extraction map corresponding to the current adhesion area image;

[0019] If the number of concave points in the current concave point extraction map exceeds two, any concave point in the current concave point extraction map is taken as the first concave point, and a coordinate system is established with the first concave point as the coordinate origin; after respectively calculating the slope values ​​between the first concave point and the first skeleton endpoint and the second skeleton endpoint in the same concave region where the first concave point is located, respectively calculate the angle between the line connecting the first skeleton endpoint and the first concave point and the positive direction of the y-axis of the coordinate system, and the angle between the line connecting the second skeleton endpoint and the first concave point and the positive direction of the y-axis of the coordinate system, to obtain the limited angle range of the first concave point; select any concave point in the current concave point extraction map except the first concave point as the second concave point; if the angle between the line connecting the second concave point and the first concave point and the positive direction of the y-axis of the coordinate system is within the limited angle range of the first concave point, If the distance between the second concave point and the first concave point is the shortest, then it is determined that the second concave point matches the first concave point successfully, and the concave point pair to be segmented corresponding to the first concave point is obtained, and the concave point pair to be segmented corresponding to the first concave point is connected; if the angle between the line connecting the second concave point and the first concave point and the positive direction of the y-axis of the coordinate system is not within the limited angle range of the first concave point, then it is determined that the second concave point matches the first concave point unsuccessfully; the concave point pairs to be segmented corresponding to each concave point are obtained in turn, and the concave point pairs to be segmented corresponding to each concave point are connected to obtain the segmented image corresponding to the current adhesion area image; wherein, the direction of any connecting line passing through the first concave point is from any skeleton endpoint / any concave point to the first concave point; any angle is the angle between the positive direction of any connecting line and the positive direction of the y-axis of the coordinate system;

[0020] If the number of concave points in the current concave point extraction map is equal to 2, the two concave points in the current concave point extraction map are paired and connected to obtain a segmented image corresponding to the current adhesion area image;

[0021] Each adhesion region image is segmented in turn to obtain a segmented image corresponding to each adhesion region image; based on the segmented image corresponding to each adhesion region image, a second segmentation result image is obtained as a final segmentation result.

[0022] Preferably, the method of taking any concave point in the current concave point extraction map as the first concave point and taking the first concave point as the coordinate origin to establish a coordinate system; and respectively calculating the slope values ​​between the first concave point and the first skeleton endpoint and the second skeleton endpoint in the same concave region where the first concave point is located comprises:

[0023] Taking any concave point in the current concave point extraction graph as the first concave point, and taking the first concave point as the coordinate origin, a coordinate system is established; wherein the coordinate of the first concave point is (0,0);

[0024] Based on the current coordinate system, the coordinates of the first skeleton endpoint are preset to be (x1, y1), and the coordinates of the second skeleton endpoint are preset to be (x2, y2); wherein the first skeleton endpoint and the second skeleton endpoint are two skeleton endpoints in the same concave area as the first concave point;

[0025] Based on the coordinates of the first concave point and the coordinates of the first skeleton endpoint, the slope value k1 of the first concave point and the first skeleton endpoint in the same region is calculated, and the expression is: k1=y1 / x1;

[0026] Based on the coordinates of the first concave point and the coordinates of the second skeleton endpoint, the slope value k2 of the first concave point and the second skeleton endpoint in the same region is calculated, and the expression is: k2=y2 / x2.

[0027] Preferably, based on the slope value between the first concave point and the first skeleton endpoint and the second skeleton endpoint in the same concave region where the first concave point is located, the angle between the line connecting the first skeleton endpoint and the first concave point and the positive direction of the y-axis of the coordinate system and the angle between the line connecting the second skeleton endpoint and the first concave point and the positive direction of the y-axis of the coordinate system are calculated respectively, and the limited angle range of the first concave point is obtained, which includes:

[0028] Based on the slope value k1 between the first concave point and the first skeleton endpoint in the same concave region, the angle θ1 between the line connecting the first skeleton endpoint and the first concave point and the positive direction of the y-axis of the coordinate system is calculated, and the expression is: θ1 = arctank1-π;

[0029] Based on the slope value k2 between the first concave point and the second skeleton endpoint in the same concave region, the angle θ2 between the line connecting the second skeleton endpoint and the first concave point and the positive direction of the y-axis of the coordinate system is calculated, and the expression is: θ2 = arctank2-π;

[0030] Based on the angle θ1 between the line connecting the first skeleton endpoint and the first concave point and the positive direction of the y-axis of the coordinate system, and the angle θ2 between the line connecting the second skeleton endpoint and the first concave point and the positive direction of the y-axis of the coordinate system, the limited angle range of the first concave point is [θ1,θ2]; among which, θ1<θ2 is preset.

[0031] Preferably, the rice shape characteristics include: the area, perimeter, aspect ratio, circularity and convexity of the rice grain adhesion area, and the area, perimeter, aspect ratio, circularity and convexity of the rice grain non-adhesion area; the characteristic parameters include: the area ratio, perimeter ratio, shape factor and depression distance ratio between the rice grain adhesion area and the rice grain non-adhesion area.

[0032] Preferably, the step of calculating the polished rice rate of the rice image comprises:

[0033] Based on the image size of the initial binary image and the pixel size of each rice grain in the initial binary image, the area of ​​each rice grain in the rice image is calculated, and the expression is:

[0034]

[0035] Among them, S n represents the area of ​​the nth rice grain in the rice image; hn (i n ,j n ) represents the pixel size of the nth rice grain in the rice image; M×N represents the image size of the initial binary image;

[0036] Based on the area of ​​each rice grain in the rice image, the area of ​​all rice grains in the rice image is calculated, and its expression is: S all =∑ n＝1 S n ;

[0037] According to the area of ​​all rice grains in the rice image and the area of ​​all polished rice grains in the rice image, the polished rice rate P of the rice image is calculated, and its expression is:

[0038]

[0039] Among them, S w Represents the area of ​​all whole rice grains in the rice image.

[0040] Preferably, the step of obtaining a rice image and preprocessing the rice image to obtain an initial binary image comprises:

[0041] The grayscale rice image is enhanced by using Gamma transformation to obtain an enhanced grayscale rice image. The enhanced rice image is binarized by using the maximum inter-class variance method to separate the outline of each rice grain in the enhanced grayscale rice image from the background to obtain a binarized grayscale rice image. The binarized grayscale rice image is denoised to obtain an initial binarized image.

[0042] The present invention also provides an online detection system for whole polished rice rate based on machine vision, comprising:

[0043] An image acquisition device, used for acquiring rice images, comprises:

[0044] A loading platform, which is equipped with a loading platform and a loading bracket;

[0045] An industrial camera is installed on the top of the loading bracket and is equipped with an industrial lens for photographing the rice on the loading platform to obtain an image of the rice;

[0046] A light source device is installed on the object carrier and is located between the object carrier platform and the industrial camera, and is used to illuminate the object carrier platform;

[0047] A light source controller, which is connected to the light source device and is used to control the brightness of the light source in the light source device;

[0048] An image processing device, used for processing a rice image and displaying the whole polished rice rate of the rice image, comprises:

[0049] A processor is connected to the industrial camera in the image acquisition device through communication; the processor is used to receive the rice image, execute the above-mentioned online detection method of whole polished rice rate based on machine vision, and obtain the whole polished rice rate of the rice image;

[0050] A display screen, which is connected to the processor via an interface and is used to display the polished rice rate of the rice image;

[0051] A control device is connected to the processor in the image processing device; the control device is used for online detection and closed-loop control of whole polished rice grains.

[0052] Preferably, the processor is a computer or a Raspberry Pi operating device.

[0053] Preferably, the control device is connected to the processor via a serial port, a network port or a wireless method.

[0054] The above technical solution of the present invention has the following beneficial effects compared with the prior art:

[0055] (1) The present invention discloses an online detection method for whole polished rice rate based on machine vision. According to the difference in appearance characteristics between sticky rice and single-grain rice, a feature vector is constructed by analyzing and measuring feature parameters such as the perimeter ratio, shape factor, area ratio and concave defect distance of each connected area in the rice image, and the feature vectors are used to train a BP neural network. The trained BP neural network performs intelligent adhesion screening on each contour area in the initialized binary image, which is convenient for subsequent segmentation of the adhesion area, and compared with the single-parameter threshold discrimination method, the recognition accuracy of the adhesion area is improved, thereby improving the segmentation accuracy and efficiency. A multiple distance transformation watershed segmentation method is designed, and multiple distance changes and screening operations are sequentially performed on the binary image to be segmented until all the sticky areas in the binary image to be segmented are successfully separated, so that the number of starting points of the sticky area can be increased and the accuracy of segmentation can be improved. At the same time, the problem that the existing sticky rice grain segmentation algorithm is difficult to adapt to the irregular rice grain size and shape and causes erroneous segmentation is solved. At the same time, the area ratio method is used on the basis of image processing to calculate and analyze the whole polished rice rate of the rice image processed by the adhesion segmentation algorithm.

[0056] (2) The present invention discloses an online detection method for whole polished rice rate based on machine vision. After the binary image to be segmented is segmented by the multiple distance transform watershed segmentation method, a first segmentation result image is obtained. If there is still a rice grain adhesion area in the first segmentation result image, the first segmentation result image is segmented by the designed angle-limited concave point matching segmentation method. By identifying and matching the concave points corresponding to the adhesion area in the first segmentation result image, accurate segmentation of each adhesion area is achieved, thereby improving the overall segmentation accuracy. The segmentation method of the multiple distance transform watershed and the angle-limited concave point matching is integrated to efficiently segment the screened adhesion area, so that the adhesion rice can be removed. Accurately segment the rice into single grains while ensuring that the shape and size of the rice are not affected, taking into account both the segmentation effect and the execution efficiency, and improving the accuracy of the whole polished rice rate detection; the segmentation method integrating multiple distance transformation watersheds and angle-limited concave point matching further solves the problem that the single segmentation algorithm is prone to serious erroneous segmentation due to the different sizes and shapes of rice grains and the different degrees of adhesion, which makes the subsequent calculation of the appearance parameters of the rice and the whole polished rice rate have serious errors. While improving the accuracy of segmentation, it does not change the original shape and size of the rice grains. Compared with a single segmentation method, it is more applicable, further improves the accuracy of segmentation, and facilitates the subsequent measurement of the characteristic parameters of single rice grains;

[0057] (3) The machine vision-based online detection system for polished rice rate of the present invention deploys a neural network model for intelligent screening of adhesion areas, a segmentation method integrating multiple distance transformation watersheds and angle-limited concave point matching, and a method for calculating polished rice rate based on an area method on the hardware of a processor. The detected polished rice rate can be displayed on a display screen in real time, and can also be output to a control device through a serial port, a network port, or wirelessly, thereby realizing online detection, accelerating the detection speed, improving production efficiency, and reducing rice processing losses. In addition, the system can be deployed by a processing enterprise on an existing production line alone, or the system can be integrated into a rice mill control system to build an intelligent rice processing production line. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to make the content of the present invention more clearly understood, the present invention is further described in detail below according to specific embodiments of the present invention in conjunction with the accompanying drawings, wherein

[0059] Figure 1 It is a flow chart of an online detection method for whole polished rice rate based on machine vision provided in Example 1 of the present invention;

[0060] Figure 2 It is a flow chart of an online detection method for whole polished rice rate based on machine vision provided by Embodiment 2 of the present invention;

[0061] Figure 3 It is a flowchart of the multiple distance transform watershed;

[0062] Figure 4 is a schematic diagram of the multiple distance transform watershed segmentation process; Figure 4 (a) in the figure represents the initial binary image; Figure 4 (b) in the figure represents the distance transform binary image; Figure 4 (c) in the figure represents a screening graph; Figure 4 (d) in the figure represents the secondary screening graph; Figure 4 (e) in the figure represents a cubic screening image, and the cubic screening image is used as the target image; Figure 4 (f) in the figure represents the foreground area map; Figure 4 (g) in the figure represents the final labeled image; Figure 4 (h) in FIG. 1 represents the first segmentation result image;

[0063] Figure 5 is a schematic diagram of the pit detection process; where Figure 5 (a) shows the adhesion area map; Figure 5 (b) in the figure shows the adhesion closed region diagram; Figure 5 (c) in the figure shows the contour curve of the adhesion closed area; Figure 5 (d) in the figure shows the concave region diagram; Figure 5 (e) in the figure shows the skeletonized image of the concave region; Figure 5 (f) in the figure represents the skeleton endpoint graph; Figure 5 (g) in the figure represents the concave point extraction map;

[0064] Figure 6 It is a schematic diagram of concave point matching;

[0065] Figure 7 It is the block diagram of the detection system;

[0066] Figure 8 It is a computer vision system. DETAILED DESCRIPTION

[0067] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it, but the embodiments are not intended to limit the present invention.

[0068] Example 1

[0069] Reference Figure 1 As shown, a flow chart of an online detection method for whole polished rice rate based on machine vision provided by Embodiment 1 of the present invention; specifically comprising:

[0070] S11: Acquire a rice image, pre-process the rice image, and obtain an initial binary image; wherein the initial binary image is a binary image consisting of a black background and a white rice area;

[0071] Among them, the preprocessing of rice images includes:

[0072] (1) Using Gamma transformation, the grayscale rice image is enhanced to obtain an enhanced grayscale rice image; wherein the transformation formula is:

[0073] s=cr γ

[0074] Among them, r represents the gray value of the input image; c represents the gray scaling factor; γ represents the gamma factor, which is used to control the image scaling degree; s represents the gray value of the transformed output image;

[0075] Set the gamma factor to a larger value to reduce the grayscale value of the brighter area, enhance the contrast of the image, highlight the edge details of the area, and improve the uneven light phenomenon in the image;

[0076] (2) using the maximum inter-class variance method to perform binarization on the enhanced rice image, separating the outline of each rice grain from the background in the enhanced grayscale rice image, and obtaining a binarized grayscale rice image; wherein, the maximum inter-class variance method is used to calculate image parameters, and a threshold is determined, and image parameters greater than the threshold are white, and image parameters less than the threshold are black, thereby separating the outline of each rice grain from the background in the enhanced grayscale rice image, and then obtaining a binarized grayscale rice image;

[0077] (3) performing a denoising operation on the binarized grayscale rice image to obtain an initial binarized image; wherein, each binarized region in the binarized grayscale rice image is traversed and the number of pixels corresponding thereto is calculated, and a small threshold parameter is set according to the pixel value of the global image region to eliminate isolated and fine noise points, thereby removing small impurities, dust, debris and other information, and finally obtaining an initial binarized image consisting of a black background and a white rice region.

[0078] During the image acquisition process, due to uneven light and dust and debris in the background, there are bright and dark areas and too small noise areas in the image, which in turn affects the subsequent separation of the background image and the extraction of the rice grain contour area. In order to eliminate this effect, the image is preprocessed through steps such as image enhancement, background segmentation, and image denoising.

[0079] In S11, by preprocessing the collected rice color image, invalid information such as noise in the image is eliminated, thereby improving the quality and analyzability of the image; at the same time, after the image is binarized, an initial binarized image of the black background and white rice area is obtained; this processing step effectively separates the contour area of ​​the rice from the background, laying the foundation for subsequent feature extraction and analysis.

[0080] S12: obtaining each characteristic parameter in the initial binary image according to the shape characteristics of rice in the initial binary image; using a BP neural network, screening the rice grain adhesion area in the initial binary image according to each characteristic parameter in the initial binary image, and obtaining a binary image to be segmented;

[0081] The rice shape characteristics include: the area, perimeter, aspect ratio, circularity and convexity of the rice grain adhesion area, and the area, perimeter, aspect ratio, circularity and convexity of the rice grain non-adhesion area; the characteristic parameters include: the area ratio, perimeter ratio, shape factor and depression distance ratio between the rice grain adhesion area and the rice grain non-adhesion area.

[0082] In S12, in order to solve the problem of rice sticking to each other in the image, it is necessary to effectively separate the sticky rice grains so as to calculate the characteristic parameters of the appearance of each individual rice grain; therefore, the present invention adopts a method of extracting rice shape features in the initial binary image, and uses a BP (Back Propagation) neural network to classify according to the extracted rice shape features, distinguish whether the rice grain area in the image is sticky, and provide an accurate data basis for the subsequent sticky rice grain segmentation problem, which specifically includes:

[0083] According to the differences in shape features such as area, perimeter, aspect ratio, circularity and convexity between the adhesion rice area and the non-adhesion rice area in the initial binary image, four feature parameters including area ratio, perimeter ratio, shape factor and concave distance ratio are selected to calculate and analyze the rice image;

[0084] 600 initial binary images of long-grain rice and 600 initial binary images of short-round rice were selected as the experimental data set, and the ratio of training set to test set was 8:2; among them, the positive sample was the adhesion rice image, and the negative sample was the non-adhesion rice image, and the ratio of positive and negative samples was the same; the four feature parameters corresponding to each initial binary image were calculated, and a 960-item 4-dimensional feature vector was constructed to train the BP neural network with a network structure of 4×9×1. After that, the adhesion rice area in the image can be screened out through the trained adhesion classification network, and each binary image to be segmented is obtained, which is convenient for subsequent image adhesion segmentation;

[0085] The present invention proposes to adopt a pre-trained BP neural network model to intelligently screen out the area of ​​adhered rice grains in the collected rice grain image, provide an effective area for subsequent image segmentation, reduce the amount of calculation, and improve the segmentation efficiency.

[0086] S13: using a multiple distance transform watershed segmentation method, segmenting each rice grain adhesion area in the binary image to be segmented, and obtaining a first segmentation result image, including:

[0087] Performing distance transformation, normalization, and binarization processing on the binary image to be segmented to obtain a distance transformation binary image;

[0088] If there are rice grain adhesion areas in the distance transformation binary image, then the various rice grain adhesion areas in the distance transformation binary image are screened out to obtain a primary screening image; a distance change operation is performed on the primary screening image to obtain a primary screening image after distance transformation; if there are rice grain adhesion areas in the primary screening image after distance transformation, then the various rice grain adhesion areas in the primary screening image after distance transformation are screened out to obtain a secondary screening image; and so on, until there are no rice grain adhesion areas in the obtained screening image after distance transformation, and the screening image obtained at this time is used as the target image; based on the distance transformation binary image and the target image, the foreground area mark of the distance transformation binary image is determined, and after obtaining the foreground area map, the foreground area boundary is determined in combination with the background area mark of the binary image to be segmented to obtain a complete marked image; based on the complete marked image, a first segmentation result map is obtained through watershed transformation;

[0089] If there is no rice grain adhesion area in the distance transform binary image, then based on the distance transform binary image, the foreground area mark of the distance transform binary image is determined, and after obtaining the foreground area map, it is combined with the background area mark of the binary image to be segmented to determine the foreground area boundary and obtain a complete marked image; based on the complete marked image, a first segmentation result map is obtained through a watershed transform method.

[0090] In existing research, the traditional watershed algorithm mainly uses the grayscale information of the image for segmentation, but it is easily affected by image noise and irregularity, and often leads to over-segmentation, that is, the generation of small areas. The watershed segmentation algorithm based on distance transformation first performs distance transformation on the binary image, returns the distance as the grayscale value, and forms a separated foreground area through threshold processing; then, it simulates the water level rising until it encounters the watershed, divides the image into different areas, and solves the problem of over-segmentation.

[0091] However, when processing a sticky target such as rice, a single distance transform watershed segmentation cannot effectively separate all the sticky rice, especially when the sticky degree is large, it may not be possible to produce an obvious local minimum area, resulting in under-segmentation; in order to solve this problem, the present invention uses a multiple distance transform watershed segmentation method until all areas are successfully separated; this can increase the number of starting points and improve the accuracy of segmentation; however, too many repetitions may cause the small target area to be excessively corroded and disappear, causing over-segmentation and affecting the subsequent rice quantity detection; in response to this situation, the distance transform target should be screened, and only the sticky area should be subjected to multiple distance transforms to retain the separated area; the specific operation steps are as follows Figure 3 As shown, in a specific embodiment of the present invention, it includes:

[0092] right Figure 4 The distance transformation is performed on the binary image to be segmented shown in (a) to reduce the number of adhesion areas and simplify the processing steps; the image data is adjusted to the range of [0,1] through normalization operation to improve the stability and efficiency of the algorithm; the processed image is binarized to obtain Figure 4 (b) in the figure; the screened area is subjected to distance transformation and screening again to confirm the separation of the adhesion area; after multiple rounds of screening and distance transformation, most of the adhesion areas can be effectively segmented after three to four treatments; the screening process is as follows Figure 4 As shown in (c), (d), and (e) in FIG. 1 ; the foreground region boundary is determined by combining the background mark of the rice grain image as shown in FIG. Figure 4 As shown in (f) in the figure, the complete labeled image is obtained, that is, Figure 4 (g) in the figure; Based on the complete labeled image, the watershed transform is used to obtain Figure 4 The segmentation result shown in (h) Figure 4 As can be seen in (h), almost all the adhered rice areas are successfully segmented and the original contour size and size are retained.

[0093] S14: Utilizing the minimum circumscribed rectangle method, based on the obtained segmentation result graph, the length of each rice grain in the rice image is obtained; based on the condition that rice grains whose length reaches three quarters or more of the reference value of the length of whole polished rice grains are whole polished rice grains, the length of each rice grain in the rice image is compared with the reference value of the length of whole polished rice grains to obtain each whole polished rice grain in the rice image; based on the area of ​​all rice grains in the rice image and the area of ​​all whole polished rice grains in the rice image, the whole polished rice rate of the rice image is calculated; wherein, the segmentation result graph obtained at this time is the first segmentation result graph.

[0094] According to national standards, rice grains whose length reaches three quarters or more of the average length of whole polished rice are judged as whole polished rice, while rice grains whose length is less than three quarters of the average length of whole polished rice are judged as broken rice.

[0095] Therefore, in S14, in order to obtain the whole polished rice rate corresponding to the rice image, it is necessary to first select a batch of complete polished rice grains, obtain their average length experimentally as a reference value for the length of the whole polished rice grains, and then calculate the length of the divided rice grains, and determine whether it is whole polished rice by comparing it with the reference value for the length of the whole polished rice grains.

[0096] In order to calculate the length of the divided rice grains, the minimum enclosing rectangle method is used for calculation. The basic idea is to rotate the outline of the target at different angles to find the minimum enclosing rectangle. At this time, the length of the minimum enclosing rectangle method is the length of the target rice grain. During the rotation process, each pixel in the image will change its position, so it is necessary to calculate the coordinates after rotation. Two methods are considered for calculation:

[0097] Method 1: If the image is rotated around the origin (0,0), the position of the rotated pixel can be expressed by the following formula:

[0098]

[0099]

[0100] Among them, (a0, b0) is the pixel coordinate before rotation, (a1, b1) is the pixel coordinate after rotation, is the angle of rotation of the target rice grain.

[0101] Method 2: If the image is rotated around a point (p, q), then the coordinate point can be translated to the point first, then rotated, and finally translated back to the original position. This ensures that the center of the rotated image remains unchanged. The position of the pixel point after rotation is expressed by the following formula:

[0102]

[0103] Among them, assuming that the rice contour has k points and the number of rotation angles is ω, then after each rotation, it is necessary to traverse all points to calculate the area of ​​the circumscribed rectangle, which requires O(k) time; therefore, the total time complexity is O(ωk); if the number of rotation angles is a constant, then the time complexity can be simplified to O(k); the calculation result of the minimum circumscribed rectangle method only depends on the shape of the rice grain contour, and is not affected by the direction of the rice grain, and is applicable to various situations.

[0104] For the same variety of rice, the density and thickness of the rice are approximately considered to be the same. The polished rice rate is the ratio of the polished rice area to the sample rice area. The rice grain area in the binary image is the number of connected pixels in the foreground of the rice grain image. The binary image of the nth rice grain is preset as h n (i n ,j n), the size of the initial binary image is M×N, and when the pixels in the foreground rice grain area are 1 and the pixels in the background area are 0, the area of ​​the nth rice grain is S n The expression is:

[0105]

[0106] Among them, S n represents the area of ​​the nth rice grain in the rice image; h n (i n ,j n ) represents the pixel size of the nth rice grain in the rice image; M×N represents the image size of the initial binary image;

[0107] All rice grains are marked to obtain the area of ​​each rice grain in the rice image, thereby calculating the area of ​​all rice grains in the rice image, and its expression is: S all =∑ n＝1 S n ;

[0108] According to the reference value of the length of the whole milled rice, the total area of ​​the whole milled rice is S w According to the area of ​​all rice grains in the rice image and the area of ​​all polished rice grains in the rice image, the polished rice rate P of the rice image is calculated, and its expression is: P = S w / S all ×100%.

[0109] Example 2

[0110] Reference Figure 2 As shown, a flow chart of an online detection method for whole polished rice rate based on machine vision provided by Embodiment 2 of the present invention; specifically comprising:

[0111] The steps of S21-S23 are consistent with the steps of S12-S13, and based on S21-S23, a first segmentation result map is obtained;

[0112] S24: If there are still rice grain adhesion areas in the first segmentation result image, segment the rice grain adhesion areas in the first segmentation result image using an angle-limited concave point matching segmentation method to obtain a second segmentation result image as a final segmentation result image, including:

[0113] Screen the rice grain adhesion areas in the first segmentation result image to obtain each adhesion area image; use the ellipse structure element to perform closing operation on the current adhesion area image to obtain the adhesion closed area map corresponding to the current adhesion area image; use the Canny operator to perform edge detection on the adhesion closed area in the current adhesion closed area map to obtain the contour curve of the current adhesion closed area; perform difference operation on the current adhesion area image and its corresponding adhesion closed area map to obtain the concave area map corresponding to the current adhesion area image; perform skeletonization operation on the current concave area map to obtain the concave area skeletonization map corresponding to the current adhesion area image; find all the endpoints in the concave area skeletonization map corresponding to the current adhesion area image to obtain the skeleton endpoint map corresponding to the current adhesion area image; calculate the shortest distance from each endpoint in the current skeleton endpoint map to the contour curve of the current adhesion closed area, take the endpoint farthest from the contour curve of the current adhesion closed area as the concave point to be segmented, and obtain the concave point extraction map corresponding to the current adhesion area image;

[0114] If the number of concave points in the current concave point extraction map exceeds two, any concave point in the current concave point extraction map is used as the first concave point, and the first concave point is used as the coordinate origin to establish a coordinate system; the slope values ​​between the first concave point and the first skeleton endpoint and the second skeleton endpoint in the same concave region where the first concave point is located are calculated respectively, including:

[0115] Taking any concave point in the current concave point extraction graph as the first concave point, and taking the first concave point as the coordinate origin, a coordinate system is established; wherein the coordinate of the first concave point is (0,0);

[0116] Based on the current coordinate system, the coordinates of the first skeleton endpoint are preset to be (x1, y1), and the coordinates of the second skeleton endpoint are preset to be (x2, y2); wherein the first skeleton endpoint and the second skeleton endpoint are two skeleton endpoints in the same concave area as the first concave point;

[0117] Based on the coordinates of the first concave point and the coordinates of the first skeleton endpoint, the slope value k1 of the first concave point and the first skeleton endpoint in the same region is calculated, and the expression is: k1=y1 / x1;

[0118] Based on the coordinates of the first concave point and the coordinates of the second skeleton endpoint, the slope value k2 of the first concave point and the second skeleton endpoint in the same region is calculated, and the expression is: k2=y2 / x2;

[0119] Based on the slope values ​​between the first concave point and the first skeleton endpoint and the second skeleton endpoint in the same concave region where the first concave point is located, the angle between the line connecting the first skeleton endpoint and the first concave point and the positive direction of the y-axis of the coordinate system and the angle between the line connecting the second skeleton endpoint and the first concave point and the positive direction of the y-axis of the coordinate system are calculated respectively, and the limited angle range of the first concave point is obtained, including:

[0120] Based on the slope value k1 between the first concave point and the first skeleton endpoint in the same concave region, the angle θ1 between the line connecting the first skeleton endpoint and the first concave point and the positive direction of the y-axis of the coordinate system is calculated, and the expression is: θ1 = arctank1-π;

[0121] Based on the slope value k2 between the first concave point and the second skeleton endpoint in the same concave region, the angle θ2 between the line connecting the second skeleton endpoint and the first concave point and the positive direction of the y-axis of the coordinate system is calculated, and the expression is: θ2 = arctank2-π;

[0122] Based on the angle θ1 between the line connecting the first skeleton endpoint and the first concave point and the positive direction of the y-axis of the coordinate system, and the angle θ2 between the line connecting the second skeleton endpoint and the first concave point and the positive direction of the y-axis of the coordinate system, the limited angle range of the first concave point is obtained as [θ1, θ2]; wherein θ1<θ2 is preset;

[0123] Select any concave point other than the first concave point in the current concave point extraction map as the second concave point; if the angle θ0 between the line connecting the second concave point and the first concave point and the positive direction of the y-axis of the coordinate system is within the limited angle range [θ1, θ2] of the first concave point, and the distance between the second concave point and the first concave point is the shortest, then it is determined that the second concave point matches the first concave point successfully, and the concave point pair to be segmented corresponding to the first concave point is obtained, and the concave point pair to be segmented corresponding to the first concave point is connected to complete the segmentation of the adhesion area corresponding to the first concave point; if the angle θ0 between the line connecting the second concave point and the first concave point and the positive direction of the y-axis of the coordinate system is not within the limited angle range of the first concave point, then it is determined that the second concave point matches the first concave point unsuccessfully; sequentially obtain the concave point pair to be segmented corresponding to each concave point, and connect the concave point pair to be segmented corresponding to each concave point to obtain the segmented image corresponding to the current adhesion area image; wherein, the direction of any line passing through the first concave point is from any skeleton endpoint / any concave point to the first concave point; any angle is the angle between the positive direction of any line and the positive direction of the y-axis of the coordinate system;

[0124] If the number of concave points in the current concave point extraction map is equal to 2, the two concave points in the current concave point extraction map are paired and connected to obtain a segmented image corresponding to the current adhesion area image;

[0125] Sequentially segmenting each adhesion region image to obtain a segmented image corresponding to each adhesion region image; based on the segmented image corresponding to each adhesion region image, obtaining a second segmentation result image as a final segmentation result image;

[0126] If there is no rice grain adhesion area in the first segmentation result image, the first segmentation result image is used as the final segmentation result image.

[0127] Among them, in S23, the watershed segmentation method based on multiple distance transforms can segment most of the areas of adhered rice grains, but there is an under-segmentation problem when processing overly complex adhesion areas; in order to solve this problem, the present invention proposes a segmentation method based on angle-limited concave point matching, which separately processes the adhesion areas that cannot be processed by multiple distance transforms, thereby further improving the segmentation accuracy.

[0128] In multiple experimental observations, it was found that the adhesion area that cannot be processed by the multiple distance transform is mainly due to the large aspect ratio of the rice grains, which causes the rice grains to adhere tightly and cannot form obvious concave points, making it impossible for the watershed algorithm to generate a watershed line; in response to this situation, the present invention proposes a concave point matching segmentation method based on angle limitation, which realizes accurate segmentation of these adhesion areas by identifying and matching concave points in the adhesion area. This method overcomes the problem that the multiple distance transform watershed algorithm is difficult to process adhesion areas of specific shapes, and improves the overall segmentation accuracy; the method includes two steps: concave point detection and concave point matching. In a specific embodiment of the present invention, the specific operations are as follows:

[0129] Step 1: Concave point detection method, first perform concave point detection on the adhesion area, that is, find the concave defects between rice grains as the basis for segmentation. This step includes:

[0130] (1) Screening the adhesion regions in the image to obtain each adhesion region image in the first segmentation result image; wherein any adhesion region image such as Figure 5 As shown in (a), the area still adhered after the second screening of the first segmentation result image is taken as the object of concave point detection for subsequent operations;

[0131] (2) Extract the concave area of ​​the adhesion area; In order to obtain independent concave areas, a large-size elliptical structure element is used to perform a closing operation on the adhesion area, and the concave part of the adhesion area image is filled to obtain a smooth area, such as Figure 5 As shown in (b) Figure 5 (b) and Figure 5 By performing a difference operation on (a) in , we can obtain mutually independent concave regions, such as Figure 5 As shown in (d);

[0132] (3) Skeletonize the concave area and extract the concave points; Figure 5 The concave region of (d) is skeletonized to obtain Figure 5 The concave region skeleton diagram shown in (e) in FIG; Find all the endpoints of the concave region skeleton diagram and obtain Figure 5The skeleton endpoint diagram shown in (e) in FIG, wherein these endpoints are located at the boundary of the concave region and have a certain distance from the edge curve of the adhesion region; the Canny operator is used to perform edge detection on the adhesion closed region and extract the boundary curve of the adhesion region, and the following is obtained: Figure 5 The contour curve of the adhesion closed area shown in (c) is calculated for each skeleton endpoint and Figure 5 The shortest distance of the contour curve of the adhesion closed area shown in (c) is determined, and the endpoint farthest from the contour curve is determined as the concave point to be segmented, and the following is obtained: Figure 5 The concave point extraction diagram shown in (g) in FIG. 1 is the concave points to be segmented in the adhesion rice grain area;

[0133] Step 2: concave point matching method; wherein, after obtaining the concave points to be segmented in the adhesion rice grain area, the concave points need to be correctly matched to obtain an accurate segmentation result; according to the number of concave points in each adhesion area, two different concave point matching strategies are adopted, as follows:

[0134] (1) When the number of concave points exceeds 2, it indicates that more than two grains of rice are stuck together. In order to accurately match all the concave points, the present invention adopts a matching method based on angle limitation. The idea of ​​this method is: for any concave point, find the angle between it and the other concave points. If the angle between the two points is within a certain range and the distance between the two points is the shortest within this range, the matching is successful.

[0135] Reference Figure 6 As shown, concave point A is the first concave point, and concave point A is selected as the origin (0,0), and the straight line angle range [θ1,θ2] between concave point A and its two skeleton endpoints M and N in the same concave area is calculated as the limited angle range [θ1,θ2] of concave point A; wherein, the direction of line MA is from M to A, the direction of line NA is from N to A, θ1 is the angle between the positive direction of line NA and the positive direction of the y-axis of the coordinate system, and θ2 is the angle between the positive direction of line MA and the positive direction of the y-axis of the coordinate system; according to the matching principle of the matching method based on angle limitation, find the concave point B that matches concave point A, at this time, concave point A and concave point B constitute a concave point pair to be segmented, connect concave point A and concave point B to complete the current segmentation; according to the same matching principle, continue to match the remaining concave points until all concave points are successfully matched; when the correctly matched concave point pairs are connected, the correct segmentation result can be obtained;

[0136] (2) When the number of concave points is equal to 2, it means that there are only two rice grains connected in the adhesion area. In this case, it is only necessary to pair and connect the two detected concave points to complete the segmentation and obtain the final segmentation result.

[0137] S25: Utilizing the minimum circumscribed rectangle method, based on the obtained segmentation result graph, the length of each rice grain in the rice image is obtained; based on the condition that rice grains whose length reaches three quarters or more of the reference value of the length of whole polished rice grains are whole polished rice grains, the length of each rice grain in the rice image is compared with the reference value of the length of whole polished rice grains to obtain each whole polished rice grain in the rice image; based on the area of ​​all rice grains in the rice image and the area of ​​all whole polished rice grains in the rice image, the whole polished rice rate of the rice image is calculated; wherein, the segmentation result graph obtained at this time is the final segmentation result graph obtained in S24.

[0138] After being processed by the traditional distance transform watershed segmentation algorithm, some adhered objects can be successfully separated. However, since there are adhered areas with different degrees of adhesion in the adhered rice grain image, a single distance transform cannot effectively separate each adhered area, resulting in the inability of the subsequent watershed algorithm to form a correct watershed boundary, resulting in under-segmentation. Although the use of multiple distance transforms can solve the under-segmentation problem, multiple uses of distance transforms on the same area will generate multiple grayscale maximum points in one area. After the binarization operation, one area will be split into multiple small areas, resulting in the watershed algorithm identifying these small areas as "flooding points", resulting in multiple watershed boundary lines in the same rice grain image area, resulting in over-segmentation. In order to solve the problem of over-segmentation, the present invention designs an angle-limited concave point matching segmentation method.

[0139] Both under-segmentation and over-segmentation phenomena will lead to errors in the subsequent calculation of the number of rice grains and appearance indicators. In order to improve this segmentation defect and to more accurately locate the regional boundary and improve the segmentation accuracy, the present invention further proposes a segmentation method that integrates multiple distance transform watersheds and angle-limited concave point matching, which makes full use of the morphological characteristics of the adhesion rice area and can more accurately segment it into independent rice images, thereby achieving the purpose of segmenting the adhesion area. In combination with S23-S24, the steps are briefly described as follows:

[0140] 1) Performing watershed segmentation with multiple distance transformations on the adhesion rice area pre-screened in the binary image to be segmented;

[0141] 2) For adhesion areas with a greater degree of adhesion, detect the number and position of the concave points;

[0142] 3) using the angle-limited concave point matching method to further segment the adhesion area with a large degree of adhesion;

[0143] The rice grain image processed through the above steps is of great significance for further improving the accuracy of rice polished rice rate detection.

[0144] The segmentation method provided by the present invention integrates multiple distance transformation watershed and angle-limited concave point matching, solves the over-segmentation and under-segmentation problems of the traditional watershed algorithm by performing multiple distance transformation on the adhesion area, and improves the segmentation accuracy; at the same time, after the multiple distance transformation watershed algorithm, the concave point matching is performed on the local adhesion area, which greatly reduces the complexity of the concave point detection and the false detection rate of the concave points, is suitable for the segmentation of adhesion rice grain images of different shapes, takes into account the segmentation effect and the execution efficiency, and improves the accuracy of the whole polished rice rate detection; after being processed by the segmentation method, an accurate single rice image can be obtained, which greatly improves the accuracy of the subsequent detection of the whole polished rice rate of rice.

[0145] The advantages of the online detection method of whole polished rice rate based on machine vision provided by the present invention are:

[0146] 1. Automation: Through image processing technology, the automatic detection of rice grain polishing rate can be realized without manual intervention, saving time and effort;

[0147] 2. Accuracy: The minimum circumscribed rectangle method is used to calculate the length of rice grains. Combined with the image scaling ratio, the actual length of rice grains is accurately calculated, which improves the accuracy of detection;

[0148] 3. Applicability: For different varieties of rice, when the density and thickness are basically the same, this method has strong applicability and can be widely used in the detection of whole polished rice rate in the rice processing industry.

[0149] Example 3

[0150] In order to obtain a rice image, the present invention uses an industrial camera in the image acquisition device provided in the present embodiment to acquire a rice image, and uses the image processing device in the present embodiment to process the rice image and display the whole polished rice rate of the rice image, that is, in the present invention, after acquiring an image through an industrial camera, the image data is transmitted to a processor, and the processor performs image preprocessing, eliminates noise in the rice grain image, performs contrast enhancement processing on the image, and binarizes the image to separate the contour of the target object from the background; a large number of known images are collected, a self-made data set is annotated, and four feature parameters such as area ratio, perimeter ratio, shape factor and concave distance ratio are used to train The BP neural network model performs intelligent rice grain adhesion area screening on the collected rice grain images based on the trained network model, and selects the rice grain adhesion area; multiple distance transformation watershed is performed on the adhesion area to segment the adhesion rice grains; for simple rice grain adhesion, the adhesion rice grains can be separated by using the multiple distance transformation watershed; for complex rice grain adhesion, after the multiple distance watershed transformation, the adhesion is segmented by using the angle-limited concave point matching method; after the segmentation of all rice grain adhesion is completed, rice grain statistics and rice grain characteristic parameters are measured, and then the whole polished rice rate is calculated and output, and the whole polished rice rate is displayed on the display screen in the image processing device.

[0151] In addition, based on the consideration of actual processing production line integration, the neural network model, adhesion area segmentation method and whole polished rice rate detection algorithm need to be deployed to the edge device. Therefore, considering the actual situation of edge hardware computing power at this stage, the model and algorithm cannot be too complex, otherwise the hardware deployment cannot be realized; the selection of BP neural network model and the segmentation method integrating multiple distance watershed transform and angle-limited concave point matching is based on the consideration of edge computing power realization; among them, the multiple distance transform watershed has low computational complexity and high program running efficiency; the concave point matching method first performs concave point detection and then concave point matching, which requires point-by-point detection and matching, with high computational complexity and time-consuming program running. First, multiple distance transform watershed is used to segment a simple adhesion area, thereby reducing the number of concave points. The fusion of multiple distance transform watershed and angle-limited concave point matching method is used for adhesion segmentation, which takes into account both segmentation effect and execution efficiency.

[0152] Reference Figure 7 As shown, a specific embodiment of the present invention further provides an online detection system for whole polished rice rate based on machine vision, comprising:

[0153] An image acquisition device, used for acquiring rice images, comprises:

[0154] A loading platform, which is equipped with a loading platform and a loading bracket;

[0155] An industrial camera is installed on the top of the loading bracket and is equipped with an industrial lens for photographing the rice on the loading platform to obtain an image of the rice;

[0156] A light source device is installed on the object carrier and is located between the object carrier platform and the industrial camera, and is used to illuminate the object carrier platform;

[0157] A light source controller, which is connected to the light source device and is used to control the brightness of the light source in the light source device;

[0158] An image processing device, used for processing a rice image and displaying a whole polished rice rate of the rice image;

[0159] A processor is connected to the industrial camera in the image acquisition device through communication; the processor is used to receive the rice image, execute the above-mentioned online detection method of whole polished rice rate based on machine vision, and obtain the whole polished rice rate of the rice image; the processor is a computer or a Raspberry Pi operating device;

[0160] A display screen, which is connected to the processor via an interface and is used to display the polished rice rate of the rice image;

[0161] A control device connected to the processor in the image processing device; the control device is used to perform online detection and closed-loop control on the whole polished rice grains; the control device is connected to the processor via a serial port, a network port or a wireless method;

[0162] Based on the provided neural network model and adhesion area segmentation method, the present invention constructs an edge-based online detection system for whole polished rice rate, which can be deployed separately to a rice processing production line to achieve online detection, thereby improving production efficiency and reducing rice processing losses. It can also be integrated into a rice mill control system to build an intelligent rice processing production line to further reduce processing losses.

[0163] The image acquisition stage is a crucial link, because the quality of the image directly affects the effect and accuracy of subsequent image processing and analysis, and correctly acquiring the image of rice can provide an accurate data basis for the detection of the whole polished rice rate. At present, many rice quality detection instruments require that the rice sample be manually flattened to make it thin and scattered, and then the rice image is collected by a camera or a scanner; although this method can ensure the clarity of the rice boundary contour in the collected image, due to the randomness of the rice samples in the actual rice processing plant, the manual separation and paving method becomes inflexible and time-consuming; at the same time, this method cannot cope with the actual situation, especially in the case where multiple grains of rice are adhered together; therefore, in order to be more in line with the actual rice detection environment, the present invention randomly collects images of the rice grains scattered on the placement table, and in order to make the subsequent image segmentation method more suitable for different adhesion situations, the rice adhesion areas with different numbers of grains and different adhesion situations are manually placed to simulate various situations that may occur in reality.

[0164] In order to obtain images, the present invention adopts a computer vision system based on rice images, which includes a computer, an industrial camera, an industrial lens, a light source, a light source controller and a stage. Figure 8 As shown; the system uses an industrial camera with an industrial fixed-focus lens to form the imaging part of the system; in order to improve the quality of the image, the system uses LED backlight and ring light source for fill light, making the imaging clearer. The stage is a rough, self-rotating diffuse reflective black platform, which can effectively reduce light reflection and avoid interference with subsequent image processing.

[0165] The present invention provides an online detection method and system for whole polished rice rate based on machine vision. The method adopts an industrial camera to collect rice images in real time, detects rice shape parameters based on image processing technology, uses a neural network model to intelligently identify adhesion areas, combines multiple distance transformation watersheds and angle-limited concave point matching methods to achieve accurate segmentation of adhered rice grains, achieves accurate identification of whole polished rice, and thus improves the measurement accuracy of the whole polished rice rate. On this basis, the detection algorithm is deployed on Raspberry Pi hardware, and the measured whole polished rice rate parameters are locally stored, displayed and data transmitted in real time, thereby greatly improving the accuracy and efficiency of whole polished rice rate detection.

[0166] Obviously, the above embodiments are merely examples for the purpose of clear explanation and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived therefrom are still within the scope of protection of the present invention.

Claims

1. A machine vision-based online detection method for whole polished rice rate, characterized in that: include: Get the initial binary image of the rice image; After obtaining each feature parameter in the initial binary image according to the shape feature of rice in the initial binary image, the feature parameters are input into the BP neural network to screen the rice grain adhesion area in the initial binary image to obtain the binary image to be segmented; The multiple distance transform watershed segmentation method is used to segment each rice grain adhesion area in the binary image to be segmented, and a first segmentation result image is obtained, including: Performing distance transformation, normalization, and binarization processing on the binary image to be segmented to obtain a distance transformation binary image; If there are rice grain adhesion areas in the distance transform binary image, then each rice grain adhesion area in the distance transform binary image is screened to obtain a primary screening image; if there are rice grain adhesion areas in the primary screening image after the distance transform, then each rice grain adhesion area in the primary screening image after the distance transform is screened to obtain a secondary screening image; and so on, until there are no rice grain adhesion areas in the obtained screening image after the distance transform, and the screening image obtained at this time is used as the target image; based on the distance transform binary image and the target image, the foreground area mark of the distance transform binary image is determined to obtain a foreground area map; If there is no rice grain adhesion area in the distance transform binary image, then based on the distance transform binary image, determine the foreground area mark of the distance transform binary image to obtain a foreground area map; The foreground region map is combined with the background region mark of the binary image to be segmented to determine the foreground region boundary and obtain a complete marked image; based on the complete marked image, a first segmentation result map is obtained by a watershed transformation method; The minimum circumscribed rectangle method is used to obtain the length of each rice grain in the rice image based on the segmentation result image, thereby calculating the whole polished rice rate of the rice image.

2. A machine vision-based online detection method for whole polished rice rate according to claim 1, characterized in that: After obtaining the first segmentation result map, the following steps are also included: If there is still a rice grain adhesion area in the first segmentation result image, the rice grain adhesion area in the first segmentation result image is segmented by using an angle-limited concave point matching segmentation method to obtain a second segmentation result image as a final segmentation result image, including: Screen the rice grain adhesion areas in the first segmentation result image to obtain each adhesion area image; use the ellipse structure element to perform closing operation on the current adhesion area image to obtain the adhesion closed area map corresponding to the current adhesion area image; use the Canny operator to perform edge detection on the adhesion closed area in the current adhesion closed area map to obtain the contour curve of the current adhesion closed area; perform difference operation on the current adhesion area image and its corresponding adhesion closed area map to obtain the concave area map corresponding to the current adhesion area image; perform skeletonization operation on the current concave area map to obtain the concave area skeletonization map corresponding to the current adhesion area image; find all the endpoints in the concave area skeletonization map corresponding to the current adhesion area image to obtain the skeleton endpoint map corresponding to the current adhesion area image; calculate the shortest distance from each endpoint in the current skeleton endpoint map to the contour curve of the current adhesion closed area, take the endpoint farthest from the contour curve of the current adhesion closed area as the concave point to be segmented, and obtain the concave point extraction map corresponding to the current adhesion area image; If the number of concave points in the current concave point extraction map exceeds two, any concave point in the current concave point extraction map is taken as the first concave point, and a coordinate system is established with the first concave point as the coordinate origin; after respectively calculating the slope values ​​between the first concave point and the first skeleton endpoint and the second skeleton endpoint in the same concave region where the first concave point is located, respectively calculate the angle between the line connecting the first skeleton endpoint and the first concave point and the positive direction of the y-axis of the coordinate system, and the angle between the line connecting the second skeleton endpoint and the first concave point and the positive direction of the y-axis of the coordinate system, to obtain the limited angle range of the first concave point; select any concave point in the current concave point extraction map except the first concave point as the second concave point; if the angle between the line connecting the second concave point and the first concave point and the positive direction of the y-axis of the coordinate system is within the limited angle range of the first concave point, If the distance between the second concave point and the first concave point is the shortest, then it is determined that the second concave point matches the first concave point successfully, and the concave point pair to be segmented corresponding to the first concave point is obtained, and the concave point pair to be segmented corresponding to the first concave point is connected; if the angle between the line connecting the second concave point and the first concave point and the positive direction of the y-axis of the coordinate system is not within the limited angle range of the first concave point, then it is determined that the second concave point matches the first concave point unsuccessfully; the concave point pairs to be segmented corresponding to each concave point are obtained in turn, and the concave point pairs to be segmented corresponding to each concave point are connected to obtain the segmented image corresponding to the current adhesion area image; wherein, the direction of any connecting line passing through the first concave point is from any skeleton endpoint / any concave point to the first concave point; any angle is the angle between the positive direction of any connecting line and the positive direction of the y-axis of the coordinate system; If the number of concave points in the current concave point extraction map is equal to 2, the two concave points in the current concave point extraction map are paired and connected to obtain a segmented image corresponding to the current adhesion area image; Each adhesion region image is segmented in turn to obtain a segmented image corresponding to each adhesion region image; based on the segmented image corresponding to each adhesion region image, a second segmentation result image is obtained as a final segmentation result.

3. A machine vision-based online detection method for whole polished rice rate according to claim 2, characterized in that: The method uses any concave point in the current concave point extraction map as the first concave point and uses the first concave point as the coordinate origin to establish a coordinate system; Calculating the slope values ​​between the first concave point and the first skeleton endpoint and the second skeleton endpoint in the same concave region respectively includes: Taking any concave point in the current concave point extraction graph as the first concave point, and taking the first concave point as the coordinate origin, a coordinate system is established; wherein the coordinate of the first concave point is (0,0); Based on the current coordinate system, the coordinates of the first skeleton endpoint are preset to be (x1, y1), and the coordinates of the second skeleton endpoint are preset to be (x2, y2); wherein the first skeleton endpoint and the second skeleton endpoint are two skeleton endpoints in the same concave area as the first concave point; Based on the coordinates of the first concave point and the coordinates of the first skeleton endpoint, the slope value k1 of the first concave point and the first skeleton endpoint in the same region is calculated, and the expression is: k1=y1 / x1; Based on the coordinates of the first concave point and the coordinates of the second skeleton endpoint, the slope value k2 of the first concave point and the second skeleton endpoint in the same region is calculated, and the expression is: k2=y2 / x2.

4. A machine vision-based online detection method for whole polished rice rate according to claim 3, characterized in that: Based on the slope value between the first concave point and the first skeleton endpoint and the second skeleton endpoint in the same concave region where the first concave point is located, the angle between the line connecting the first skeleton endpoint and the first concave point and the positive direction of the y-axis of the coordinate system and the angle between the line connecting the second skeleton endpoint and the first concave point and the positive direction of the y-axis of the coordinate system are calculated respectively, and the limited angle range of the first concave point is obtained, which includes: Based on the slope value k1 between the first concave point and the first skeleton endpoint in the same concave region, the angle θ1 between the line connecting the first skeleton endpoint and the first concave point and the positive direction of the y-axis of the coordinate system is calculated, and the expression is: θ1 = arctank1-π; Based on the slope value k2 between the first concave point and the second skeleton endpoint in the same concave region, the angle θ2 between the line connecting the second skeleton endpoint and the first concave point and the positive direction of the y-axis of the coordinate system is calculated, and the expression is: θ2 = arctank2-π; Based on the angle θ1 between the line connecting the first skeleton endpoint and the first concave point and the positive direction of the y-axis of the coordinate system, and the angle θ2 between the line connecting the second skeleton endpoint and the first concave point and the positive direction of the y-axis of the coordinate system, the limited angle range of the first concave point is [θ1,θ2]; among which, θ1<θ2 is preset.

5. A machine vision-based online detection method for whole polished rice rate according to claim 1, characterized in that: The rice shape characteristics include: the area, perimeter, aspect ratio, circularity and convexity of the rice grain adhesion area, and the area, perimeter, aspect ratio, circularity and convexity of the rice grain non-adhesion area; the characteristic parameters include: the area ratio, perimeter ratio, shape factor and depression distance ratio between the rice grain adhesion area and the rice grain non-adhesion area.

6. A machine vision-based online detection method for whole polished rice rate according to claim 1, characterized in that: The method of calculating the polished rice rate of the rice image comprises: Based on the image size of the initial binary image and the pixel size of each rice grain in the initial binary image, the area of ​​each rice grain in the rice image is calculated, and its expression is: Among them, S n represents the area of ​​the nth rice grain in the rice image; h n (i n ,j n ) represents the pixel size of the nth rice grain in the rice image; M×N represents the image size of the initial binary image; Based on the area of ​​each rice grain in the rice image, the area of ​​all rice grains in the rice image is calculated, and its expression is: S all =∑ n＝1 S n ; According to the area of ​​all rice grains in the rice image and the area of ​​all polished rice grains in the rice image, the polished rice rate P of the rice image is calculated, and its expression is: Among them, S w Represents the area of ​​all whole rice grains in the rice image.

7. A machine vision-based online detection method for whole polished rice rate according to claim 1, characterized in that: The step of obtaining a rice image and preprocessing the rice image to obtain an initial binary image comprises: The grayscale rice image is enhanced by using Gamma transformation to obtain an enhanced grayscale rice image. The enhanced rice image is binarized by using the maximum inter-class variance method to separate the outline of each rice grain in the enhanced grayscale rice image from the background to obtain a binarized grayscale rice image. The binarized grayscale rice image is denoised to obtain an initial binarized image.

8. A machine vision-based online detection system for whole polished rice rate, characterized in that: include: An image acquisition device, used for acquiring rice images, comprises: A loading platform, which is equipped with a loading platform and a loading bracket; An industrial camera is installed on the top of the loading bracket and is equipped with an industrial lens for photographing the rice on the loading platform to obtain an image of the rice; A light source device is installed on the object carrier and is located between the object carrier platform and the industrial camera, and is used to illuminate the object carrier platform; A light source controller, which is connected to the light source device and is used to control the brightness of the light source in the light source device; An image processing device, used for processing a rice image and displaying the whole polished rice rate of the rice image, comprises: A processor is connected to the industrial camera in the image acquisition device through communication; the processor is used to receive the rice image, execute the online detection method of the whole polished rice rate based on machine vision according to any one of claims 1 to 7, and obtain the whole polished rice rate of the rice image; A display screen, which is connected to the processor via an interface and is used to display the polished rice rate of the rice image; A control device is connected to the processor in the image processing device; the control device is used for online detection and closed-loop control of whole polished rice grains.

9. A machine vision-based online detection system for whole polished rice rate according to claim 8, characterized in that: The processor is a computer or a Raspberry Pi operating device.

10. The machine vision-based online detection system for whole polished rice rate according to claim 8, characterized in that: The control device is connected to the processor via a serial port, a network port or a wireless method.

Images