Stone gradation detection method, device, medium and equipment based on image recognition

Through the stone grading detection method based on image recognition, the physical characteristics of stone particles are identified by video acquisition and example segmentation models, the problem of insufficient grading detection accuracy in the stone stacking state in the existing technology is solved, and the rapid and accurate stone is achieved Grading inspection.

CN118799349BActive Publication Date: 2024-11-15GUODIAN DADU RIVER JINCHUAN HYDROPOWER CONSTR CO LTD
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
CN202411261120.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-09-10
Publication Date
2024-11-15
Estimated Expiration
2044-09-10

AI Technical Summary

Technical Problem

In the prior art, the grading detection accuracy in the stone stacking state is insufficient, the manual screening efficiency is low and the accuracy is low, so it is impossible to achieve rapid grading detection in the stone stacking state.

Method used

Using the stone grading detection method based on image recognition, the video data during the dynamic sliding of stone particles is collected through the video acquisition device, pre-processing and instance segmentation model annotation, identify the physical characteristics of the stone particles, generate particle size and volume data, and then Implement grading inspection.

Benefits of technology

提高了石料级配检测的精度和效率,能够快速、准确地进行石料颗粒的级配检测,适用于不同粒径分布的石料,实现了从动态滑落到视频获取、轮廓识别、粒径及体积转换 The integrated process of algorithms.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118799349B_ABST
    Figure CN118799349B_ABST
Patent Text Reader

Abstract

The present disclosure relates to a method, device, medium and equipment for detecting stone gradation based on image recognition, the method comprising: obtaining a video of stone particles collected by a video acquisition device, preprocessing each frame image in the video of stone particles, generating a stone particle image of the stone particles, annotating the stone particle contours in the stone particle image through an instance segmentation model, generating a contour information image of the stone particles, the contour information image including the contour annotation information of the stone particles, identifying the physical properties of the stone particles in the contour information image based on the contour annotation information, generating stone particle size data and stone volume data of the stone particles, and generating a gradation detection result of the stone particles according to the stone particle size data and the stone volume data. Thus, the stone particles are quickly graded and detected, and the detection efficiency is improved, while generating an efficient and stable gradation detection result.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, and in particular, to a method, device, medium and equipment for detecting stone gradation based on image recognition. Background Art

[0002] Rockfill is used as the main material of water retaining dam in the construction of concrete face rockfill dam. The stability and anti-seepage capacity of rockfill dam are mainly determined by the characteristics of rockfill, such as mechanical properties and gradation state of rockfill. Rockfill with good gradation state and mechanical properties is very important for the main body of rockfill dam. The gradation detection technology of rockfill is a technology to detect the gradation state of rockfill. At present, the gradation detection by manual screening is not only inefficient, but also the detection level is completely determined by the detection personnel, resulting in low accuracy of rockfill gradation. Summary of the invention

[0003] In order to overcome the technical problem of low gradation detection accuracy in the related art, the present disclosure provides a stone gradation detection method, device, medium and equipment based on image recognition to solve the technical problem of insufficient accuracy in the rapid gradation detection of stones in a stacked state in the related art.

[0004] In order to achieve the above-mentioned object, the first aspect of the present disclosure provides a stone gradation detection method based on image recognition, comprising:

[0005] Obtaining a video of stone particles collected by a video acquisition device, wherein the video acquisition device is used to collect video data of the stone particles in a dynamic sliding process;

[0006] Preprocessing each frame image in the stone particle video to generate a stone particle image of the stone particles;

[0007] Annotating the stone particle contours in the stone particle image by using an instance segmentation model to generate a contour information image of the stone particles, wherein the contour information image includes contour annotation information of the stone particles;

[0008] Based on the contour annotation information, the physical properties of the stone particles in the contour information image are identified to generate stone particle size data and stone volume data of the stone particles;

[0009] According to the stone particle size data and the stone volume data, a gradation detection result of the stone particles is generated.

[0010] Optionally, the identifying the physical characteristics of the stone particles in the contour information image based on the contour annotation information to generate the stone particle size data and stone volume data of the stone particles includes:

[0011] Extracting contour quantification indexes and shape characteristic parameters of the contour annotation information in the contour information image to generate characteristic parameters of the stone particles;

[0012] The characteristic parameters are converted into particle size and volume to generate the stone particle size data and the stone volume data of the stone particles.

[0013] Optionally, the preprocessing of each frame image in the stone particle video to generate a stone particle image of the stone particles includes:

[0014] Determine the video frame rate information of the stone particle video and the sliding speed information of each stone particle in the stone particle video;

[0015] According to the video frame rate information and the sliding speed information, frame extraction processing is performed on the stone particle video to extract a first stone particle image of each stone particle in the stone particle video;

[0016] The first stone particle image is preprocessed to generate the stone particle image.

[0017] Optionally, preprocessing the first stone particle image to generate the stone particle image includes:

[0018] Performing a histogram equalization process on the first stone particle image to rearrange the distribution of pixel point values ​​in the first stone particle image to generate a second stone particle image of the stone particles;

[0019] Based on a preset correction algorithm, motion blur correction is performed on the stone particles in the second stone particle image to generate the stone particle image.

[0020] Optionally, preprocessing the first stone particle image to generate the stone particle image includes:

[0021] Performing blur processing on the first stone particle image by a preset low-pass filter device to generate a blurred image of the stone particles;

[0022] Subtracting the pixel value of each pixel point in the first stone particle image from the pixel value of each pixel point in the blurred image to generate a mask image of the stone particles;

[0023] The pixel value of each pixel point in the first stone particle image is added to the pixel value of each pixel point in the mask image to generate the stone particle image.

[0024] Optionally, the step of labeling the stone particle contours in the stone particle image by using an instance segmentation model to generate a contour information image of the stone particles includes:

[0025] Converting the stone particle image into a stone feature image through the backbone network of the instance segmentation model;

[0026] A network module is established through a preset area of ​​the instance segmentation model, stone particles in the stone feature image are framed, and a candidate frame of the stone particles is generated;

[0027] Classifying the candidate frames pixel by pixel to generate mask information of the stone particles;

[0028] The mask information is marked at the corresponding position of the stone particle image to generate the contour information image.

[0029] Optionally, the identifying the physical characteristics of the stone particles in the contour information image based on the contour annotation information to generate the stone particle size data and stone volume data of the stone particles includes:

[0030] Determine the perimeter data, area data, and major and minor axis data of the stone particles according to the mask information corresponding to the stone particles in the contour information image;

[0031] According to the perimeter data, the area data and the major and minor axis data, the stone particle size data is determined by using an equivalent particle size calculation formula;

[0032] The stone volume data is determined according to the stone particle size data.

[0033] According to a second aspect of an embodiment of the present disclosure, a stone gradation detection device based on image recognition is provided, comprising:

[0034] An acquisition module, used to acquire the video of the stone particles acquired by the video acquisition device, wherein the video acquisition device is used to acquire video data of the stone particles in the dynamic sliding process;

[0035] A first generating module is used to pre-process each frame image in the stone particle video to generate a stone particle image of the stone particles;

[0036] A second generation module is used to mark the contours of the stone particles in the stone particle image by using an instance segmentation model to generate a contour information image of the stone particles, wherein the contour information image includes contour marking information of the stone particles;

[0037] A third generating module is used to identify the physical characteristics of the stone particles in the contour information image based on the contour annotation information, and generate stone particle size data and stone volume data of the stone particles;

[0038] An execution module is used to generate a gradation detection result of the stone particles according to the stone particle size data and the stone volume data.

[0039] According to a third aspect of an embodiment of the present disclosure, there is provided an electronic device, including:

[0040] a memory having a computer program stored thereon;

[0041] A processor is used to execute the computer program in the memory to implement the steps of the stone gradation detection method based on image recognition in any one of the first aspects of the present disclosure.

[0042] According to a fourth aspect of an embodiment of the present disclosure, there is provided a non-temporary computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the stone grading detection method based on image recognition as described in any one of the first aspects of the present disclosure.

[0043] Through the above method, the video of stone particles collected by the video acquisition device is obtained. The video acquisition device is used to collect video data of the dynamic sliding process of stone particles. Each frame image in the stone particle video is preprocessed to generate a stone particle image of the stone particles. The stone particle contours in the stone particle image are annotated by the instance segmentation model to generate a contour information image of the stone particles. The contour information image includes the contour annotation information of the stone particles. Based on the contour annotation information, the physical characteristics of the stone particles in the contour information image are identified to generate the stone particle size data and stone volume data of the stone particles. According to the stone particle size data and the stone volume data, the gradation detection result of the stone particles is generated. In this way, the stone particles are quickly graded and detected, and the detection efficiency is improved, while generating efficient and stable gradation detection results. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The accompanying drawings are used to provide a further understanding of the present disclosure and constitute a part of the specification. Together with the following specific embodiments, they are used to explain the present disclosure but do not constitute a limitation of the present disclosure. In the accompanying drawings:

[0045] Figure 1 The present invention is a flow chart of a method for detecting stone gradation based on image recognition according to an exemplary embodiment.

[0046] Figure 2 The figure is a top view of a stone gradation detection device according to an exemplary embodiment.

[0047] Figure 3 It is a cross-sectional view of a stone gradation detection device according to an exemplary embodiment.

[0048] Figure 4 The figure is a side view of a stone gradation detection device according to an exemplary embodiment.

[0049] Figure 5 The figure is a flow chart of a method for generating a stone particle image according to an exemplary embodiment.

[0050] Figure 6 The figure is a flow chart of a method for generating a contour information image according to an exemplary embodiment.

[0051] Figure 7 It is a schematic diagram of a stone gradation detection method based on image recognition according to an exemplary embodiment.

[0052] Figure 8 It is a block diagram of a stone gradation detection device based on image recognition according to an exemplary embodiment.

[0053] Fig. 9 is a block diagram of an electronic device 900 according to an exemplary embodiment.

[0054] Description of Reference Numerals

[0055] Stone grading detection device based on image recognition-100; acquisition module-110; first generation module-120; second generation module-130; third generation module-140; execution module-150; electronic device-900; processor-901; memory-902; multimedia component-903; input / output interface-904; communication component-905; industrial camera-11; slide rail-12; fir board-13; steel plate-14; fixing pin-15; bottom baffle-16. DETAILED DESCRIPTION

[0056] The specific implementation of the present disclosure is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described herein is only used to illustrate and explain the present disclosure, and is not used to limit the present disclosure.

[0057] It should be noted that all actions of acquiring signals, information or data in the present disclosure are carried out in compliance with the relevant data protection laws and policies of the country where the device is located and with the authorization given by the owner of the corresponding device.

[0058] In the related art, the grading detection method based on the image segmentation method has certain limitations when performing overall grading detection on stones in a stacked state. Since the stones in the stacked state are blocked by the surface stones, it is impossible to collect a complete image of the stones. Therefore, only the surface layer of stones is identified. Moreover, since there is a certain difference between the surface grading and the overall grading in the stacked state, the surface grading cannot represent the overall grading. Therefore, it is impossible to achieve rapid grading detection of stones in a stacked state.

[0059] In view of this, the present invention proposes a stone gradation detection method based on image recognition. Figure 1 FIG. 1 is a flow chart showing a method for detecting stone gradation based on image recognition according to an exemplary embodiment. Figure 1 As shown, the method is executed by the terminal and includes the following steps.

[0060] In step S11, a video of stone particles captured by a video capture device is obtained, where the video capture device is used to capture video data of the stone particles in a dynamic sliding process.

[0061] For example, the video acquisition device in this embodiment is a device for capturing the dynamic sliding process of rocks, which is used to capture video data of the process of piled rocks sliding from a high place. For example, the bottom baffle of the capture device is fixed on the latch, the rocks start to slide from the top of the device, and the images of the rocks are photographed by an industrial camera. The conversion relationship between the real size and the image size is obtained according to the length of the telescopic rod, so as to obtain the video data of the stone particles in the process of the dynamic sliding of the stone particles.

[0062] It should be noted that in order to ensure the clarity of the stone particle video and facilitate the subsequent detection of the particle size, volume, contour and gradation of the stone particles in the stone particle video, in this embodiment, the capture device parameters of 1080p and a frame rate of 240FPS can be used to collect the stone image during the stone sliding process.

[0063] Use a device to capture the dynamic sliding process of stones to obtain a video of stone particles sliding down. The main body of the stone image acquisition device is a chute. A baffle is provided at the bottom of the chute, and there are sliding tracks on both sides. An industrial camera is provided between the two tracks. The stone slides out through the top of the chute. This device can capture images of stone paving at any position and any height of the chute and obtain the conversion relationship between the image size and the actual size. The placement height of the chute can be adjusted according to the actual stone particle situation of the stone pile.

[0064] Figure 2 FIG. 1 is a top view of a stone gradation detection device according to an exemplary embodiment. Figure 2As shown, the device includes an inclined chute, and the piled stone slides out from the top of the chute. An industrial camera 11 is set at the corresponding position of the inclined plane above the chute as a device for capturing the dynamic sliding process of the stone. The device carrying the industrial camera 11 can slide on the slide rail 12 to adjust the position of the industrial camera 11 when performing image acquisition. The industrial camera 11 is used to capture the video of the stone particles sliding down the chute. For example, the inclination angle between the chute and the horizontal plane can be set to 39°, and fir boards are set on both sides of the chute as baffles for the stone particles to prevent the stone particles from rolling out of the chute during the rolling process.

[0065] Figure 3 FIG. 1 is a cross-sectional view of a stone gradation detection device according to an exemplary embodiment. Figure 3 As shown, a slide rail 12 and a fir board 13 are arranged on the side of the chute, a steel plate 14 is arranged at the bottom of the chute, and a column is arranged every 1m in the chute to fix the oblique fir board 13.

[0066] Figure 4 FIG. 1 is a side view of a stone gradation detection device according to an exemplary embodiment. Figure 4 As shown, a fixing pin 15 and a bottom baffle 16 are provided at the bottom of the chute to prevent the stones from sliding to other positions, so as to facilitate the collection of the piled stones.

[0067] In step S12, each frame image in the stone particle video is preprocessed to generate a stone particle image of the stone particles.

[0068] For example, in order to improve the recognition accuracy of stone particles, it is necessary to pre-process each frame image in the stone particle video to eliminate interference factors in each frame image, so as to more accurately recognize the stone particle images in the stone particle video.

[0069] Figure 5 FIG. 1 is a flow chart of a method for generating a stone particle image according to an exemplary embodiment. Figure 5 As shown, in some embodiments, the above step S12 includes:

[0070] In step S121, the video frame rate information of the stone particle video and the sliding speed information of each stone particle in the stone particle video are determined.

[0071] In step S122, the stone particle video is subjected to frame extraction processing according to the video frame rate information and the sliding speed information, so as to extract the first stone particle image of each stone particle in the stone particle video.

[0072] In step S123, the first stone particle image is preprocessed to generate a stone particle image.

[0073] For example, in this embodiment, frame extraction is performed on the video according to the sliding speed of each stone particle in the stone particle video and the sampled video frame rate, and representative frames of the stone particles in the stone particle video are extracted. The number of frames in which the stone particles appear in the stone particle video is calculated, and then the extraction interval frame number of each representative frame is determined to ensure that the stone appears only once in each representative frame, thereby obtaining the first stone particle image corresponding to each stone particle in the stone particle video.

[0074] Optionally, in some embodiments, the above step S123 includes:

[0075] Performing histogram equalization processing on the first stone particle image to rearrange the distribution of pixel values ​​in the image to generate a second stone particle image of the stone particles;

[0076] Based on a preset correction algorithm, motion blur correction is performed on the stone particles in the second stone particle image to generate a stone particle image.

[0077] For example, in this embodiment, the histogram equalization processing is to rearrange the distribution of pixel values ​​in the image to make the distribution more uniform. Based on the particularity of image acquisition in the first stone particle image, the overexposed and dark parts in the stone particle video are clarified to make the overexposed and dark parts clear, thereby generating a second stone particle image of the stone particles.

[0078] Based on a preset correction algorithm, motion blur correction is performed on the stone particles in the second stone particle image, thereby generating a stone particle image corresponding to the stone particles.

[0079] For example, when a stone particle enters a video shooting area, the initial speed of the stone particle when entering the video shooting area can be determined based on the video frame rate information between each frame image in the stone particle video and the displacement of the stone particle in the stone particle video. , determine the actual size of the video area as ,in, is the height of the video area, is the width of the video area, representing the pixel size of the frame image: ,in, , Represent the row and column data of the image matrix respectively. Assuming that the shooting frame rate of the device capturing the dynamic sliding process of the stone is F, then the horizontal coordinate of the stone in the video image is The velocity at position is:

[0080]

[0081] in, is the free fall acceleration, From the edge of the image to the horizontal coordinate The distance at the location, for The speed at the position, is the initial velocity of the stone particles, The calculation formula is:

[0082]

[0083] According to the shooting frame rate of the capture device during the dynamic sliding of the stone, the time taken by the capture device to shoot one frame of picture is determined as:

[0084]

[0085] Where F is the shooting frame rate, The time taken by the capture device to capture a frame of picture can be used to determine the time period of the stone particles in the video data. The falling height inside is:

[0086]

[0087] in, for The falling height of the internal stone particles, assuming that the maximum pixel horizontal coordinate and the minimum pixel horizontal coordinate of the correction object in the stone particle image are , , then the longitudinal motion blur correction formula is:

[0088]

[0089] in, represents the image after longitudinal motion blur correction, is the horizontal coordinate of the j-th pixel point of the correction object.

[0090] Optionally, in some embodiments, the above step S123 includes:

[0091] The first stone particle image is blurred by a preset low-pass filter device to generate a blurred image of the stone particles;

[0092] Subtracting the pixel value of each pixel point in the first stone particle image from the pixel value of each pixel point in the blurred image to generate a mask image of the stone particles;

[0093] The pixel value of each pixel point in the first stone particle image is added to the pixel value of each pixel point in the mask image to generate a stone particle image.

[0094] For example, in this embodiment, the first stone particle image is subjected to unsharp mask processing. The first stone particle image is first blurred by a preset low-pass filter device to generate a blurred image of the stone particles. The pixel value of each pixel in the first stone particle image is subtracted from the pixel value of each pixel in the blurred image to generate a mask image of the stone particles. The pixel subtraction is based on the calculation of the pixel value of the pixel at the same position in the first stone particle image and the blurred image to generate a mask image of the stone particles. The pixel value of each pixel in the first stone particle image is added to the pixel value of each pixel in the mask image to generate a preprocessed stone particle image.

[0095] For example, a low-pass filter is used to blur the original image to obtain a blurred image. The blur operation can be represented by the following convolution operation:

[0096]

[0097] in, represents a blurred image, is the original image, is the convolution kernel in the low-pass filter. Subtract the original image from the blurred image to get a mask image:

[0098]

[0099] in, Represents the mask image. Add the original image to the mask image to get the enhanced image. , where the enhanced image It can be calculated as follows:

[0100]

[0101] In step S13, the stone particle contours in the stone particle image are annotated by using the instance segmentation model to generate a stone particle contour information image, wherein the contour information image includes the stone particle contour annotation information.

[0102] For example, the preprocessed stone particle image is input into the instance segmentation model, and the contours in the stone particles are annotated based on the instance segmentation model to generate a contour information image of the stone particles, wherein the contour information image includes contour annotation information of the stone particles.

[0103] The instance segmentation model used in this embodiment is Mask-RCNN (Mask Region-based Convolutional Neural Network). The original image is converted into a feature map through the backbone network, the region construction network model generates a candidate frame, and the output module classifies the candidate frame pixel by pixel to obtain the mask of the stone particles. Finally, the stone particle image with the mask is exported to generate a contour information image annotated with contour annotation information.

[0104] Figure 6 FIG. 1 is a flow chart of a method for generating a contour information image according to an exemplary embodiment. Figure 6 As shown, the above step S13 includes the following steps.

[0105] In step S131, the stone particle image is converted into a stone feature image through the backbone network of the instance segmentation model.

[0106] In step S132, a network module is established through a preset area of ​​the instance segmentation model, the stone particles in the stone feature image are framed, and a candidate frame of the stone particles is generated.

[0107] In step S133, the candidate boxes are classified pixel by pixel to generate mask information of the stone particles.

[0108] In step S134, the mask information is marked at the corresponding position of the stone particle image to generate a contour information image.

[0109] For example, in this embodiment, the stone particle image is converted into a stone feature image through the backbone network in the instance segmentation model, and the network module is established through the preset area of ​​the instance segmentation model to select the stone particles in the stone feature image and generate a candidate frame of the stone particles. The candidate frame is classified pixel by pixel based on the output module to obtain the mask information of the stone particles, and then the mask information is annotated at the corresponding position of the stone particle image to generate a contour information image of the stone particles. The contour information of the stone particles is annotated at the corresponding position of the stone particles in the contour information image.

[0110] In step S14, the physical properties of the stone particles in the contour information image are identified based on the contour marking information, and the stone particle size data and stone volume data of the stone particles are generated.

[0111] For example, in this embodiment, the shape characteristic parameters and contour quantification indicators of stone particles in the contour information image of the stone particles are first extracted, the extracted morphological characteristic parameters are converted into particle size and the total volume of stone particles in each particle size range is converted to generate stone particle size data and stone volume data of the stone particles.

[0112] Optionally, in some embodiments, the above step S14 includes:

[0113] Extracting contour quantification indexes and shape characteristic parameters of contour annotation information in the contour information image to generate characteristic parameters of stone particles;

[0114] The characteristic parameters are converted into particle size and volume to generate stone particle size data and stone volume data of the stone particles.

[0115] For example, in this embodiment, the shape characteristic parameters and contour quantification index of the stone particles are extracted from the stone contour, and the extracted morphological characteristic parameters are converted into particle size and the volume of the stone particles in each particle size range is converted. For example, the contour quantification index is calculated using the OpenCV image processing library, and the perimeter of the stone particle can be calculated using the cv2.arcLength function, which is calculated by recursively adding the distances of the coordinates of adjacent points in the contour. The area of ​​the stone particle mask is calculated using the cv2.contourArea function; the fitted ellipse of the stone particle mask can be calculated using the cv2.fitEllipse function.

[0116] Optionally, in some embodiments, the above step S14 includes:

[0117] According to the mask information corresponding to the stone particles in the contour information image, the perimeter data, the area data, and the major and minor axis data of the stone particles are determined;

[0118] According to the perimeter data, area data and major and minor axis data, the stone particle size data is determined by using the particle size calculation equivalent formula;

[0119] Determine the stone volume data based on the stone particle size data.

[0120] For example, in this embodiment, when performing particle size conversion, it is necessary to calculate the perimeter, area, major and minor axes, and equivalent particle size of the mask in the stone particle image. By setting the rectangular coordinates of the scattered points on the edge line as , and the scattered points numbered 1-n are arranged counterclockwise around the stone particles, then the formula for calculating the circumference of the stone particles is:

[0121]

[0122] in, is the horizontal coordinate of the first scattered point, is the vertical coordinate of the first scattered point, is the horizontal coordinate of the ith scatter point, is the ordinate of the i-th scattered point, is the horizontal coordinate of the i+1th scattered point, is the ordinate of the i+1th scattered point, is the horizontal coordinate of the nth scattered point, is the ordinate of the nth scattered point, the term under the first square root represents the straight-line distance between adjacent scattered points, and the term under the second square root represents the straight-line distance between the first scattered point and the last scattered point. The two terms are added together to form the perimeter of the closed curve, where n is the number of scattered point coordinates on the edge line of the stone particle, and P is the perimeter of the stone particle.

[0123] Based on the above calculation method, the area calculation formula for stone particles is determined as follows:

[0124]

[0125] Where n is the number of scattered coordinates on the edge line of the stone particle, S is the area of ​​the stone particle, Pixel The rectangular coordinates of Pixel The rectangular coordinates of Pixel The rectangular coordinates of Pixel The rectangular coordinates of .

[0126] The calculation formula for the long axis of stone particles is:

[0127]

[0128] The corresponding calculation formula for the minor axis of the stone particles is:

[0129]

[0130] Based on the above calculation formulas for the major axis and minor axis, the equivalent formula for calculating the particle size of stone particles is:

[0131]

[0132] in, is the particle size of the stone particles. After the stone particle size data of the stone particles are determined by the above method, the stone volume data of the stone particles are calculated according to the particle size.

[0133] In step S15, a gradation detection result of the stone particles is generated according to the stone particle size data and the stone volume data.

[0134] For example, in this embodiment, the gradation of stone particles is detected based on the stone particle size data and the stone volume data to generate the gradation detection result of the stone particles. For example, the gradation calculation formula of three-dimensional stone particles is:

[0135]

[0136] in, is the minor semi-axis of the stone particle, is the geometric projection area of ​​the stone particles in the image.

[0137]

[0138] Where n represents the total number of stone particles, Indicates that the particle size is smaller than The percentage of stone mass.

[0139] For example, the particle sizes of stones are arranged from large to small, with 100mm, 80mm, 60mm, 40mm, 20mm, and 10mm as boundaries to generate a stone particle sequence. From the beginning of the stone particle sequence to each boundary, the volume of all stones in the range is summed and divided by the sum of the volumes of all stones to calculate the pass rate of each particle size boundary point. The horizontal axis of the grading curve is the stone particle size, and the vertical axis is the pass rate of stones passing the particle size. The position of each boundary point can be plotted, and adjacent boundary points can be connected with straight lines to generate the grading test results of stone particles.

[0140] Through the above method, the rockfill grading detection device based on image recognition can quickly detect the grading of stone particles in a dynamic sliding state, and has good adaptability to stones with different particle size distributions. In addition, the device realizes an integrated process from the dynamic sliding of stones to video acquisition, representative frame acquisition, contour recognition, particle size and volume conversion algorithm and finally grading curve drawing, without the need for multiple processes, and the photographed stones can be quickly collected and loaded.

[0141] Figure 7 FIG. 1 is a schematic diagram showing a method for detecting stone gradation based on image recognition according to an exemplary embodiment. Figure 7 As shown, the method includes the following steps.

[0142] (1) Use a device to capture the dynamic sliding process of stones to obtain a video of stone particles during the sliding process;

[0143] (2) Preprocessing the acquired stone particle video to obtain a stone particle image;

[0144] (3) Input the preprocessed stone particle image into the instance segmentation model to obtain an image containing the stone particle contour information;

[0145] (4) The contour information of the stone particles is subjected to contour quantification indexes and contour feature extraction of shape characteristic parameters, and the extracted characteristic parameters are converted into particle size and total volume of stone within each particle size range;

[0146] (5) The characteristic parameters extracted in the above step (4) are used to convert the particle size and volume obtained after the particle size and volume conversion into the gradation of three-dimensional stone particles and draw a gradation curve.

[0147] Through the above method, a method and device for detecting the gradation of rockfill materials based on image recognition are proposed. The problem of insufficient accuracy in the rapid gradation detection of stones in the stacked state can be effectively overcome by photographing the dynamic sliding process of stone particles. Stone particles with a large particle size span can be effectively identified and gradation calculated. And for the photographed stones, the calculation and loading can be done immediately after the photographing.

[0148] Figure 8 is a block diagram of a stone gradation detection device based on image recognition according to an exemplary embodiment. Figure 8 As shown, the stone gradation detection device 100 based on image recognition includes:

[0149] An acquisition module 110 is used to acquire a video of stone particles acquired by a video acquisition device, where the video acquisition device is used to acquire video data of the stone particles in a dynamic sliding process;

[0150] The first generating module 120 is used to pre-process each frame image in the stone particle video to generate a stone particle image of the stone particles;

[0151] The second generating module 130 is used to mark the stone particle contours in the stone particle image by using the instance segmentation model to generate a stone particle contour information image, wherein the contour information image includes the stone particle contour marking information;

[0152] The third generating module 140 is used to identify the physical characteristics of the stone particles in the contour information image based on the contour annotation information, and generate the stone particle size data and stone volume data of the stone particles;

[0153] The execution module 150 is used to generate a gradation detection result of stone particles according to the stone particle size data and the stone volume data.

[0154] Optionally, the third generating module 140 is used to:

[0155] Extracting contour quantification indexes and shape characteristic parameters of contour annotation information in the contour information image to generate characteristic parameters of stone particles;

[0156] The characteristic parameters are converted into particle size and volume to generate stone particle size data and stone volume data of the stone particles.

[0157] Optionally, the first generating module 120 includes:

[0158] A determination submodule is used to determine the video frame rate information of the stone particle video and the sliding speed information of each stone particle in the stone particle video;

[0159] An extraction submodule, used for performing frame extraction processing on the stone particle video according to the video frame rate information and the sliding speed information, so as to extract the first stone particle image of each stone particle in the stone particle video;

[0160] The first generating submodule is used to preprocess the first stone particle image to generate a stone particle image.

[0161] Optionally, generate submodules for:

[0162] Performing histogram equalization processing on the first stone particle image to rearrange the distribution of pixel point values ​​in the first stone particle image to generate a second stone particle image of the stone particles;

[0163] Based on a preset correction algorithm, motion blur correction is performed on the stone particles in the second stone particle image to generate a stone particle image.

[0164] Optionally, the first generation submodule is used to:

[0165] The first stone particle image is blurred by a preset low-pass filter device to generate a blurred image of the stone particles;

[0166] Subtracting the pixel value of each pixel point in the first stone particle image from the pixel value of each pixel point in the blurred image to generate a mask image of the stone particles;

[0167] The pixel value of each pixel point in the first stone particle image is added to the pixel value of each pixel point in the mask image to generate a stone particle image.

[0168] Optionally, the second generating module 130 includes:

[0169] A conversion submodule, used for converting the stone particle image into a stone feature image through the backbone network of the instance segmentation model;

[0170] The second generation submodule is used to establish a network module through a preset area of ​​the instance segmentation model, select the stone particles in the stone feature image, and generate a candidate frame of the stone particles;

[0171] The third generation submodule is used to classify the candidate frames pixel by pixel and generate mask information of stone particles;

[0172] The fourth generating submodule is used to mark the mask information at the corresponding position of the stone particle image to generate a contour information image.

[0173] Optionally, the third generating module 140 is used to:

[0174] According to the mask information corresponding to the stone particles in the contour information image, the perimeter data, the area data, and the major and minor axis data of the stone particles are determined;

[0175] According to the perimeter data, area data and major and minor axis data, the stone particle size data is determined by using the particle size calculation equivalent formula;

[0176] Determine the stone volume data based on the stone particle size data.

[0177] Through the above method, the video of stone particles collected by the video acquisition device is obtained. The video acquisition device is used to collect video data of the dynamic sliding process of stone particles. Each frame image in the stone particle video is preprocessed to generate a stone particle image of the stone particles. The stone particle contours in the stone particle image are annotated by the instance segmentation model to generate a contour information image of the stone particles. The contour information image includes the contour annotation information of the stone particles. Based on the contour annotation information, the physical characteristics of the stone particles in the contour information image are identified to generate the stone particle size data and stone volume data of the stone particles. According to the stone particle size data and the stone volume data, the gradation detection result of the stone particles is generated. In this way, the stone particles are quickly graded and detected, and the detection efficiency is improved, while generating efficient and stable gradation detection results.

[0178] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0179] Fig. 9 FIG. 9 is a block diagram of an electronic device 900 according to an exemplary embodiment. Fig. 9 As shown, the electronic device 900 may include: a processor 901 and a memory 902. The electronic device 900 may also include one or more of a multimedia component 903, an input / output interface 904, and a communication component 905.

[0180] The processor 901 is used to control the overall operation of the electronic device 900 to complete all or part of the steps in the above-mentioned stone gradation detection method based on image recognition. The memory 902 is used to store various types of data to support the operation of the electronic device 900, and these data may include, for example, instructions for any application or method used to operate on the electronic device 900, and application-related data, such as contact data, sent and received messages, pictures, audio, video, etc. The memory 902 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (Static Random Access Memory, referred to as SRAM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, referred to as EEPROM), erasable programmable read-only memory (Erasable Programmable Read-Only Memory, referred to as EPROM), programmable read-only memory (Programmable Read-Only Memory, referred to as PROM), read-only memory (Read-Only Memory, referred to as ROM), magnetic memory, flash memory, magnetic disk or optical disk. The multimedia component 903 may include a screen and an audio component. The screen may be, for example, a touch screen, and the audio component is used to output and / or input audio signals. For example, the audio component may include a microphone, which is used to receive external audio signals. The received audio signal may be further stored in the memory 902 or sent through the communication component 905. The audio component also includes at least one speaker for outputting audio signals. The input / output interface 904 provides an interface between the processor 901 and other interface modules, and the above-mentioned other interface modules may be keyboards, mice, buttons, etc. These buttons may be virtual buttons or physical buttons. The communication component 905 is used for wired or wireless communication between the electronic device 900 and other devices. Wireless communication, such as Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G, 4G, NB-IOT, eMTC, or other 5G, etc., or a combination of one or more of them, is not limited here. Therefore, the corresponding communication component 905 may include: Wi-Fi module, Bluetooth module, NFC module, etc.

[0181] In an exemplary embodiment, the electronic device 900 can be implemented by one or more application specific integrated circuits (ASIC), digital signal processors (DSP), digital signal processing devices (DSPD), programmable logic devices (PLD), field programmable gate arrays (FPGA), controllers, microcontrollers, microprocessors or other electronic components to execute the above-mentioned stone grading detection method based on image recognition.

[0182] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided, and when the program instructions are executed by a processor, the steps of the above-mentioned stone gradation detection method based on image recognition are implemented. For example, the computer-readable storage medium can be the above-mentioned memory 902 including program instructions, and the above-mentioned program instructions can be executed by the processor 901 of the electronic device 900 to complete the above-mentioned stone gradation detection method based on image recognition.

[0183] In another exemplary embodiment, a computer program product is also provided. The computer program product includes a computer program that can be executed by a programmable device, and the computer program has a code portion for executing the above-mentioned stone gradation detection method based on image recognition when executed by the programmable device.

[0184] The preferred embodiments of the present disclosure are described in detail above in conjunction with the accompanying drawings; however, the present disclosure is not limited to the specific details in the above embodiments. Within the technical concept of the present disclosure, a variety of simple modifications can be made to the technical solution of the present disclosure, and these simple modifications all fall within the protection scope of the present disclosure.

[0185] It should also be noted that the various specific technical features described in the above specific implementation methods can be combined in any suitable manner without contradiction.

[0186] In addition, various embodiments of the present disclosure may be arbitrarily combined, and as long as they do not violate the concept of the present disclosure, they should also be regarded as the contents disclosed by the present disclosure.

Claims

1. A stone gradation detection method based on image recognition, characterized in that: include: Obtaining a video of stone particles collected by a video collection device, wherein the video collection device is used to collect video data of the stone particles in a dynamic sliding process; Preprocessing each frame image in the stone particle video to generate a stone particle image of the stone particles; Annotating the stone particle contours in the stone particle image by using an instance segmentation model to generate a contour information image of the stone particles, wherein the contour information image includes contour annotation information of the stone particles; Based on the contour annotation information, the physical properties of the stone particles in the contour information image are identified to generate stone particle size data and stone volume data of the stone particles; Generating a gradation test result of the stone particles according to the stone particle size data and the stone volume data; The identifying the physical characteristics of the stone particles in the contour information image based on the contour marking information to generate the stone particle size data and stone volume data of the stone particles includes: Extracting contour quantification indexes and shape characteristic parameters of the contour annotation information in the contour information image to generate characteristic parameters of the stone particles; Convert the characteristic parameters into particle size and volume to generate the stone particle size data and the stone volume data of the stone particles; The preprocessing of each frame image in the stone particle video to generate a stone particle image of the stone particles includes: Determine the video frame rate information of the stone particle video and the sliding speed information of each stone particle in the stone particle video; According to the video frame rate information and the sliding speed information, frame extraction processing is performed on the stone particle video to extract a first stone particle image of each stone particle in the stone particle video; Preprocessing the first stone particle image to generate the stone particle image; The preprocessing of the first stone particle image to generate the stone particle image includes: Performing a histogram equalization process on the first stone particle image to rearrange the distribution of pixel point values ​​in the first stone particle image to generate a second stone particle image of the stone particles; Based on a preset correction algorithm, motion blur correction is performed on the stone particles in the second stone particle image to generate the stone particle image; Alternatively, the preprocessing of the first stone particle image to generate the stone particle image includes: Performing blur processing on the first stone particle image by a preset low-pass filter device to generate a blurred image of the stone particles; Subtracting the pixel value of each pixel point in the first stone particle image from the pixel value of each pixel point in the blurred image to generate a mask image of the stone particles; Adding the pixel value of each pixel point in the first stone particle image and the pixel value of each pixel point in the mask image to generate the stone particle image; The step of labeling the stone particle contours in the stone particle image by using an instance segmentation model to generate a contour information image of the stone particles includes: Converting the stone particle image into a stone feature image through the backbone network of the instance segmentation model; A network module is established through a preset area of ​​the instance segmentation model, stone particles in the stone feature image are framed, and a candidate frame of the stone particles is generated; Classifying the candidate frames pixel by pixel to generate mask information of the stone particles; Marking the mask information at the corresponding position of the stone particle image to generate the contour information image; The identifying the physical characteristics of the stone particles in the contour information image based on the contour marking information to generate the stone particle size data and stone volume data of the stone particles includes: Determine the perimeter data, area data, and major and minor axis data of the stone particles according to the mask information corresponding to the stone particles in the contour information image; According to the perimeter data, the area data and the major and minor axis data, the stone particle size data is determined by using an equivalent particle size calculation formula; The stone volume data is determined according to the stone particle size data.

2. A stone gradation detection device based on image recognition, characterized in that: include: An acquisition module, used to acquire the video of the stone particles acquired by the video acquisition device, wherein the video acquisition device is used to acquire video data of the stone particles in the dynamic sliding process; A first generating module is used to pre-process each frame image in the stone particle video to generate a stone particle image of the stone particles; The preprocessing of each frame image in the stone particle video to generate the stone particle image of the stone particles includes: determining the video frame rate information of the stone particle video and the sliding speed information of each stone particle in the stone particle video; performing frame extraction processing on the stone particle video according to the video frame rate information and the sliding speed information to extract the first stone particle image of each stone particle in the stone particle video; preprocessing the first stone particle image to generate the stone particle image; The preprocessing of the first stone particle image to generate the stone particle image includes: performing histogram equalization processing on the first stone particle image to rearrange the distribution of pixel point values ​​in the first stone particle image to generate a second stone particle image of the stone particles; performing motion blur correction on the stone particles in the second stone particle image based on a preset correction algorithm to generate the stone particle image; or, the preprocessing of the first stone particle image to generate the stone particle image includes: blurring the first stone particle image by a preset low-pass filtering device to generate a blurred image of the stone particles; subtracting the pixel value of each pixel point in the first stone particle image from the pixel value of each pixel point in the blurred image to generate a mask image of the stone particles; adding the pixel value of each pixel point in the first stone particle image to the pixel value of each pixel point in the mask image to generate the stone particle image; The second generation module is used to annotate the stone particle contours in the stone particle image through an instance segmentation model to generate a contour information image of the stone particles, wherein the contour information image includes the contour annotation information of the stone particles; the annotating the stone particle contours in the stone particle image through the instance segmentation model to generate the contour information image of the stone particles includes: converting the stone particle image into a stone feature image through the backbone network of the instance segmentation model; establishing a network module through the preset area of ​​the instance segmentation model to select the stone particles in the stone feature image to generate a candidate frame of the stone particles; and performing pixel-by-pixel classification on the candidate frame. , generating mask information of the stone particles; marking the mask information at the corresponding position of the stone particle image to generate the contour information image; based on the contour marking information, identifying the physical characteristics of the stone particles in the contour information image to generate stone particle size data and stone volume data of the stone particles, including: determining the perimeter data, area data, and major and minor axis data of the stone particles according to the mask information corresponding to the stone particles in the contour information image; determining the stone particle size data according to the perimeter data, the area data, and the major and minor axis data through a particle size calculation equivalent formula; determining the stone volume data according to the stone particle size data; A third generating module is used to identify the physical characteristics of the stone particles in the contour information image based on the contour annotation information, and generate the stone particle size data and stone volume data of the stone particles; the identifying the physical characteristics of the stone particles in the contour information image based on the contour annotation information, and generating the stone particle size data and stone volume data of the stone particles, comprises: extracting the contour quantization index and shape characteristic parameters of the contour annotation information in the contour information image to generate the characteristic parameters of the stone particles; performing particle size and volume conversion on the characteristic parameters to generate the stone particle size data and the stone volume data of the stone particles; An execution module is used to generate a gradation detection result of the stone particles according to the stone particle size data and the stone volume data.

3. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to claim 1 are implemented.

4. An electronic device, characterized in that: include: a memory having a computer program stored thereon; A processor, configured to execute the computer program in the memory to implement the steps of the method according to claim 1.

Citation Information

Patent Citations

  • Stone grain composition detection device and method based on image detection

    CN117129388A

  • Rock-fill material grading detection method and device based on unmanned aerial vehicle aerial survey and instance segmentation

    CN117576080A

  • Method and device for determining contour information of rockfill material, storage medium and electronic equipment

    CN118608553A

  • System and method for processing multiple loose gemstones using image-based analysis techniques

    US20210201461A1