Intelligent Recognition Method and Device for Stones in Unmanned Crushing of Excavators

Abstract

The present invention relates to an intelligent stone recognition method and device for unmanned crushing of an excavator, comprising the following steps: A. Obtaining an image of the environment around the excavator; B. Performing ellipse detection on the image, classifying the detected ellipses to determine the ellipses for attitude detection, and obtaining the projected ellipses of the cylinders; C. Correcting the centroid offset of the ellipses according to the actual contour of the stones; and D. Calculating the coordinates of the stones in the camera coordinate system based on the corrected ellipses; thereby enabling accurate recognition of the stones in the environment around the excavator and preparing for subsequent automated crushing work.

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition, and particularly to an intelligent recognition method and device for stones for unmanned crushing of an excavator. Background Art

[0002] Excavators are widely used in construction sites of projects such as mine exploitation, building construction, and road and bridge construction. In the prior art, excavators generally adopt a hydraulic system for driving and are manually operated for construction. However, due to the relatively harsh environment of the construction site, the work comfort of the staff is relatively poor. Moreover, with the increase in labor costs and the influence of the physical strength of workers and external weather, etc., excavators cannot work all day long, resulting in huge resource waste for enterprises and owners. For the crushing operation of an excavator, in order to realize the automatic construction of a breaker, machine vision to identify the stones to be crushed is an important link. Only by being able to accurately identify the stones to be crushed can the breaker be smoothly guided to crush the stones and complete the automatic construction.

[0003] In the prior art, generally, a target recognition algorithm / template matching algorithm is used to identify stones: The target recognition algorithm generally adopts methods such as feature point extraction or template matching. SURF and SIFT are the most commonly used local feature detection algorithms at present, and they are robust to illumination, noise, and small-range perspective changes. However, these two algorithms are prone to misdetecting feature points. In addition, there are not many obvious feature points that can be obtained for stones. When there are few local feature points, the effect of the feature point detection algorithm drops significantly. The template matching algorithm refers to finding an optimal pose that can make the similarity function or search function between the template and the current image reach an extreme value. However, the calculation and storage amounts of this detection method are also very high, and the commonly used algorithms such as SCV, NCC, and SSD are also less robust to occlusion. Considering the complexity of the working environment of an excavator, it is very easy for foreign objects to form occlusion. Therefore, the template matching algorithm is also not suitable. Summary of the Invention

[0004] Object of the Invention: In order to overcome the drawbacks pointed out in the background art, the embodiments of the present invention provide an intelligent recognition method and device for stones for unmanned crushing of an excavator, which can effectively solve the problems involved in the above background art.

[0005] Technical Solution: An intelligent recognition method for stones for unmanned crushing of an excavator includes the following steps: A. Obtain an image of the environment around the excavator; B. Perform ellipse detection on the image, classify the detected ellipses to determine the ellipses for pose detection, and obtain the projected ellipses of cylinders; C. Perform centroid offset correction of the ellipses according to the actual contour of the stones; and D. Calculate the coordinates of the stones in the camera coordinate system according to the corrected ellipses.

[0006] As a preferred embodiment of the present invention, step B includes: performing Canny edge detection on the image, converting the image into an edge image, detecting arc segments through gradient direction constraints, and aggregating the arc segments to generate ellipses according to the curvature of the arc segments.

[0007] As a preferred embodiment of the present invention, step A includes: automatically adjusting the exposure parameters when the image is acquired, and the adjustment process is: indexing in the query table according to the ambient brightness when the image is taken to query the image target brightness, wherein the query table is pre-established according to the optimal matching relationship between the ambient brightness and the image target brightness; calculating the exposure target output value according to the queried image target brightness, and the calculation formula is expressed as: Input = 255x(Output / 255)gamma, wherein Input is the image target brightness, Gamma is 2.2, and Output is the exposure target output value, that is, the exposure brightness of 18% gray in the partition exposure method; evaluating each parameter from multiple angles according to the optimization parameter algorithm, and the formula of the optimization parameter algorithm is expressed as: Where Π is the ambient brightness T ab The amount of exposure time adjustment within the interval, : Ambient brightness T ab The exposure needs to be adjusted within the range. : Ambient brightness T ab Grayscale adjustment ratio effect within the interval, : Ambient brightness T ab Negative effects of exposure adjustment within the range.

[0008] As a preferred embodiment of the present invention, step C includes: using an ellipse gravity center offset correction algorithm to correct the gravity center offset of the ellipse; wherein the ellipse gravity center offset correction algorithm is divided into two regions and applied to the first quadrant: a unit step length is taken in the x direction in the first region where the absolute value of the slope is less than 1, and a unit step length is taken in the y direction in the second region where the absolute value of the slope is greater than 1; taking (x c ,y c )=(0,0), and define the elliptic function as: f ellipse (x, y) is the decision parameter; from (0, r y ), take a unit step in the x direction until the boundary between the first area and the second area, then switch back to a unit step in the y direction, and then cover the remaining curve segments in the first quadrant, and detect the slope value of the curve at each step; the slope equation is: At the border between the first and second areas, and It turns out that the condition for shifting the first area is: The decision function is evaluated by the center of gravity offset to determine the next position along the elliptical trajectory: At the next sampling position (x k+1 +1 = x k +2), the decision parameter of the first region can be evaluated as: where y k+1 According to the sign of p1 k takes the value of y k or y k-1 ; if p1 k < 0, the increment is If p1 k ≥ 0, the increment is In the second region, sample at unit steps in the negative direction;

[0009] At the next position y k+1 -1 = y k -2 evaluate the elliptic function, or where x k+1 is set according to the sign of p2 k takes the value of x k or x k+1 .

[0010] As a preferred embodiment of the present invention, an elliptical centroid offset correction algorithm is used to correct the centroid offset of the ellipse, including: inputting r x , r y and the center of the ellipse (x c , y c ), and obtaining the first point on the ellipse: (x0, y0) = (0, r y ), calculating the initial value of the centroid offset decision parameter in the first region: At each x k position in the first region, starting from k = 0, if p1 k < 0, the next point along the ellipse centered at (0, 0) is (x k+1 , y k ), and Otherwise, the next point along the ellipse is (x k +1, y k -1), and where, And until Use the last point (x0, y0) calculated in the first region to calculate the initial value of the parameter in the second region:

[0011] At each position of y k in the second region, starting from k = 0, if p2 k > 0, the next point along the ellipse centered at (0, 0) is (x k, y k -1), and Otherwise, for the next point (x along the ellipse k+1 , y k -1), and Perform calculations using the same x and y increments as in the first region until y = 0; determine the symmetric points in the other three quadrants; move each calculated pixel position (x, y) to the ellipse trajectory centered at (x c , y c ), and plot points according to the coordinate values: x = x + x c , y = y + y c ; The redrawn ellipse is the ellipse after the center of gravity offset correction.

[0012] As a preferred embodiment of the present invention, step B includes: obtaining a constraint condition by making the planes of the spatial circles fitted by the stones parallel to each other, specifically: calculating the normal vector of the plane where the spatial circle is located, and the cone formed by the spatial circle and the camera center intersects with the plane where the spatial circle is located, and the formed figure is a circle on this plane, thus obtaining the parallelism constraint condition.

[0013] As a preferred embodiment of the present invention, the center coordinates [x'0 y'0 z'0] of the spatial circle and the normal vector [n x ' n y '

[0014] n' z of the plane where the spatial circle is located are respectively:

[0015]

[0016]

[0017] As a preferred embodiment of the present invention, step B includes: calculating two sets of solutions for the position and attitude of the spatial circle feature based on the elliptical projection of the spatial circle feature under the camera plane, where the position and attitude in each set of solutions correspond to each other; where, assuming the true normal vector of the plane of the spatial circle feature in the camera coordinate system is n1, then n1 ≈ kN, k ≠ 0; substituting n1 into the formula to calculate the attitude parameters of the spatial circle feature.

[0018] The present invention achieves the following beneficial effects:

[0019] 1. By classifying the stones into ellipses, detecting ellipses in the image, then classifying the ellipses to determine the ellipses for attitude detection, obtaining the projected ellipses of the cylinders, and then correcting the center of gravity offset of the ellipses according to the actual contours of the stones, and finally calculating the coordinates of the stones in the camera coordinate system based on these corrected ellipses, the present invention can accurately identify the stones in the environment around the excavator, preparing for subsequent automated crushing work.

[0020] 2. When the camera of the present invention acquires the image of the environment around the excavator, the automatic camera exposure adjustment algorithm is used to automatically adjust the exposure parameters during image acquisition, so that different exposure parameters can be automatically adjusted according to the environmental brightness to improve the contrast and clarity of the image. In this way, the quality is guaranteed at the source of the photo, which is beneficial to the improvement of the subsequent recognition accuracy.

[0021] 3. The present invention adopts the adjacent circle coplanar constraint technology to eliminate the ambiguity of the single circle pose, thereby realizing the certainty of the stone pose. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 is a schematic flowchart of a stone intelligent recognition method provided by one embodiment of the present invention;

[0023] Figure 2 is a schematic diagram of stone fitting circle detection pose estimation provided by one embodiment of the present invention;

[0024] Figure 3 is a schematic diagram of single circle pose measurement provided by one embodiment of the present invention;

[0025] Figure 4 is a schematic structural diagram of a stone intelligent recognition device provided by one embodiment of the present invention;

[0026] Figure 5 is a schematic structural diagram of an excavator provided by one embodiment of the present invention;

[0027] Figure 6 is a schematic structural diagram of a computer-readable storage medium provided by one embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0028] The embodiments of the present invention will be described in detail below. The examples of the embodiments are shown in the drawings, where the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the drawings are exemplary and are intended to explain the present invention and should not be construed as limiting the present invention.

[0029] Refer to Figures 1 to 3 As shown, this embodiment provides a stone intelligent recognition method for unmanned crushing of an excavator. The method includes the following steps:

[0030] A. Acquire the image of the environment around the excavator;

[0031] B. Perform ellipse detection on the image, classify the detected ellipses to determine the ellipses for pose detection, and obtain the projected ellipses of the cylinders;

[0032] C. Perform the centroid offset correction of the ellipse according to the actual contour of the stone; and

[0033] D. Calculate the coordinates of the stone in the camera coordinate system according to the corrected ellipse.

[0034] In one way, in step A, a camera is used to obtain an image of the environment around the excavator, and then the obtained image is transmitted to an image processor, and the image processor performs the above steps B, C, and D on the obtained image.

[0035] The camera used in this embodiment can be installed on the excavator or independently set with a distance interval from the excavator in space. The image can be transmitted between the camera and the image processor either by wire or wirelessly. In a preferred way, the camera used in this embodiment is composed of a left camera and a right camera, and each of them obtains an image and transmits it to the image processor. Among them, the camera used in this embodiment can be an independent camera, a camera carried by a mobile device, a dome camera, a camera of an augmented reality (AR) / virtual reality (VR) device, etc., and the present application does not limit this.

[0036] In some embodiments, when the camera obtains an image of the environment around the excavator, a camera exposure automatic adjustment algorithm is used to automatically adjust the exposure parameters during image acquisition, so as to be able to automatically adjust different exposure parameters according to the ambient brightness. In one way, the specific adjustment process of automatically adjusting different exposure parameters according to the ambient brightness is as follows:

[0037] (1) Index in a look-up table according to the ambient brightness at the time of image shooting to query the target brightness of the image, where the look-up table is pre-established according to the optimal matching relationship between the ambient brightness and the target brightness of the image.

[0038] When the camera takes an image, the current ambient brightness, i.e., the light value, is detected by an ambient brightness detection device / sensor. Then, the detected ambient brightness is brought into the lookup table for indexing to query the target image brightness, and the queried target image brightness is the optimal image brightness corresponding to the ambient brightness. Among them, the relationship between the ambient brightness and the target image brightness in the lookup table can be an interval correspondence relationship. For example, an ambient brightness interval corresponds to a target image brightness interval, an ambient brightness interval corresponds to a target image brightness, or an ambient brightness corresponds to a target image brightness interval; it can also be a non-interval correspondence relationship, that is, a single value correspondence. (2) Calculate the exposure target output value according to the queried target image brightness. Its calculation formula is expressed as: Input = 255x(Output / 255)^gamma, where Input is the target image brightness, Gamma is 2.2, and Output is the exposure target output value, that is, the exposure brightness of 18% gray in the zonal exposure method.

[0039] According to the zonal exposure method, the gradual change of human perception from black to white is divided into 11 levels (i.e., 0, I, II, III, IV, V, VI, VII, VIII, IX, X; among them, pure black is level 0 and pure white is level X). The block V in the middle is considered to have a moderate exposure intensity and is called medium gray. The light reflectance of the V-level block is 18%, which is the defined 18% gray.

[0040] (3) Evaluate each parameter from multiple perspectives according to the optimization parameter algorithm. The formula of the optimization parameter algorithm is expressed as:

[0041]

[0042] Among them, Π: the adjustment amount of the exposure time within the ambient brightness T ab interval, : the adjustment amount required for the exposure degree within the ambient brightness T ab interval, : the gray scale adjustment ratio effect within the ambient brightness T ab interval, : the negative impact brought by the exposure adjustment within the ambient brightness T ab interval.

[0043] Adjust the real-time exposure amount according to the above formula to improve the contrast and clarity of the image. In this way, the quality is guaranteed first at the photo source, which is beneficial to the improvement of the subsequent recognition accuracy.

[0044] In practical applications, stones may have multiple irregular edge features, and pseudo edges may also exist due to factors such as noise, lighting, and shooting angle. Therefore, it is necessary to classify the edges on the image, select reasonable edge features to establish an object coordinate system, and then estimate the target posture. The present invention classifies the existence of stones as a structure called a rotating body, in which the planes where the space circles on this structure are located are parallel to each other, and the straight line passing through the center of the circle is perpendicular to the plane where it is located. Based on this, the present invention classifies it into ellipse classification, proposes parallelism and verticality constraints of the projected ellipse, obtains the projection of the upper space circle, and then proposes a spatial multi-ellipse posture estimation method. The stones in the image are identified by implementing the above steps B, C, and D, that is: first, the image is ellipse detected, and then the ellipse is classified using a random sampling consistency algorithm to obtain the projection ellipse of the cylinder, and then the center of gravity offset of the ellipse is corrected according to the actual contour of the stone, and finally the coordinates of the stone in the camera coordinate system are calculated based on these corrected ellipses.

[0045] The algorithm based on arc segment extraction is used to detect ellipses. The algorithm combines arc segment extraction and classification, is simple to calculate, and is very fast. In step B, the target image is first subjected to Canny edge detection to convert the image into an edge image, and then the arc segments are detected by gradient direction constraints, and finally the arc segments are aggregated to generate ellipses by the curvature of these arc segments. After the ellipses on the image are detected, these ellipses need to be classified to distinguish reasonable ellipses from multiple interfering ellipses or pseudo-ellipses for posture detection.

[0046] In some embodiments, step C includes: using an ellipse gravity center offset correction algorithm to correct the gravity center offset of the ellipse. The ellipse gravity center offset correction algorithm is divided into two regions and applied to the first quadrant: a unit step length is taken in the x direction in the first region where the absolute value of the slope is less than 1, and a unit step length is taken in the y direction in the second region where the absolute value of the slope is greater than 1;

[0047] Take (x c ,y c )=(0,0), and define the elliptic function as: f ellipse (x,y) is the decision parameter;

[0048] From (0, r y ), take a unit step in the x direction until the boundary between the first area and the second area, then switch back to a unit step in the y direction, and then cover the remaining curve segments in the first quadrant, and detect the slope value of the curve at each step;

[0049] The slope equation is:

[0050]

[0051] At the border between the first and second areas, And It is obtained that the condition for offsetting the first region is: Evaluate the decision function through the centroid offset to determine the next position along the elliptical trajectory:

[0052]

[0053] At the next sampling position (x k+1 +1 = x k +2), the decision parameter of the first region can be evaluated as:

[0054]

[0055] where, y k+1 According to the sign of p1 k take the value of y k or y k-1 ;

[0056] If p1 k <0, the increment is If p1 k ≥0, the increment is In the second region, sample at a unit step size in the negative direction;

[0057]

[0058] At the next position y k+1 -1 = y k -2 evaluate the elliptic function, or

[0059]

[0060] where, x k+1 is set according to the sign of p2 k and can take the value of x k or x k+1 .

[0061] The specific process of performing the centroid offset correction of the ellipse using the elliptic centroid offset correction algorithm is as follows:

[0062] (1) Input r x , r y and the ellipse center (x c , y c ), and obtain the first point on the ellipse: (x0, y0) = (0, r y ).

[0063] (2) Calculate the initial value of the centroid offset decision parameter in the first region:

[0064] (3) For each x in the first region k position, starting from k = 0, if p1 k < 0, the next point on the ellipse centered at (0, 0) is (x k+1 , y k ), and otherwise, the next point on the ellipse is (x k +1, y k -1), and where and until

[0065] (4) Use the last point (x0, y0) calculated in the first region to calculate the initial values of the parameters in the second region:

[0066]

[0067] (5) At each position of y in the second region k , starting from k = 0, if p2 k > 0, the next point on the ellipse centered at (0, 0) is (x k , y k -1), and otherwise, the next point (x k+1 , y k -1), and Calculate using the same x and y increments as in the first region until y = 0;

[0068] (6) Determine the symmetric points in the other three quadrants.

[0069] (7) Move each calculated pixel position (x, y) to the ellipse trajectory centered at (x c , y c ), and plot points according to the coordinate values: x = x + x c , y = y + y c ;

[0070] (8) The redrawn ellipse is the ellipse after centroid offset correction.

[0071] In some embodiments, step B includes: obtaining a constraint condition by using the planes of the spatial circles fitted by the stones to be parallel to each other. Specifically: calculate the normal vector of the plane where the spatial circle is located. The cone formed by the spatial circle and the camera center intersects with the plane where the spatial circle is located, and the formed figure is a circle on this plane, that is, the parallelism constraint condition is obtained.

[0072] The projection of the spatial circle feature on the camera imaging plane is an ellipse, assuming the radius of the circle feature is R. The optical center of the camera and the ellipse projection can uniquely determine an elliptic cone. When a plane forms a circle with a radius of R with the cross-section of this elliptic cone, the center coordinates of this cross-section are the position of the spatial circle feature, and the normal vector of the cross-section ring contains the attitude information of the spatial circular feature. However, the solution for the pose measurement of a single circle feature by a monocular camera is not unique, as Figure 3 shown. Where E is the elliptical projection of the spatial circle feature on the imaging plane, and C1 and C2 are two spatial circle features corresponding to the elliptical projection. Without constraint conditions, the correspondence between E, C1, and C2 cannot be determined. The center coordinates [x'0 y'0 z'0] of the spatial circle and the normal vector [n' x n y 'n' z of the plane where the spatial circle is located are respectively:

[0073]

[0074]

[0075] In some embodiments, in order to eliminate the ambiguity of the single circle pose, the adjacent circle coplanar constraint technology is adopted to achieve the certainty of the stone pose, that is, step B further includes: calculating two sets of solutions for the position and pose of the spatial circle feature according to the elliptical projection of the spatial circle feature under the camera plane, where the position and pose in each set of solutions correspond to each other; where, assuming the true normal vector of the spatial circle feature plane in the camera coordinate system is n1, then n1≈kN, k≠0; substituting n1 into the formula to calculate the pose parameters of the spatial circle feature.

[0076] Specifically, two sets of solutions for the position and pose of the spatial circle feature can be calculated according to the elliptical projection of the single circle feature under the camera plane, where the position and pose in each set of solutions correspond to each other. Therefore, the true pose solution of the circular feature object can be obtained by eliminating the false pose. Assuming the normal vectors of the corresponding circular feature planes in the camera coordinate system are n1 and n2. Since the plane where the adjacent circles are located is coplanar with the plane where the circular feature is located and parallel to the plane where the circular feature is located, the true normal vector of the plane where the circular feature is located in the camera coordinate system is parallel to the normal vector of the plane obtained by the rectangular constraint. Assuming the true normal vector of the circular feature plane in the camera coordinate system is n1, then n1≈kN, k≠0. Through this constraint, the false solutions of the circular feature pose can be effectively eliminated. Substituting n1 into the formula can solve the pose parameters of the circular feature. From the above derivation, it can be seen that under the least constraint conditions, it is possible to eliminate the false solutions of the circular pose without knowing the geometric dimensions and coordinates of the rectangle in the world coordinate system.

[0077] Under different lighting and angle changes, the accurate locking of the position of the key positioning target is achieved. According to the mutual relationship information, the accurate pose can be obtained.

[0078] In some embodiments, a binocular vision system application is utilized to obtain the coordinates of the stone in the camera coordinate system by including steps such as camera calibration, stereo rectification, stereo matching, and 3D reconstruction.

[0079] In some embodiments, referring to Figure 4 As shown, a stone intelligent recognition device 10 is provided, including: a memory 101 and a processor 102. The memory 101 is used to store a computer program, and the processor 102 is used to call the computer program to execute the corresponding steps of the stone intelligent recognition method provided in this embodiment.

[0080] In some embodiments, referring to Figure 5 As shown, an excavator 20 is provided, which includes a central controller 201 and a vision module 202. The vision module 202 is used to provide an image signal for the central controller 201, and it includes an image processor 203 connected to the central controller 201 and cameras 204, 205 disposed on the excavator 20, and the cameras 204, 205 are connected to the image processor 203.

[0081] In some embodiments, the excavator 20 further includes a supplementary light source 206 connected to the central controller 201. The function of the supplementary light source 206 is to target some areas with low illuminance. The central controller 201 can control the operation of the supplementary light source 206 according to the information of the image, thereby increasing the illuminance of the working area, facilitating the operation of the excavator 20, and also facilitating the vision module 202 to obtain clear images.

[0082] The image processor 203, the left and right cameras 204, 205, and the supplementary light source 206 in the excavator are combined to form the vision module 202 of the present invention, which is used to be connected to the central controller 201 to realize the recognition of stones around the excavator 20, further determine the working face and the operation target stone, and realize distance measurement of the operation target stone.

[0083] In some embodiments, a computer-readable storage medium is provided. A computer program is stored in the computer-readable storage medium, and when the computer program runs on a computer, it executes the corresponding steps of the stone intelligent recognition method provided in this embodiment.

[0084] Referring to Figure 6 As shown, the computer-readable storage medium includes a computer program for executing a computer process on a computing device. In some embodiments, the computer-readable storage medium is provided using a signal-bearing medium 300. The signal-bearing medium 300 may include one or more program instructions, which when run by one or more processors can provide the above for Figure 1the described functionality or portions thereof. Thus, for example, one or more of the features A - D in reference Figure 1 may be borne by one or more instructions associated with the signal - bearing medium 300. Additionally, Figure 6 the program instructions in also describe example instructions. In some examples, the signal - bearing medium 300 may include a computer - readable medium 301, such as but not limited to, a hard disk drive, a compact disk (CD), a digital video disk (DVD), a digital tape, a memory, a read - only memory (ROM), or a random access memory (RAM), etc.

[0085] In some embodiments, the signal - bearing medium 300 may include a computer - recordable medium 302, such as but not limited to, a memory, a read / write (R / W) CD, an R / W DVD, etc.

[0086] In some embodiments, the signal - bearing medium 300 may include a communication medium 303, such as but not limited to, a digital and / or analog communication medium (e.g., a fiber optic cable, a waveguide, a wired communication link, a wireless communication link, etc.).

[0087] The signal - bearing medium 300 may be conveyed by a wireless form of the communication medium 303 (e.g., a wireless communication medium compliant with the IEEE 802.11 standard or other transmission protocols). One or more program instructions may be, for example, computer - executable instructions or logic implementation instructions.

[0088] It should be understood that the arrangements described herein are for illustrative purposes only. Thus, those skilled in the art will understand that other arrangements and other elements (e.g., machines, interfaces, functions, orders, and function groups, etc.) can be used instead, and some elements may be omitted altogether depending on the desired results. Additionally, many of the elements described can be implemented as discrete or distributed components, or as functional entities combined with other components in any suitable combination and location.

[0089] In the description of this specification, the descriptions referring to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples", etc. mean that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, without contradiction, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0090] Although the embodiments of the present invention have been shown and described above, it can be understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those of ordinary skill in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present invention.

Claims

1. An intelligent recognition method for stones in unmanned crushing of excavators, characterized in that It includes the following steps: A. Automatically adjust the exposure parameters during image acquisition to obtain an image of the environment around the excavator; B. Perform ellipse detection on the image, classify the detected ellipses to determine the ellipses for attitude detection, and obtain the projected ellipses of the cylinders; C. Use the ellipse centroid offset correction algorithm to correct the centroid offset of the ellipse according to the actual contour of the stone; and D. Calculate the coordinates of the stone in the camera coordinate system based on the corrected ellipse; Among them, the process of automatically adjusting the exposure parameters is as follows: Index in the lookup table according to the environmental brightness during image shooting to query the target brightness of the image. The lookup table is pre-established according to the optimal matching relationship between the environmental brightness and the target brightness of the image; Calculate the exposure target output value according to the queried target brightness of the image. Its calculation formula is: Input = 255 x (Output / 255)^gamma, where Input is the target brightness of the image, Gamma is 2.2, and Output is the exposure target output value, that is, the exposure brightness of 18% gray in the zonal exposure method; Evaluate each parameter from multiple perspectives according to the optimized parameter algorithm, and the formula of the optimized parameter algorithm is expressed as: Among them, The adjustment amount of the exposure time within the environmental brightness T ab interval, The adjustment amount required for the exposure within the environmental brightness T ab interval, The gray-scale adjustment ratio effect within the environmental brightness T ab interval, The negative impact brought by the exposure adjustment within the environmental brightness T ab interval; Among them, the process of using the ellipse centroid offset correction algorithm to correct the centroid offset of the ellipse is as follows: Input r x , r y and the center of the ellipse (x c , y c ), and obtain the first point on the ellipse: (x0, y0) = (0, r y ). Calculate the initial value of the centroid offset decision parameter in the first region: For each x in the first region k position, starting from k = 0, if p1 k < 0, the next point on the ellipse centered at (0, 0) is (x k+1 , y k ), and otherwise, the next point on the ellipse is (x k +1, y k -1), and Among them, and until Use the last point (x0, y0) calculated in the first region to calculate the initial value of the parameters in the second region: At each y k position in the second region, starting from k = 0, if p2 k > 0, the next point along the ellipse centered at (0, 0) is (x k , y k - 1), and otherwise, the next point along the ellipse (x k + 1, y k - 1), and Use the same x and y increments as in the first region for calculation until y = 0; Determine the symmetric points in the other three quadrants; Move each calculated pixel position (x, y) to an elliptical trajectory centered at (x c , y c ), and draw points according to the coordinate values: x = x + x c , y = y + y c ; The redrawn ellipse is the ellipse after centroid offset correction.

2. The intelligent recognition method for stones used in the unmanned crushing of an excavator according to claim 1, characterized in that, Step B includes: Perform Canny edge detection on the image, convert the image into an edge image, detect arc segments through gradient direction constraints, and aggregate the arc segments according to the curvature of the arc segments to generate an ellipse.

3. The intelligent recognition method for stones used in the unmanned crushing of an excavator according to claim 1, characterized in that Step C includes: Use the ellipse centroid offset correction algorithm to correct the centroid offset of the ellipse; Among them, the ellipse centroid offset correction algorithm is applied to the first quadrant in two regions: take a unit step in the x direction in the first region where the absolute value of the slope is less than 1, and take a unit step in the y direction in the second region where the absolute value of the slope is greater than 1; Take (x c , y c ) = (0, 0), and define the elliptic function as: f ellipse (x, y) which is the decision parameter; Starting from (0, r y ), take unit steps in the x-direction until the boundary between the first and second regions, then switch to unit steps in the y-direction and cover the remaining curve segments in the first quadrant, detecting the curve slope value at each step; The slope equation is: At the boundary region between the first region and the second region, and it is obtained that the condition for offsetting the first region is: Evaluate the decision function through the centroid offset to determine the next position along the elliptical trajectory: At the next sampling position (x k+1 + 1 = x k + 2), the decision parameter for the first region can be evaluated as: where y k+1 according to p1 k the symbol value of which is y k or y k-1 ; If p1 k < 0, the increment is If p1 k ≥ 0, the increment is In the second region, sample at a unit step in the negative direction; At the next position y k+1 -1 = y k - Evaluate the elliptic function, or Among them, x k+1 is set according to p2 k The symbol of can take the value of x k or x k+1 .

4. The intelligent recognition method for stones used in the unmanned crushing of an excavator according to claim 1, characterized in that Step B includes: Use the fact that the planes where the spatial circles fitted by the stones are parallel to each other to obtain a constraint condition. Specifically: calculate the normal vector of the plane where the spatial circle is located. The cone formed by the spatial circle and the camera center intersects with the plane where the spatial circle is located, and the formed figure is a circle on this plane, that is, the parallelism constraint condition is obtained.

5. An intelligent stone recognition device, characterized in that, It includes: A memory and a processor. The memory is used to store a computer program, and the processor is used to call the computer program to execute the method according to any one of claims 1-4.

6. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program. When the computer program runs on a computer, the computer is caused to execute the method according to any one of claims 1-4.

Images