Silo grille blockage detection and automatic crushing motion control system based on machine vision

Abstract

The present invention provides a silo grille blockage detection and automatic crushing motion control system based on machine vision, which relates to the field of image data processing and includes: a blockage sensing module, which is used to obtain object sensing data of the silo grille based on infrared rays to determine whether blockage occurs on the silo grille; an image acquisition module, which is used to acquire an image of the silo grille according to the object sensing data of the silo grille when the blockage sensing module determines that blockage occurs on the silo grille; a blockage recognition module, which is used to determine whether blockage occurs on the silo grille according to the image and object sensing data of the silo grille, and when it is determined that blockage occurs on the silo grille, determine the blockage information of the silo grille according to the image and object sensing data of the silo grille; an ore crushing module, which is used to generate a crushing plan according to the blockage information of the silo grille, and control an ore crushing mechanical arm to perform ore crushing according to the crushing plan, which has the advantage of realizing automatic detection and crushing of silo grille blockage.

Description

Technical Field

[0001] The invention relates to the field of image data processing, and in particular to a machine vision-based silo grille blockage detection and automatic crushing motion control system. Background Art

[0002] Large pieces of ore pose potential hazards to the subsequent silo ore discharging machine, jaw crusher and conveyor belt. Silo screens are set to reduce the impact damage of large pieces of ore. Due to the size limit of the silo screen, large pieces of ore often get stuck or blocked, resulting in poor discharge of the screen. Therefore, it is necessary to use a crushing robot arm to perform secondary crushing or unblocking on the silo screen. Due to frequent blockages, the daily workload of clearing blockages is large, there is a shortage of full-time personnel, the manual operation intensity is high, and the working hours are long, which seriously affects production efficiency.

[0003] Therefore, it is necessary to provide a silo grille blockage detection and automatic crushing motion control system based on machine vision to realize automatic detection and crushing of silo grille blockage. Summary of the invention

[0004] The present invention provides a machine vision-based silo grille blockage detection and automatic crushing motion control system, comprising: a blockage sensing module, used to obtain object sensing data of a silo grille based on infrared rays, and to determine whether a blockage occurs on the silo grille according to the object sensing data of the silo grille; an image acquisition module, used to acquire an image of the silo grille according to the object sensing data of the silo grille when the blockage sensing module determines that a blockage occurs on the silo grille; a blockage recognition module, used to determine whether a blockage occurs on the silo grille according to the image and object sensing data of the silo grille, and also used to determine the blockage information of the silo grille according to the image and object sensing data of the silo grille when it is determined that a blockage occurs on the silo grille according to the image and object sensing data of the silo grille, wherein the blockage information of the silo grille includes at least an ore position and an ore contour; an ore crushing module, used to generate a crushing scheme according to the blockage information of the silo grille, and to control an ore crushing mechanical arm to perform ore crushing according to the crushing scheme.

[0005] Furthermore, the congestion sensing module includes an infrared sensing unit and a congestion prediction unit, wherein the infrared sensing unit is used to obtain object sensing data of the silo grille, and the congestion prediction unit determines whether congestion occurs on the silo grille according to the object sensing data of the silo grille; the infrared sensing unit includes a plurality of transverse infrared ranging components and a plurality of longitudinal infrared ranging components arranged on the silo grille, the infrared ranging planes of any two transverse infrared ranging components are parallel, the plurality of transverse infrared ranging components are respectively arranged at different heights, the infrared ranging planes of any two longitudinal infrared ranging components are parallel, and the plurality of longitudinal infrared ranging components are The components are respectively arranged at different heights, the transverse infrared ranging component includes a plurality of transverse infrared ranging devices arranged along the length direction of the silo grille, the longitudinal infrared ranging component includes a plurality of longitudinal infrared ranging devices arranged along the width direction of the silo grille, the infrared ranging direction of the transverse infrared ranging device is perpendicular to the infrared ranging direction of the longitudinal infrared ranging device, and the object perception data of the silo grille includes the transverse ranging information of the blockage on the silo grille measured by each of the transverse infrared ranging devices and the longitudinal ranging information of the blockage on the silo grille measured by each of the longitudinal infrared ranging devices.

[0006] Furthermore, the blockage prediction unit determines whether a blockage occurs on the silo grille according to the object sensing data of the silo grille, including: for each of the lateral infrared ranging devices, according to the lateral ranging information of the blockage on the silo grille measured by the lateral infrared ranging device and the lateral ranging information of the blockage on the silo grille measured by the adjacent lateral infrared ranging device of the lateral infrared ranging device, calculating the ranging difference parameter corresponding to the lateral infrared ranging device, and determining whether the lateral infrared ranging device is a target lateral infrared ranging device according to the ranging difference parameter; for each of the longitudinal infrared ranging devices, according to the longitudinal infrared ranging device The longitudinal distance measurement information of the blockage on the silo grille measured by the external distance measuring device and the longitudinal distance measurement information of the blockage on the silo grille measured by the adjacent longitudinal infrared distance measuring device of the longitudinal infrared distance measuring device are used to calculate the distance measurement difference parameter corresponding to the longitudinal infrared distance measuring device, and whether the longitudinal infrared distance measuring device is the target longitudinal infrared distance measuring device is judged according to the distance measurement difference parameter; the obstacle length is determined according to the transverse coordinate of each of the target transverse infrared distance measuring devices; the obstacle width is determined according to the longitudinal coordinate of each of the target longitudinal infrared distance measuring devices; and whether a blockage occurs on the silo grille is judged according to the obstacle length and obstacle width.

[0007] Furthermore, a plurality of transverse infrared ranging devices are arranged along the length direction of the silo grille, including: acquiring a historical image of the silo grille; determining the blocking frequency of different transverse coordinates based on the historical image of the silo grille; determining the spacing between two adjacent transverse infrared ranging devices of the transverse infrared ranging assembly according to the blocking frequency of different transverse coordinates; according to the spacing between two adjacent transverse infrared ranging devices of the transverse infrared ranging assembly, a plurality of transverse infrared ranging devices are arranged along the length direction of the silo grille; a plurality of longitudinal infrared ranging devices are arranged along the width direction of the silo grille, including: determining the blocking frequency of different longitudinal coordinates based on the historical image of the silo grille; determining the spacing between two adjacent longitudinal infrared ranging devices of the longitudinal infrared ranging assembly according to the blocking frequency of different longitudinal coordinates; according to the spacing between two adjacent longitudinal infrared ranging devices of the longitudinal infrared ranging assembly, a plurality of longitudinal infrared ranging devices are arranged along the length direction of the silo grille.

[0008] Furthermore, the image acquisition module includes a lighting component, an image acquisition component and a control component, wherein the lighting component includes a lighting posture adjustment device and a lighting device arranged on the lighting posture adjustment device, and the control component is used to control the lighting component to adjust the lighting posture of the lighting device and control the image acquisition component to capture the image of the silo grille according to the object perception data of the silo grille.

[0009] Furthermore, the image acquisition module acquires an image of the silo grille according to the object perception data of the silo grille, including: determining a blockage area according to the lateral coordinates of each of the target lateral infrared ranging devices and the longitudinal coordinates of each of the target longitudinal infrared ranging devices; determining an optimal lighting posture according to the blockage area; adjusting the lighting posture of the lighting device according to the optimal lighting posture; after adjusting the lighting posture of the lighting device, controlling the image acquisition component to acquire the image of the silo grille.

[0010] Furthermore, the blockage identification module determines whether blockage occurs on the silo grille based on the image of the silo grille and the object perception data, including: capturing an image of an area of ​​interest from the image of the silo grille based on the coordinate information of the blocked area; and determining whether blockage occurs on the silo grille based on the image of the area of ​​interest.

[0011] Furthermore, the blockage recognition module determines the blockage information of the silo grille according to the image of the silo grille and the object perception data, including: determining the ore position and ore contour on the silo grille according to the image of the region of interest through a blockage recognition model.

[0012] Furthermore, the ore crushing module generates a crushing plan according to the blockage information of the silo grille, including: generating a crushing path according to the blockage information of the silo grille, wherein the crushing path includes multiple crushing positions; generating crushing parameters for each of the crushing positions according to the blockage information of the silo grille, wherein the crushing parameters include at least crushing force and crushing angle.

[0013] Furthermore, the ore crushing module generates a crushing path according to the blockage information of the silo grille, including: determining the relative position relationship between any two pieces of ores on the silo grille according to the ore position on the silo grille; determining the optimal crushing direction of each piece of ore on the silo grille according to the relative position relationship between any two pieces of ores on the silo grille and the ore contour of each piece of ore on the silo grille; generating the crushing path according to the optimal crushing direction of each piece of ore on the silo grille.

[0014] Compared with the prior art, the machine vision-based silo grille blockage detection and automatic crushing motion control system provided by the present invention has at least the following beneficial effects:

[0015] 1. The system can obtain the object perception data and image information of the silo grille in real time and accurately through the blockage perception module and image acquisition module. The blockage recognition module uses machine vision technology to accurately determine whether there is a blockage on the silo grille and determine the specific information of the blockage, such as the ore position and ore contour. This greatly improves the accuracy and speed of blockage detection, and is more efficient and reliable than the traditional manual detection method. Traditional manual detection of silo grille blockage is not only time-consuming and labor-intensive, but also requires a lot of human resources. The system can realize the automatic detection, identification and crushing process without manual intervention, greatly reducing labor costs. Silo grille blockage may cause equipment failure, production interruption and even safety accidents. The system can effectively prevent equipment failure and safety accidents and ensure production safety through timely blockage detection and automatic crushing processing. The automatic crushing module can generate a crushing plan based on the blockage information and control the ore crushing robot arm for precise crushing, thereby improving production efficiency.

[0016] 2. By setting up multiple horizontal and vertical infrared ranging components on the silo grille, the system can obtain detailed object perception data, including horizontal and vertical ranging information of the blockage. The jam prediction unit uses this ranging information to calculate the ranging difference parameters to accurately identify the location and scale of the jam, thereby improving the accuracy of jam detection. The system analyzes the jam frequency of different coordinates based on the historical images of the silo grille, thereby intelligently determining the spacing and layout of the infrared ranging devices. This intelligent layout design not only improves the detection efficiency of the system, but also optimizes the use of resources and reduces unnecessary costs. The jam perception module only collects images of the silo grille when it determines that a jam has occurred on the silo grille, effectively reducing the collection and identification of invalid images and further reducing the operating costs of the system.

[0017] 3. The image acquisition module can intelligently adjust the lighting posture of the lighting device according to the object perception data of the silo grille to ensure that the blocked area is fully illuminated when the image is collected. This precise lighting adjustment can significantly improve the quality of the image, making the image clearer and easier to analyze. By intercepting the image of the area of ​​interest from the image based on the coordinate information of the blocked area, the blockage recognition module can focus on analyzing the key areas and reduce unnecessary calculations. This enables the system to identify blockages more quickly and improves overall work efficiency. The lighting posture adjustment device of the lighting component enables the system to adapt to silo grilles of different shapes and sizes, as well as blockages of different locations and scales. This adaptability enables the system to work normally in various complex environments, increasing its scope of application.

[0018] 4. The blockage identification module uses advanced machine vision technology and blockage identification models to accurately identify the position and contour of the ore on the silo grille. This precise positioning and contour recognition provides key information for subsequent crushing operations, ensuring the accuracy and effectiveness of crushing. The ore crushing module generates the crushing path and crushing parameters for each crushing position based on the blockage information, including crushing force and crushing angle. This optimized crushing scheme ensures that the ore is effectively crushed while reducing unnecessary energy consumption and mechanical wear. By analyzing the relative position relationship between ores and the ore contour, the ore crushing module can intelligently plan the optimal crushing path. This path planning ensures the efficiency and safety of the crushing operation, allowing the system to complete the crushing task in a shorter time, thereby improving overall work efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] This specification will be further described in the form of exemplary embodiments, which will be described in detail by the accompanying drawings. These embodiments are not restrictive, and in these embodiments, the same number represents the same structure, wherein:

[0020] Figure 1 It is a module schematic diagram of a machine vision-based silo grille blockage detection and automatic crushing motion control system according to some embodiments of this specification;

[0021] Figure 2 is a schematic diagram of an infrared sensing unit according to some embodiments of this specification;

[0022] Figure 3 is a schematic diagram of multiple lateral infrared ranging components according to some embodiments of this specification;

[0023] Figure 4 It is a schematic diagram of a flow chart for determining whether a blockage occurs on a silo grille according to some embodiments of this specification.

[0024] In the figure, 1. silo grille; 2. horizontal infrared ranging component; 21. horizontal infrared ranging device; 3. vertical infrared ranging component; 31. vertical infrared ranging device. DETAILED DESCRIPTION

[0025] In order to more clearly illustrate the technical solutions of the embodiments of this specification, the following is a brief introduction to the drawings required for the description of the embodiments. Obviously, the drawings described below are only some examples or embodiments of this specification. For ordinary technicians in this field, this specification can also be applied to other similar scenarios based on these drawings without creative work. Unless it is obvious from the language environment or otherwise explained, the same reference numerals in the figures represent the same structure or operation.

[0026] Figure 1 is a schematic diagram of a module of a machine vision based silo grille blockage detection and automatic crushing motion control system according to some embodiments of this specification, such as Figure 1 As shown, the machine vision-based silo grille blockage detection and automatic crushing motion control system may include a blockage sensing module, an image acquisition module, a blockage recognition module and an ore crushing module.

[0027] The blockage sensing module can be used to obtain object sensing data of the silo grille based on infrared rays, and determine whether blockage occurs on the silo grille based on the object sensing data of the silo grille.

[0028] In some embodiments, the congestion sensing module includes an infrared sensing unit and a congestion prediction unit, wherein the infrared sensing unit is used to obtain object sensing data of the silo grille, and the congestion prediction unit determines whether congestion occurs on the silo grille based on the object sensing data of the silo grille.

[0029] Figure 2 is a schematic diagram of an infrared sensing unit according to some embodiments of this specification, Figure 3is a schematic diagram of multiple lateral infrared ranging components 2 according to some embodiments of this specification, such as Figure 2 and Figure 3 As shown, the infrared sensing unit includes a plurality of transverse infrared ranging components 2 and a plurality of longitudinal infrared ranging components 3 arranged on the silo grille 1, the infrared ranging planes of any two transverse infrared ranging components 2 are parallel, and the plurality of transverse infrared ranging components 2 are respectively arranged at different heights, the infrared ranging planes of any two longitudinal infrared ranging components 3 are parallel, and the plurality of longitudinal infrared ranging components 3 are respectively arranged at different heights, the transverse infrared ranging component 2 includes a plurality of transverse infrared ranging devices arranged along the length direction of the silo grille 1, the longitudinal infrared ranging component 3 includes a plurality of longitudinal infrared ranging devices 31 arranged along the width direction of the silo grille 1, the infrared ranging direction of the transverse infrared ranging device is perpendicular to the infrared ranging direction of the longitudinal infrared ranging device 31, and the object sensing data of the silo grille 1 includes the transverse ranging information of the blockage on the silo grille 1 measured by each transverse infrared ranging device and the longitudinal ranging information of the blockage on the silo grille 1 measured by each longitudinal infrared ranging device 31.

[0030] Specifically, the lateral infrared distance measuring device 21 and the longitudinal infrared distance measuring device 31 are technical devices that use infrared rays to measure distances. The principle is mainly based on the time difference between the reflection and reception of infrared light or the change in the intensity of the reflected light to measure the distance from the target object to the infrared distance measuring device. Infrared rays are electromagnetic waves with a frequency higher than that of visible light and cannot be directly seen by the human eye. The infrared distance measuring device consists of an infrared transmitter and an infrared receiver. When the infrared transmitter emits an infrared beam, the beam will irradiate the target object and be reflected. The receiver will receive the reflected infrared light. The infrared transmitter emits an infrared signal with a specific frequency or phase; when the infrared signal encounters the target, it will be reflected back and interfere with the transmitted signal; the infrared receiver measures the phase difference between the reflected signal and the transmitted signal; according to the principle of interference, the phase difference is proportional to the distance, so the lateral distance between the blockage and the lateral infrared distance measuring device 21 (i.e., the lateral distance measurement information of the blockage) and the longitudinal distance between the blockage and the longitudinal infrared distance measuring device 31 (i.e., the longitudinal distance measurement information of the blockage) can be calculated by measuring the phase difference.

[0031] Figure 4 is a schematic diagram of a process for determining whether a silo grille is blocked according to some embodiments of this specification, such as Figure 4 As shown, in some embodiments, the blockage prediction unit determines whether blockage occurs on the silo grille according to the object sensing data of the silo grille, including:

[0032] For each transverse infrared ranging device, a ranging difference parameter corresponding to the transverse infrared ranging device is calculated based on the transverse ranging information of the blockage on the silo grille measured by the transverse infrared ranging device and the transverse ranging information of the blockage on the silo grille measured by an adjacent transverse infrared ranging device of the transverse infrared ranging device, and whether the transverse infrared ranging device is a target transverse infrared ranging device is determined based on the ranging difference parameter;

[0033] For each longitudinal infrared ranging device, according to the longitudinal ranging information of the obstruction on the silo grille measured by the longitudinal infrared ranging device and the longitudinal ranging information of the obstruction on the silo grille measured by the longitudinal infrared ranging device adjacent to the longitudinal infrared ranging device, a ranging difference parameter corresponding to the longitudinal infrared ranging device is calculated, and according to the ranging difference parameter, whether the longitudinal infrared ranging device is the target longitudinal infrared ranging device is determined;

[0034] Determine the obstacle length based on the lateral coordinates of each target's lateral infrared ranging device;

[0035] Determine the obstacle width based on the longitudinal coordinates of each target's longitudinal infrared ranging device;

[0036] Determine whether there is blockage on the silo grille based on the obstacle length and obstacle width.

[0037] Specifically, the adjacent lateral infrared ranging device of the lateral infrared ranging device may be a lateral infrared ranging device located on the left side, right side, above or below the lateral infrared ranging device.

[0038] The distance measurement difference parameter corresponding to the lateral infrared distance measurement device can be calculated according to the following formula:

[0039]

[0040] in, is the distance measurement difference parameter corresponding to the i-th lateral infrared ranging device, is the total number of adjacent lateral infrared ranging devices of the i-th lateral infrared ranging device, is the lateral distance between the obstruction measured by the i-th lateral infrared distance measuring device and the n-th adjacent lateral infrared distance measuring device of the i-th lateral infrared distance measuring device, The lateral distance between the blockage and the i-th lateral infrared ranging device measured by the i-th lateral infrared ranging device.

[0041] When the ranging difference parameter corresponding to the lateral infrared ranging device is less than the ranging difference parameter threshold, the lateral infrared ranging device is determined to be the target lateral infrared ranging device.

[0042] The method of calculating the distance measurement difference parameter corresponding to the longitudinal infrared ranging device is similar to the method of calculating the distance measurement difference parameter corresponding to the transverse infrared ranging device, which will not be repeated here. When the distance measurement difference parameter corresponding to the longitudinal infrared ranging device is less than the distance measurement difference parameter threshold, the longitudinal infrared ranging device is determined to be the target longitudinal infrared ranging device.

[0043] For each transverse infrared ranging component, according to the transverse coordinates of the target transverse infrared ranging device included in the transverse infrared ranging component, determine the target transverse infrared ranging device with the largest transverse coordinate and the target transverse infrared ranging device with the smallest transverse coordinate corresponding to the transverse infrared ranging component, and according to the transverse coordinates of the target transverse infrared ranging device with the largest transverse coordinate and the transverse coordinates of the target transverse infrared ranging device with the smallest transverse coordinate corresponding to the transverse infrared ranging component, determine the obstacle coordinate range corresponding to the transverse infrared ranging component. For example, if the transverse coordinate of the target transverse infrared ranging device with the largest transverse coordinate corresponding to the transverse infrared ranging component is 7, and the transverse coordinate of the target transverse infrared ranging device with the smallest transverse coordinate corresponding to the transverse infrared ranging component is 3, then the obstacle coordinate range corresponding to the transverse infrared ranging component is (3, 7).

[0044] For each lateral infrared ranging component, the obstacle length corresponding to the lateral infrared ranging component is calculated according to the obstacle coordinate range corresponding to the lateral infrared ranging component. For example, if the obstacle coordinate range corresponding to the lateral infrared ranging component is (3, 7), then the obstacle length corresponding to the lateral infrared ranging component = 7-3 = 4.

[0045] The maximum value of the obstacle lengths corresponding to the lateral infrared ranging components is taken as the obstacle length. For example, if the obstacle length corresponding to the lateral infrared ranging component A is 4 and the obstacle length corresponding to the lateral infrared ranging component B is 3, then the obstacle length is 4.

[0046] For each longitudinal infrared ranging component, according to the longitudinal coordinates of the target longitudinal infrared ranging device included in the longitudinal infrared ranging component, determine the target longitudinal infrared ranging device with the largest longitudinal coordinate and the target longitudinal infrared ranging device with the smallest longitudinal coordinate corresponding to the longitudinal infrared ranging component, and according to the longitudinal coordinates of the target longitudinal infrared ranging device with the largest longitudinal coordinate and the longitudinal coordinates of the target longitudinal infrared ranging device with the smallest longitudinal coordinate corresponding to the longitudinal infrared ranging component, determine the obstacle coordinate range corresponding to the longitudinal infrared ranging component. For each longitudinal infrared ranging component, according to the obstacle coordinate range corresponding to the longitudinal infrared ranging component, calculate the obstacle length corresponding to the longitudinal infrared ranging component. Take the maximum value of the obstacle length corresponding to the longitudinal infrared ranging component as the obstacle width.

[0047] When the obstacle length is greater than the obstacle length threshold and the obstacle width is greater than the obstacle width threshold, it is determined that a blockage occurs on the silo grille.

[0048] In some embodiments, a plurality of transverse infrared distance measuring devices are arranged along the length direction of the silo grille, including:

[0049] Acquire a historical image of the silo grille, wherein the historical image of the silo grille is an image when the silo grille is blocked;

[0050] Based on historical images of silo grilles, determine the blockage frequency at different lateral coordinates;

[0051] Determine the distance between two adjacent lateral infrared ranging devices of the lateral infrared ranging assembly according to the blocking frequencies at different lateral coordinates;

[0052] According to the distance between two adjacent transverse infrared distance measuring devices of the transverse infrared distance measuring assembly, a plurality of transverse infrared distance measuring devices are arranged along the length direction of the silo grille.

[0053] Specifically, the blockage recognition model can be used to identify the blockage area corresponding to the historical image based on the historical image of the silo grille, thereby determining the horizontal coordinate range of the blockage corresponding to the historical image. The blockage recognition model can be a convolutional neural network (CNN) model.

[0054] The blocking frequency of the horizontal coordinate is calculated according to the following formula:

[0055]

[0056] in, is the blocking frequency of the i-th horizontal coordinate, is the total number of historical images whose i-th horizontal coordinate is within the horizontal coordinate range of the historical image where the blockage occurs, is the total number of historical images of silo grilles obtained.

[0057] According to the blocking frequencies of different horizontal coordinates, a plurality of horizontal coordinate segments are determined, wherein a difference between the blocking frequencies of any two horizontal coordinates included in each horizontal coordinate segment is less than a frequency difference threshold.

[0058] For each horizontal coordinate segment, the distance between two adjacent horizontal infrared ranging devices in the horizontal coordinate segment is determined according to the following formula:

[0059]

[0060] in, is the distance between two adjacent lateral infrared ranging devices in the i-th lateral coordinate segment, is the total number of horizontal coordinates in the i-th horizontal coordinate segment, is the preset frequency, is the blocking frequency of the jth transverse coordinate in the i-th transverse coordinate segment, The preset spacing.

[0061] In some embodiments, a plurality of longitudinal infrared distance measuring devices are arranged along the width direction of the silo grille, including:

[0062] Based on historical images of silo grilles, determine the blockage frequency at different longitudinal coordinates;

[0063] Determine the distance between two adjacent longitudinal infrared ranging devices of the longitudinal infrared ranging assembly according to the blocking frequency of different longitudinal coordinates;

[0064] According to the distance between two adjacent longitudinal infrared distance measuring devices of the longitudinal infrared distance measuring assembly, a plurality of longitudinal infrared distance measuring devices are arranged along the length direction of the silo grille.

[0065] Specifically, the blockage recognition model may be used to identify the blockage area corresponding to the historical image based on the historical image of the silo grille, thereby determining the lateral coordinate range of the blockage corresponding to the historical image.

[0066] The method for calculating the blocking frequency of the longitudinal coordinate is similar to the method for calculating the blocking frequency of the transverse coordinate. The method for determining the spacing between two adjacent longitudinal infrared ranging devices of the longitudinal infrared ranging component is similar to the method for determining the spacing between two adjacent transverse infrared ranging devices of the transverse infrared ranging component, which will not be repeated here.

[0067] The image acquisition module can be used to acquire an image of the silo grille based on object perception data of the silo grille when the blockage perception module determines that blockage occurs on the silo grille.

[0068] In some embodiments, the image acquisition module includes a lighting component, an image acquisition component and a control component, wherein the lighting component includes a lighting posture adjustment device and a lighting device arranged on the lighting posture adjustment device, and the control component is used to control the lighting component to adjust the lighting posture of the lighting device and control the image acquisition component to capture an image of the silo grille according to the object perception data of the silo grille.

[0069] Specifically, the mechanical structure of the lighting posture adjustment device usually includes a support frame, a transmission mechanism, a guide mechanism and other parts. The support frame is used to fix and support the lighting device, and the transmission mechanism such as a motor, gear, screw and the like is responsible for driving the movement of the lighting device. The guide mechanism ensures that the lighting device can maintain a stable motion trajectory during the adjustment process. The drive device is the power source of the lighting posture adjustment device, and is usually powered by electric, pneumatic or hydraulic methods. The electric drive device has the advantages of precise control and fast response speed, and is suitable for occasions where the lighting posture needs to be adjusted frequently. The pneumatic and hydraulic drive devices have the characteristics of large output force and simple structure, and are suitable for occasions where a large load needs to be borne. The control component is the core part of the lighting posture adjustment device, which is responsible for receiving instructions from the control component and controlling the movement of the drive device, thereby adjusting the posture of the lighting device. The control system usually uses intelligent control devices such as microprocessors or programmable logic controllers (PLCs), which have the advantages of high precision, high reliability and easy programming. The control component controls the movement of the drive device according to the received parameters, thereby adjusting the position and posture of the lighting device. By precisely adjusting the lighting position, it can be ensured that the lighting device can illuminate the objects on the silo grille, improving the clarity and accuracy of image acquisition.

[0070] In some embodiments, the image acquisition module acquires an image of the silo grille according to the object perception data of the silo grille, including:

[0071] Determine the blocked area according to the lateral coordinate of each target lateral infrared ranging device and the longitudinal coordinate of each target longitudinal infrared ranging device;

[0072] Determine the optimal lighting position based on the blocked area;

[0073] According to the optimal lighting posture, adjust the lighting posture of the lighting device;

[0074] After adjusting the lighting posture of the lighting device, the image acquisition component is controlled to acquire the image of the silo grille.

[0075] Specifically, the obstacle coordinate ranges corresponding to all lateral infrared ranging components are combined to obtain the lateral coordinate range of the blocked area, and the obstacle coordinate ranges corresponding to all longitudinal infrared ranging components are combined to obtain the longitudinal coordinate range of the blocked area, thereby determining the blocked area.

[0076] The optimal lighting posture can be determined according to the lateral coordinate range and the longitudinal coordinate range of the blocked area through a posture optimization model, wherein the posture optimization model can be a Convolutional Neural Network (CNN) model.

[0077] The blockage recognition module can be used to determine whether a blockage occurs on the silo grille based on the image and object perception data of the silo grille.

[0078] Specifically include:

[0079] According to the coordinate information of the blocked area, an image of the area of ​​interest is intercepted from the image of the silo grille;

[0080] Based on the image of the area of ​​interest, determine whether blockage occurs on the silo grille.

[0081] Specifically, all optical camera lenses have distortion problems. Distortion is a geometric distortion of the image. It is a phenomenon of image distortion caused by different magnifications of the image in different areas on the focal plane. The degree of this deformation increases from the center of the image to the edge of the image, and is mainly reflected at the edge of the image. This distortion will cause the coordinates in the image to deviate from the coordinates in reality, so the image needs to be corrected for rectangles.

[0082] The algorithm flow for rectangular correction is as follows:

[0083] 1. Median filtering denoising;

[0084] 2. Extract different color channels (BGR) detection rectangles in sequence;

[0085] 3. Use canny to detect edges for each channel or use multiple thresholds for binarization;

[0086] 4. Use findContours function to find contours;

[0087] 5. Use the approxPolyDP function to remove some small twists and turns of the polygon outline;

[0088] 6. Find a quadrilateral that satisfies both the large area and the convex shape;

[0089] 7. Determine whether the cosine of the angle between two adjacent straight lines in the contour is less than 0.3 (which means the angle is around 90 degrees). If so, the quadrilateral is the rectangle found.

[0090] Through coordinate conversion, the horizontal coordinate range and vertical coordinate range of the blocked area in the world coordinate system are converted into the pixel horizontal coordinate range and pixel vertical coordinate range of the area of ​​interest in the pixel coordinate system, so as to intercept the image of the area of ​​interest from the image of the silo grille. Among them, the world coordinate system: its coordinate origin can be determined according to the situation, and can represent objects in space, and the unit is a length unit, such as mm. Pixel coordinate system: the coordinate origin is in the upper left corner, in pixels, with obvious range restrictions, that is, it is used to represent the pixel length and pixel length and width of the full screen, and is represented by a matrix.

[0091] The blockage determination model can be used to determine whether blockage occurs on the silo grille according to the image of the region of interest. The blockage determination model can be a convolutional neural network (CNN) model.

[0092] The blockage identification module can also be used to determine the blockage information of the silo grille based on the image and object perception data of the silo grille when blockage occurs on the silo grille, wherein the blockage information of the silo grille includes at least the ore position and the ore contour.

[0093] In some embodiments, the blockage identification module determines the blockage information of the silo grille according to the image of the silo grille and the object sensing data, including:

[0094] The blockage recognition model is used to determine the ore position and ore contour on the silo grid based on the area of ​​interest image.

[0095] The ore crushing module can be used to generate a crushing plan based on the blockage information of the silo grille, and control the ore crushing mechanical arm to crush the ore according to the crushing plan.

[0096] Specifically, the ore crushing robot arm is mainly composed of the following key parts:

[0097] Drive system:

[0098] Travel pump: Provides the necessary hydraulic power to ensure that each movement of the robotic arm can be performed smoothly and continuously.

[0099] Proportional valve: By adjusting the flow and pressure of the hydraulic oil, the robot arm can be precisely controlled. The high precision and response speed of the proportional valve enable the robot arm to be quickly adjusted according to different crushing requirements.

[0100] Transmission and control system:

[0101] Clutch: Ensures that the robot arm can start or stop quickly and smoothly when needed, while protecting the transmission system from overload damage.

[0102] Control system: Integrates advanced control algorithms and sensor technology to monitor the working status of the robotic arm in real time and issue corresponding control instructions based on the crushing plan.

[0103] Actuator:

[0104] Boom: Responsible for moving the main working part of the crusher above the ore for crushing operations.

[0105] Hammer lifting mechanism: controls the up and down movement of the breaker hammer to ensure that the breaker hammer can hit the ore accurately.

[0106] Second arm: Assists the main arm in its work and realizes all-round crushing of ore by adjusting its position and angle.

[0107] Left-right swivel mechanism: enables the robot arm to rotate on the horizontal plane to cover a larger working area.

[0108] The working principle of the ore crushing mechanical arm is as follows:

[0109] According to the crushing plan, the control system issues corresponding control instructions.

[0110] The stroke pump in the drive system starts working to provide hydraulic power to the robotic arm.

[0111] The proportional valve adjusts the flow and pressure of the hydraulic oil according to the control instructions, thereby achieving precise control of the movement of the robotic arm.

[0112] The clutch starts or stops the robot's drive train as needed.

[0113] The upper arm, hammer lifting mechanism, second arm and left and right rotating mechanism in the actuator perform corresponding actions according to the control instructions to achieve precise crushing of the ore.

[0114] In some embodiments, the ore crushing module generates a crushing plan according to the blockage information of the silo grille, including:

[0115] Generate a crushing path according to the blockage information of the silo grille, wherein the crushing path includes a plurality of crushing positions, each crushing position corresponding to a piece of ore blocked on the silo grille;

[0116] According to the blockage information of the silo grille, the crushing parameters of each crushing position are generated, wherein the crushing parameters at least include the crushing force and the crushing angle.

[0117] In some embodiments, the ore crushing module generates a crushing path according to the blockage information of the silo grille, including:

[0118] According to the position of the ore on the silo grid, determine the relative position relationship between any two pieces of ore on the silo grid;

[0119] According to the relative position relationship between any two pieces of ore on the silo grid and the ore profile of each piece of ore on the silo grid, the optimal crushing direction of each piece of ore on the silo grid is determined;

[0120] The crushing path is generated according to the optimal crushing direction of each piece of ore on the silo grid.

[0121] Specifically, crushing the ore directly from top to bottom may produce a large impact force on the silo grille, causing damage or deformation of the silo grille. This impact may also cause the ore to bounce or splash, increasing safety hazards. Crushing the ore from the side can disperse the impact force and reduce the direct impact on the silo grille. Side crushing can also utilize the interaction force between ores to make the crushing process smoother and more efficient. Therefore, when possible, it is preferred to crush the ore from the side. Side crushing should choose the weak parts or fragile points of the ore as much as possible to reduce the difficulty of crushing and energy consumption.

[0122] The theoretical optimal crushing direction of each ore on the silo grille can be determined by a direction determination model based on the relative position relationship between any two ores on the silo grille and the ore profile of each ore on the silo grille, wherein the direction determination model can be a convolutional neural network model.

[0123] A path evaluation system may be established, wherein the path evaluation system is composed of a plurality of path evaluation indicators, wherein the plurality of path evaluation indicators at least include a crushing direction optimization indicator, and the plurality of path evaluation indicators may also include an operation time indicator, an operation energy consumption indicator, and the like.

[0124] The solution generation model generates candidate crushing paths according to the blockage information of the silo grille and the optimal crushing direction of each ore on the silo grille. The solution generation model can be a convolutional neural network model.

[0125] For each candidate broken path, the scores of the candidate broken path in multiple path evaluation indicators are calculated, and the scores of the candidate broken paths in multiple path evaluation indicators are weighted and summed to obtain the path score of the candidate broken path. The candidate broken path with the highest path score is selected as the broken path.

[0126] For each crushing position of each candidate crushing path, the relative position relationship between any two pieces of ores on the silo grille after the crushing work at the crushing position is completed can be determined. The direction determination model is used to predict the actual optimal crushing direction corresponding to the next crushing position based on the relative position relationship between any two pieces of ores on the silo grille after the crushing work at the crushing position is completed and the ore contour of each piece of ore on the silo grille.

[0127] For example, the score of the candidate crushing path in the crushing direction optimization index can be calculated according to the following formula:

[0128]

[0129] in, is the score of the candidate crushing path optimization index in the crushing direction, is the preset parameter, greater than 0, is the total amount of ore to be crushed on the silo grid, is the theoretical optimal crushing direction corresponding to the mth piece of ore to be crushed on the silo grid, It is the actual optimal crushing direction corresponding to the mth piece of ore to be crushed on the silo grid in the candidate crushing path.

[0130] The crushing force at each crushing position may be generated according to the blockage information and crushing path of the silo grille through a parameter determination model, wherein the parameter determination model may be a Convolutional Neural Network (CNN) model.

[0131] The operation process of the machine vision silo grille blockage detection and automatic crushing motion control system is as follows:

[0132] S11. Identify that the vehicle arrives at the unloading location and identify the completion status of the vehicle unloading operation.

[0133] S12. After unloading is completed and a certain period of time has passed, object perception data of the silo grille is obtained, and based on the object perception data of the silo grille, whether a blockage occurs on the silo grille is determined; when the blockage perception module determines that a blockage occurs on the silo grille, an image of the silo grille is collected based on the object perception data of the silo grille, and whether a blockage occurs on the silo grille is determined based on the image of the silo grille and the object perception data; when it is determined that a blockage occurs on the silo grille based on the image of the silo grille and the object perception data, the blockage information of the silo grille is determined based on the image of the silo grille and the object perception data, and a crushing plan is generated based on the blockage information of the silo grille.

[0134] S13. Arrange the task sequence according to the crushing plan and send the task control instruction to the control system of the ore crushing robot arm, and prompt the user on the platform page to start the crushing operation.

[0135] S14. The control system of the ore crushing mechanical arm controls the ore crushing mechanical arm to enter the operating posture from the safe posture, and controls the ore crushing mechanical arm to complete the striking task according to the task control instruction list. During the movement and striking process of the breaker hammer, it operates at a fixed height.

[0136] S15. If a new vehicle enters the unloading area during the striking operation of the ore crushing robot arm, the ore crushing module sends a stop command to the control system of the ore crushing robot arm to control the ore crushing robot arm to return to a safe posture and wait for the vehicle unloading operation to be completed.

[0137] S16. After the ore crushing robot arm completes its striking task, it should return to a safe position and wait for the next operation.

[0138] Finally, it should be understood that the embodiments described in this specification are only used to illustrate the principles of the embodiments of this specification. Other variations may also fall within the scope of this specification. Therefore, as an example and not a limitation, alternative configurations of the embodiments of this specification may be considered consistent with the teachings of this specification. Accordingly, the embodiments of this specification are not limited to the embodiments explicitly introduced and described in this specification.

Claims

1. Based on machine vision, the silo grille blockage detection and automatic crushing motion control system is characterized by: include: A blockage sensing module, used to obtain object sensing data of the silo grille based on infrared rays, and determine whether blockage occurs on the silo grille according to the object sensing data of the silo grille; An image acquisition module is used to acquire an image of the silo grille according to object perception data of the silo grille when the blockage perception module determines that blockage occurs on the silo grille; a blockage identification module, used to determine whether blockage occurs on the silo grille according to the image and object perception data of the silo grille, and to determine blockage information of the silo grille according to the image and object perception data of the silo grille when blockage occurs on the silo grille, wherein the blockage information of the silo grille includes at least an ore position and an ore contour; An ore crushing module, used for generating a crushing scheme according to the blockage information of the silo grille, and controlling the ore crushing mechanical arm to perform ore crushing according to the crushing scheme; The blockage sensing module includes an infrared sensing unit and a blockage prediction unit, wherein the infrared sensing unit is used to obtain object sensing data of the silo grille, and the blockage prediction unit determines whether blockage occurs on the silo grille according to the object sensing data of the silo grille; The infrared sensing unit includes a plurality of transverse infrared ranging components and a plurality of longitudinal infrared ranging components arranged on the silo grille, the infrared ranging planes of any two transverse infrared ranging components are parallel, and the plurality of transverse infrared ranging components are arranged at different heights, the infrared ranging planes of any two longitudinal infrared ranging components are parallel, and the plurality of longitudinal infrared ranging components are arranged at different heights, the transverse infrared ranging component includes a plurality of transverse infrared ranging devices arranged along the length direction of the silo grille, the longitudinal infrared ranging component includes a plurality of longitudinal infrared ranging devices arranged along the width direction of the silo grille, the infrared ranging direction of the transverse infrared ranging device is perpendicular to the infrared ranging direction of the longitudinal infrared ranging device, and the object sensing data of the silo grille includes the transverse ranging information of the blockage on the silo grille measured by each of the transverse infrared ranging devices and the longitudinal ranging information of the blockage on the silo grille measured by each of the longitudinal infrared ranging devices; The blockage prediction unit determines whether blockage occurs on the silo grille according to the object sensing data of the silo grille, including: For each of the transverse infrared ranging devices, a ranging difference parameter corresponding to the transverse infrared ranging device is calculated based on the transverse ranging information of the blockage on the silo grille measured by the transverse infrared ranging device and the transverse ranging information of the blockage on the silo grille measured by an adjacent transverse infrared ranging device of the transverse infrared ranging device, and whether the transverse infrared ranging device is a target transverse infrared ranging device is determined based on the ranging difference parameter; For each of the longitudinal infrared ranging devices, according to the longitudinal ranging information of the blockage on the silo grille measured by the longitudinal infrared ranging device and the longitudinal ranging information of the blockage on the silo grille measured by the adjacent longitudinal infrared ranging device of the longitudinal infrared ranging device, a ranging difference parameter corresponding to the longitudinal infrared ranging device is calculated, and according to the ranging difference parameter, whether the longitudinal infrared ranging device is the target longitudinal infrared ranging device is determined; Determining the obstacle length according to the lateral coordinate of each target lateral infrared ranging device; Determining the obstacle width according to the longitudinal coordinate of each of the target longitudinal infrared ranging devices; Whether a blockage occurs on the silo grille is determined according to the obstacle length and obstacle width.

2. The machine vision-based silo grille blockage detection and automatic crushing motion control system according to claim 1 is characterized in that: A plurality of transverse infrared distance measuring devices are arranged along the length direction of the silo grille, including: Acquire historical images of the silo grille; determining the blockage frequency at different lateral coordinates based on the historical images of the silo grid; Determine the distance between two adjacent lateral infrared ranging devices of the lateral infrared ranging assembly according to the blocking frequencies at different lateral coordinates; According to the distance between two adjacent transverse infrared distance measuring devices of the transverse infrared distance measuring assembly, a plurality of transverse infrared distance measuring devices are arranged along the length direction of the silo grille; A plurality of longitudinal infrared distance measuring devices are arranged along the width direction of the silo grille, including: Determining the blockage frequency at different longitudinal coordinates based on the historical images of the silo grille; Determine the distance between two adjacent longitudinal infrared ranging devices of the longitudinal infrared ranging assembly according to the blocking frequency of different longitudinal coordinates; According to the distance between two adjacent longitudinal infrared distance measuring devices of the longitudinal infrared distance measuring assembly, a plurality of longitudinal infrared distance measuring devices are arranged along the length direction of the silo grille.

3. The machine vision-based silo grille blockage detection and automatic crushing motion control system according to any one of claim 2, characterized in that: The image acquisition module includes a lighting component, an image acquisition component and a control component, wherein the lighting component includes a lighting posture adjustment device and a lighting device arranged on the lighting posture adjustment device, and the control component is used to control the lighting component to adjust the lighting posture of the lighting device and control the image acquisition component to capture the image of the silo grille according to the object perception data of the silo grille.

4. The machine vision-based silo grille blockage detection and automatic crushing motion control system according to claim 3 is characterized in that: The image acquisition module acquires an image of the silo grille according to the object perception data of the silo grille, including: Determine the blocked area according to the lateral coordinate of each target lateral infrared ranging device and the longitudinal coordinate of each target longitudinal infrared ranging device; Determining an optimal lighting posture according to the blocked area; According to the optimal lighting posture, adjusting the lighting posture of the lighting device; After adjusting the lighting posture of the lighting device, the image acquisition component is controlled to acquire the image of the silo grille.

5. The machine vision-based silo grille blockage detection and automatic crushing motion control system according to claim 4 is characterized in that: The blockage identification module determines whether blockage occurs on the silo grille according to the image of the silo grille and the object sensing data, including: According to the coordinate information of the blocked area, an image of an area of ​​interest is intercepted from the image of the silo grille; Whether blockage occurs on the silo grille is determined according to the image of the region of interest.

6. The machine vision-based silo grille blockage detection and automatic crushing motion control system according to claim 5 is characterized in that: The blocking identification module determines the blocking information of the silo grille according to the image of the silo grille and the object sensing data, including: The ore position and ore contour on the silo grid are determined based on the region of interest image through the blockage recognition model.

7. The machine vision-based silo grille blockage detection and automatic crushing motion control system according to claim 6 is characterized in that: The ore crushing module generates a crushing plan according to the blockage information of the silo grille, including: Generate a crushing path according to the blockage information of the silo grille, wherein the crushing path includes a plurality of crushing positions; According to the blockage information of the silo grille, the crushing parameters of each crushing position are generated, wherein the crushing parameters at least include the crushing force and the crushing angle.

8. The machine vision-based silo grille blockage detection and automatic crushing motion control system according to claim 7 is characterized in that: The ore crushing module generates a crushing path according to the blockage information of the silo grille, including: Determine the relative position relationship between any two pieces of ores on the silo grid according to the position of the ores on the silo grid; Determine the optimal crushing direction of each ore on the silo grid according to the relative position relationship between any two ores on the silo grid and the ore profile of each ore on the silo grid; The crushing path is generated according to the optimal crushing direction of each piece of ore on the silo grid.

Images