A method for metering and filling of spheroidal material based on image analysis

Abstract

The application relates to the technical field of image analysis, in particular to a spherical material metering and filling control method based on image analysis. The method comprises the following steps: performing configuration analysis operation on initial instructions to extract related configuration parameters; collecting and outputting mirror high spot images and structured light texture images through cross field cameras; outputting non-repeated contour coordinate sets through edge constraint completion operation and image registration and deduplication operation; sequentially performing grid counting operation and volume integral projection operation based on cross-section statistical grids to output a total number of global materials; detecting and outputting in-transit falling speed and in-transit particle number; and performing cutoff threshold comparison operation to output a feeding blockage instruction and cut off a material falling track. The application constructs a deep coupling network from bottom layer multi-dimensional pixel tensor extraction to spatial topological serialization prediction, and fills the sampling calculation time lag deviation between virtual state parameter deduction values and frame interval discrete pixel array evolution in a multi-frame space-time calculation pipeline.

Description

Technical Field

[0001] This invention relates to the field of image analysis technology, specifically to a method for controlling the metering and filling of spherical materials based on image analysis. Background Technology

[0002] In applications involving the quantitative packaging and industrial metering of discrete spherical materials, image feature analysis and 3D spatial topology reconstruction of large-scale materials in complex physical environments (such as high-reflectivity, high-density random stacking) based on computer vision and image data processing technologies is an important current technological application direction. Existing technologies generally employ planar mapping and edge detection methods based on conventional image sensors. This involves acquiring a 2D image snapshot of the receiving container or feeding area using image acquisition equipment, extracting the visible surface contour using basic image segmentation algorithms, and then using the extracted 2D feature parameters to perform quantity statistics or spatial distribution attribute discrimination of material objects to output trigger commands for external mechanisms to execute actions. This is currently the mainstream technology for achieving multi-target image recognition and visual parameter output.

[0003] However, existing technical solutions have shortcomings in image distortion analysis and multidimensional topology deduction. Its basic image processing mechanism is highly susceptible to optical interference from specular reflections on the surface of spherical materials. Furthermore, in the random stacking state within the receiving container, strong physical occlusion occurs between materials, resulting in breakpoints and structural gaps in the target contours of the original image stream output by the image sensor. This makes it difficult to reconstruct the true geometric boundaries using conventional connected component segmentation. When processing single-view 2D images, the system only extracts and maps surface features. Lacking the computational capability to transform 2D planar scalars into 3D hidden spatial depth, it cannot directly deconstruct the spatial distribution characteristics and 3D reconstruction state of the occluded materials at the bottom layer based on the local surface image stream. This leads to the final image calculation and deduction results deviating from the actual physical quantity within the container. In addition, when processing image sequences of continuous dynamic targets, its temporal feature tracking mechanism exhibits lag. It primarily relies on isolated static frame images as the basis for object parsing, failing to consider the displacement characteristics and dynamic connected component changes of materials in high-speed motion states such as free fall within the feeding channel at the pixel level. This results in a temporal deviation between the predicted output generated based on the existing visual image stream and the continuous falling physical dynamics, ultimately causing the final output truncation command to deviate from the target value.

[0004] To address this, a method for metering and filling spherical materials based on image analysis is proposed. Summary of the Invention

[0005] The purpose of this invention is to provide a method for metering and filling spherical materials based on image analysis, so as to solve the problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a method for metering and filling spherical materials based on image analysis, comprising: Receive initial control commands and output task parameter sets. Through configuration parsing, output target loading quantity and image statistical grid. Perform polarization alternating illumination acquisition operation on spherical materials in the receiving container through a cross-field camera, and output specular highlight image and structured light texture image. Edge constraint completion is performed on the specular highlight image and structured light texture image using the pre-set spherical curvature a priori equation, and a complete contour in a single field of view is output. Image registration and deduplication are performed on the complete contour in the single field of view, and a set of contour coordinates without repetition is output. The set of contour coordinates without repetition is input into the image statistical grid to perform grid operation, and a local surface density matrix is ​​output. The preset depth inference model is invoked to perform image integral projection on the local surface density matrix and output the total number of global materials; the free-falling materials in the feeding channel are detected by the side vision sensor and the falling speed and the number of particles in the path are output. The total number of materials, the falling speed in transit, the number of particles in transit, and the target filling quantity are compared with the cutoff threshold to calculate the feeding interruption command; the feeding interruption command is transmitted to the actuator to drive the actuator to cut off the material falling trajectory.

[0007] Preferably, the specific generation process of the target loading quantity and the image statistical grid includes: receiving the initial control command, performing a transformation operation on the initial control command to generate an instruction feature stream; performing a field truncation operation on the instruction feature stream to extract the task parameter set; performing a separation and decomposition operation on the task parameter set to segment and generate a main path parameter set, an auxiliary path parameter set, and a timing configuration parameter; performing an addressing mapping operation on the main path parameter set to output the target loading quantity and the single-particle reference area; performing a splitting and unpacking operation on the auxiliary path parameter set to separate the first feature set; and converting the first feature set into a planar grid structure as the image statistical grid.

[0008] Preferably, the specific generation process of the specular highlight image and the structured light texture image includes: extracting the timing configuration parameters encapsulated in the task parameter set; performing frequency division calculations using the timing configuration parameters to output a main trigger pulse; driving a polarized illumination source to perform an illumination action in conjunction with the main trigger pulse; driving the cross-field camera to perform an exposure capture action and output original image stream data when the illumination action is enabled; performing brightness truncation filtering on the original image stream data to remove background interference information and output a set of bright pixels as the specular highlight image; performing secondary frequency division calculations using the timing configuration parameters to output a secondary trigger pulse; enabling the grid projection array to perform a projection action in conjunction with the secondary trigger pulse; driving the cross-field camera to perform a secondary exposure capture action and output secondary image stream data when the projection action is enabled; performing distortion correction filtering on the secondary image stream data to enhance edge transition coordinates and output the structured light texture image.

[0009] Preferably, the specific construction process of the single-view complete contour includes: performing connected component segmentation on the specular highlight image to extract the extreme centroid vector; extracting the optical axis angle matrix bound to the preset spherical curvature a priori equation; performing spatial offset permutation calculation on the extreme centroid vector and the optical axis angle matrix to output the spherical center reference coordinates; performing contour separation operation on the structured light texture image to output the incomplete contour matrix; combining the spherical center reference coordinates and the incomplete contour matrix into a joint parameter set and inputting it into the spherical curvature a priori equation; extracting the standard radius parameter contained in the spherical curvature a priori equation and performing curve fitting operation on the spherical center reference coordinates to output a virtual completion set; and performing matrix merging operation on the incomplete contour matrix and the virtual completion set to output the single-view complete contour.

[0010] Preferably, the specific generation process of the local surface density matrix includes: extracting the center anchor point parameters within all received single-view complete contours; performing spatial projection operations based on the center anchor point parameters to output a panoramic contour set; performing distance comparison on the panoramic contour set, removing overlapping contours, and outputting the non-repeating contour coordinate set; performing coordinate dimensionality reduction operations on the non-repeating contour coordinate set and importing it into the two-dimensional isolated space defined by the image statistical grid; extracting the spatial scalar data within the non-repeating contour coordinate set and performing boundary constraint condition comparison operations to locate and output the target grid address; performing incremental iterative operations on all acquired target grid addresses to output density count parameters; extracting the depth extremum scalar within the non-repeating contour coordinate set as the corresponding grid elevation parameter; and performing spatial topology reconstruction operations on the density count parameter and the grid elevation parameter to output the local surface density matrix.

[0011] Preferably, the specific process for generating the total global material count includes: extracting the bottom reference elevation nested within a preset depth extrapolation model; performing dimensionality reduction and unpacking on the matrix elements within the local surface density matrix to output a surface elevation matrix; performing a subtraction comparison operation between the surface elevation matrix and the bottom reference elevation to output the effective filling thickness; inputting the local surface density matrix into the depth extrapolation model for forward propagation to output an algebraic descent parameter; performing a division operation by dividing the effective filling thickness by a preset slice reference thickness to output the number of depth cutting layers; performing calculus accumulation on the local surface density matrix within the layered space defined by the number of depth cutting layers according to the algebraic descent parameter to output a particle packing algebraic value; and performing an algebraic aggregation operation on all the particle packing algebraic values ​​to output the total global material count.

[0012] Preferably, the specific process of determining the in-transit falling speed and the number of in-transit particles includes: constructing a sliding window array to drive the lateral vision sensor to perform scanning actions on the falling material and output a continuous image stream; pushing the continuous image stream into the sliding window array in sequence to perform queuing operations, separating the current extraction frame and the previous extraction frame; performing pixel difference comparison on the current extraction frame and the previous extraction frame to output an isolated pixel set; extracting the centroid of the current frame and the centroid of the previous frame within each isolated pixel set and performing spatial differentiation to obtain a spatial displacement scalar; using the spatial displacement scalar combined with a preset fixed time interval to output the in-transit falling speed; performing morphological separation on all the obtained isolated pixel sets to extract the target connected components of independent closed outer loops; performing a pixel area statistical algorithm on the target connected components to output the connected component area; dividing the connected component area by the single particle reference area to output the equivalent particle multiple; performing a traversal and cumulative calculation operation on all the equivalent particle multiples to output the number of in-transit particles in the suspended state.

[0013] Preferably, the specific generation process of the feeding interruption command includes: establishing a multi-channel variable area and importing the total global material count, the in-transit falling speed, the in-transit particle count, and the target filling quantity into each of the multi-channel variable areas; extracting the inertia compensation coefficient mapped within the preset joint judgment equation; performing a multiplication scaling operation on the in-transit falling speed and the inertia compensation coefficient to output speed-equivalent particles; performing a joint accumulation operation on the in-transit particle count, the speed-equivalent particles, and the total global material count to output the predicted final inventory; performing a difference algebra judgment operation on the predicted final inventory and the target filling quantity to obtain judgment difference data; performing a verification operation on the judgment difference data and the cutoff tolerance range set in the joint judgment equation, and outputting a threshold compliance status when the judgment difference data falls within the cutoff tolerance range; performing a communication message packet operation based on the obtained threshold compliance status to output the feeding interruption command; and transmitting the feeding interruption command to the actuator to drive the actuator to cut off the material falling trajectory.

[0014] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. By using a cross-field camera to perform alternating polarization illumination acquisition on spherical materials inside the receiving container, specular highlight images and structured light texture images are output in a decoupled manner, transforming complex visual feedback into a high signal-to-noise ratio basic data stream. In densely stacked scenarios, it can lock feature anchor points with extremely low data redundancy; it enables the sensing cycle to tightly engage with the high-speed feeding process, effectively mitigating the temporal deviation between the predicted output and the continuous falling physical dynamics from the sensing source, constructing a data base with extremely low time loss, thereby ensuring that subsequent blocking commands can smoothly converge to the target setting.

[0015] 2. By combining the a priori equation of spherical curvature, edge constraint completion operations are performed on specular highlight images and structured light texture images, thereby outputting a complete contour in a single field of view. This ensures the continuity and uniqueness of the calculation even when faced with highly incomplete physical occlusion, avoiding drastic fluctuations in processing time caused by sudden changes in local occlusion area. The smoothness and predictability of the calculation cycle eliminate computational lag in the image topology analysis process. In the data processing relay stage, the temporal deviation between the predicted output and the continuous falling physical dynamics is further bridged, ensuring that contour analysis and actual material feeding actions are synchronously mapped on the time axis.

[0016] 3. The total number of global materials, the falling speed in transit, the number of particles in transit, and the target loading quantity are combined and input into the joint judgment equation to perform a cutoff threshold comparison calculation, and then output a feeding interruption command. By introducing speed and quantity parameters that characterize physical inertia, the judgment model internalizes the objective laws of the evolution of spatial physical quantities over time, incorporates the delayed response cycle of mechanical actions into the calculation framework in advance, and resolves the time sequence deviation between the predicted output results and the physical dynamics of continuous falling from the top level of the decision-making mechanism, so as to smoothly drive the actuator to cut off the material falling trajectory with an appropriate lead time.

[0017] 4. A deep coupling mechanism from bottom-level visual perception to top-level feedforward decision-making is constructed in the overall architecture. The efficient information source generated by the polarization-alternating illumination acquisition operation performed by the cross-field-of-view camera provides a seamless data foundation for subsequent edge constraint completion calculations combined with the a priori equations of the spherical curvature, thus ensuring a stable output of the total global material count. Based on this, the static analysis results and the on-the-way falling velocity and on-the-way particle count acquired by the lateral visual sensor naturally converge within the joint decision equation. This high degree of synergy between optomechanical physical characteristics, geometric mathematical models, and dynamic feedforward logic effectively smooths out the time loss during cross-step communication and multi-dimensional data alignment, bridging the temporal discrepancy between the predicted output and the physical dynamics of continuous falling from the end-to-end collaborative level. Attached Figure Description

[0018] Figure 1 The flowchart illustrates a method for metering and filling spherical materials based on image analysis, as proposed in an embodiment of this invention. Figure 2 This is a flowchart of the dual-field optical fusion and edge completion data proposed in an embodiment of this invention application; Figure 3 This is a flowchart of the dynamic feedforward and feed cutoff judgment data proposed in the embodiments of this invention application. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] Please see Figures 1-3 The present invention provides a method for metering and filling control of spherical materials based on image analysis, the specific steps of which are as follows: Receive initial control commands and output task parameter sets. Through configuration parsing, output target loading quantity and image statistical grid. Perform polarization alternating illumination acquisition operation on spherical materials in the receiving container through a cross-field camera, and output specular highlight image and structured light texture image. Edge constraint completion is performed on the specular highlight image and structured light texture image using the pre-set spherical curvature a priori equation, and a complete contour in a single field of view is output. Image registration and deduplication are performed on the complete contour in the single field of view, and a set of contour coordinates without repetition is output. The set of contour coordinates without repetition is input into the image statistical grid to perform grid operation, and a local surface density matrix is ​​output. The preset depth inference model is invoked to perform image integral projection on the local surface density matrix and output the total number of global materials; the free-falling materials in the feeding channel are detected by the side vision sensor and the falling speed and the number of particles in the path are output. The total number of materials, the falling speed in transit, the number of particles in transit, and the target filling quantity are compared with the cutoff threshold to calculate the feeding interruption command; the feeding interruption command is transmitted to the actuator to drive the actuator to cut off the material falling trajectory.

[0021] The technical solution of the present invention will be further described in detail below with reference to specific embodiments.

[0022] Example 1 This application discloses an image analysis-based method for metering and filling spherical materials. (See attached document.) Figure 1 The specific steps proposed in this invention include: S1. Receiving the initial control command and outputting the task parameter set, and through configuration parsing, outputting the target loading quantity and image statistical grid; performing polarization alternating illumination acquisition operation on the spherical material in the receiving container through a cross-field camera, and outputting specular highlight image and structured light texture image; S2. Performing edge constraint completion on the specular highlight image and structured light texture image through a preset spherical curvature a priori equation, and outputting a complete single-field contour; performing image registration and deduplication operation on the complete single-field contour, and outputting a set of non-repeating contour coordinates; and then performing image registration and deduplication operation on the non-repeating contour. S3. The input image statistical grid of the complex contour coordinate set is used to perform grid operation and output a local surface density matrix; S4. The preset depth inference model is called to perform image integral projection on the local surface density matrix and output the total number of global materials; The free-falling materials in the feeding channel are detected by the side vision sensor and the falling speed and the number of particles in the path are output; S5. The total number of global materials, the falling speed in the path, the number of particles in the path and the target filling quantity are compared with the cutoff threshold and the feeding blocking command is output; The feeding blocking command is transmitted to the actuator to drive the actuator to cut off the falling trajectory of the material.

[0023] Further, the initial control command outputs a task parameter set, and through configuration parsing, outputs the target loading quantity and image statistical grid; a polarization-alternating illumination acquisition operation is performed on the spherical material inside the receiving container using a cross-field camera, outputting a specular highlight image and a structured light texture image; corresponding to step S1 above; the specific implementation process includes: The system receives the initial control command, performs a transformation operation on the initial control command to generate a command feature stream, performs a field truncation operation on the command feature stream to extract the task parameter set, performs a separation and decomposition operation on the task parameter set to segment and generate a main path parameter set, an auxiliary path parameter set, and timing configuration parameters, performs an addressing mapping operation on the main path parameter set to output the target loading quantity and single-particle reference area, performs a splitting and unpacking operation on the auxiliary path parameter set to separate the first feature set, and transforms the first feature set into a planar mesh structure as the image statistical mesh.

[0024] Specifically, the generation process for the target loading quantity and the image statistical grid is as follows: In the initial stage of the control cycle, the communication interface obtains batch production instructions from the host computer and receives initial control instructions. It performs conversion operations on these initial control instructions, stripping the network layer header according to a preset communication protocol stack to generate an instruction feature stream in the form of a data payload. Subsequently, it performs field extraction operations on the instruction feature stream, reading the data frame header length variable from the data stream start address and offsetting by the corresponding number of bytes to locate the data area. It then continues reading subsequent bytes until it encounters a checksum flag; the extracted data block serves as the task parameter set. During this process, the communication dictionary structure is selected by calling the underlying industrial Ethernet communication standard library file of the host computer operating system.

[0025] The task parameter set is split and decomposed in memory partitions to generate a main path parameter set, an auxiliary path parameter set, and timing configuration parameters. The main path parameter set records the material loading target; the auxiliary path parameter set records the spatial dimension configuration of visual statistics; and the timing configuration parameters are used for the synchronous triggering of subsequent hardware devices.

[0026] An addressing mapping operation is performed on the main path parameter set. The underlying machine value representing the current target weight, stored at a specific byte address within the main path parameter set, is read. This machine value is multiplied by a set physical equivalent coefficient, and the resulting product is used as the output target loading quantity. The physical equivalent coefficient is selected by taking the average slope of multiple fitting calibration experiments conducted in a standard laboratory environment using actual material weighing weights and statistical quantities. Simultaneously, the single-particle reference area is calculated. First, the single-pixel physical scale parameter (i.e., the actual physical length represented by a single pixel) is extracted based on the pre-calibrated camera hardware physical resolution and installation height. The known physical diameter parameter is divided by this single-pixel physical scale parameter to obtain the single-pixel diameter. Then, the single-pixel diameter is multiplied by itself and multiplied by a constant of 1 / 4 of pi. The resulting floating-point result is rounded to the nearest integer, and the final output is the single-particle reference area value.

[0027] A splitting and unpacking operation is performed on the auxiliary path parameter set to separate the first feature set, which contains the three-dimensional physical dimension information of the target filling container. This first feature set is then converted into a planar mesh structure as the image statistical mesh. The specific calculation process for mesh structure conversion is as follows: the physical length and width values ​​of the container are divided by the physical dimension value of the set minimum statistical unit, respectively, to obtain the number of equal divisions of the target monitoring image in the horizontal and vertical directions. A two-dimensional matrix structure is then constructed based on these equal divisions. The physical dimension value of the minimum statistical unit (i.e., the mesh division accuracy) is selected by extracting the physical diameter value of the material and multiplying it by a preset spatial tolerance coefficient. This spatial tolerance coefficient is selected based on the empirical value of the maximum porosity when spherical materials are randomly stacked; in this embodiment, it is fixed at 1.2.

[0028] The instruction feature stream is separated, decomposed, and unpacked to extract a structured multidimensional feature set, which provides support for the reconstruction of the underlying model and equations and ensures the efficient flow of initialization configuration.

[0029] Extract the timing configuration parameters encapsulated in the task parameter set; perform frequency division calculations using the timing configuration parameters to output a main trigger pulse; combine the main trigger pulses to drive a polarized illumination source to perform an illumination action; drive the cross-field camera to perform an exposure capture action and output original image stream data when the illumination action is enabled; perform brightness truncation filtering on the original image stream data to remove background interference information and output a set of bright pixels as the specular highlight image; perform secondary frequency division calculations using the timing configuration parameters to output a secondary trigger pulse; combine the secondary trigger pulses to enable the grid projection array to perform a projection action; drive the cross-field camera to perform a secondary exposure capture action and output secondary image stream data when the projection action is enabled; perform distortion correction filtering on the secondary image stream data to enhance edge transition coordinates and output the structured light texture image.

[0030] Specifically, the generation process of specular highlight images and structured light texture images is as follows: The reference clock frequency, encapsulated in the timing configuration parameters in memory, is extracted. The microcontroller's timer performs a frequency division operation using these timing configuration parameters, dividing the reference clock frequency by a set division factor. The resulting quotient is used as the output frequency of the main trigger pulse. This division factor is selected based on half of the highest supported stable acquisition frame rate recorded in the camera hardware manual. The duty cycle of the main trigger pulse is determined based on the heat dissipation limit and thermal equilibrium safe operation curve of the LED; in this embodiment, it is set to 10%.

[0031] Combined with the main trigger pulse, a polarized illumination source consisting of a ring array with vertical linear polarizers is driven to perform an illumination operation. While the illumination operation is active, the cross-field camera is simultaneously driven to perform an exposure capture operation, outputting original image stream data. The exposure time parameter is selected as follows: the maximum possible falling speed of the extracted material is calculated, the physical size corresponding to a single pixel is divided by this maximum speed value, and to prevent motion blur, half of the resulting value is used as the set exposure time.

[0032] In this embodiment, the non-illuminated edge areas of the original image stream data are extracted, the ambient baseline grayscale mean is calculated, and this mean is compared with a preset reference grayscale to generate a global gain coefficient. This coefficient is then used to perform pixel-level multiplicative scaling on the original image stream data to eliminate ambient light fluctuations. Specifically, for the output original image stream data, a rectangular annular region with a width of 10 pixels around the image edges is defined in memory as a non-primary illumination sensitive area. The grayscale values ​​of all pixels within this rectangular annular region are accumulated and divided by the total number of pixels in the region to output the ambient baseline grayscale mean. The preset reference grayscale value stored in the read-only memory during system initialization is retrieved, and the preset reference grayscale value is divided by the calculated ambient baseline grayscale mean to obtain a floating-point global gain coefficient. If this coefficient falls within the range of 0.8 to 1.2, it is used; if it is below 0.8, it is forcibly assigned a value of 0.8; if it is above 1.2, it is forcibly assigned a value of 1.2. Using this global gain coefficient, each pixel of the original image stream data is traversed, and multiplicative scaling operations are performed. After the calculation, the rounding function is called to round down, and a grayscale overflow protection condition is set: pixel values ​​exceeding 255 are forcibly truncated to 255. After completing the above traversal, the original memory address is overwritten to generate the original image stream data after ambient light compensation. The above process cancels out the local grayscale disturbances caused by the drift of stray light from the external environment over time.

[0033] The original image stream data undergoes brightness truncation filtering. The grayscale value of each pixel in the image is read row by row and column by column. A preset brightness truncation threshold is subtracted from the read grayscale value. If the difference is greater than 0, the grayscale value of the pixel is reset to the maximum allowed grayscale value. If the difference is less than or equal to 0, the grayscale value of the pixel is forcibly assigned to 0. This process removes background interference and outputs a set of bright pixels as the specular highlight image. The brightness truncation threshold is selected by taking multiple sets of images under a reference light source in the working environment during idle operation, extracting the maximum background grayscale value from the statistical distribution, and adding a certain safety margin (in this embodiment, the safety margin is set to 15 to 20 grayscale levels) before determining the threshold.

[0034] During the low-level phase of the main pulse, a secondary frequency division operation is performed using the timing configuration parameters to output a secondary trigger pulse. The phase difference parameter of the secondary trigger pulse is selected to be half a trigger cycle to ensure that the secondary trigger pulse and the main trigger pulse are completely staggered on the time axis. Combined with the secondary trigger pulse, the grid projection array is activated to perform projection, projecting structured light in the form of sinusoidal wavy stripes, and driving the camera to perform a secondary exposure capture operation to output secondary image stream data.

[0035] Distortion correction filtering is performed on the secondary image stream data. First, the Euclidean distance from the original pixel to the optical center of the image is calculated. When calculating the radial correction offset, the horizontal radial correction offset is equal to the original horizontal coordinate multiplied by (the square of the Euclidean distance multiplied by the set first radial distortion coefficient); the vertical radial correction offset is equal to the original vertical coordinate multiplied by (the square of the Euclidean distance multiplied by the set first radial distortion coefficient). Next, the tangential correction offset is calculated based on geometric relationships. The specific process is as follows: extract the first tangential distortion coefficient and the second tangential distortion coefficient; the horizontal tangential correction offset is equal to twice the product of the first tangential distortion coefficient and the original horizontal and vertical coordinates, plus the sum of the second tangential distortion coefficient multiplied by the square of the Euclidean distance and twice the square of the horizontal coordinate; the vertical tangential correction offset is equal to the sum of the first tangential distortion coefficient multiplied by the square of the Euclidean distance and twice the square of the vertical coordinate, plus twice the product of the second tangential distortion coefficient and the original horizontal and vertical coordinates; the final corrected pixel horizontal coordinate is the original horizontal coordinate plus the sum of the horizontal radial correction offset and the horizontal tangential correction offset; the corrected pixel vertical coordinate is the original vertical coordinate plus the sum of the vertical radial correction offset and the vertical tangential correction offset.

[0036] The radial and tangential distortion coefficients were selected by taking multiple shots within the camera's field of view using a standard black and white checkerboard calibration board, and then calculating the matrix using Zhang's camera calibration algorithm.

[0037] After correction, a high-pass filter operator is used to perform spatial convolution calculation. The gray value of the center pixel is multiplied by a specific weight, and then the sum of the gray values ​​of its surrounding neighboring pixels is subtracted to output the structured light texture image. The specific weight is selected by making it equal to the total number of selected surrounding neighboring pixels.

[0038] By performing frequency division calculations through the clock module to output primary and secondary trigger pulses, crosstalk between polarized illumination and grid projection light fields is avoided, enabling a stable output of a pure set of high-brightness pixels and structured light edge transition coordinates.

[0039] Furthermore, edge constraint completion is performed on the specular highlight image and structured light texture image using a pre-defined spherical curvature a priori equation, outputting a complete single-viewpoint contour; image registration and deduplication are performed on the complete single-viewpoint contour, outputting a unique contour coordinate set; the unique contour coordinate set is input into the image statistical grid for grid operation, outputting a local surface density matrix; this corresponds to step S2 above; see [link to relevant documentation]. Figure 2 The specific implementation process includes: The specular highlight image is segmented using connected components to extract the extremum centroid vector; the optical axis angle matrix bound to the preset spherical curvature a priori equation is extracted; spatial offset transformation is performed on the extremum centroid vector and the optical axis angle matrix to output the spherical center reference coordinates; contour separation is performed on the structured light texture image to output the incomplete contour matrix; the spherical center reference coordinates and the incomplete contour matrix are combined into a joint parameter set and input into the spherical curvature a priori equation; the standard radius parameter contained in the spherical curvature a priori equation is extracted and combined with the spherical center reference coordinates to perform curve fitting operation to output the virtual complete set; the incomplete contour matrix and the virtual complete set are combined to perform matrix merging operation to output the single-viewpoint complete contour.

[0040] Specifically, the construction process of the complete outline in a single field of view is as follows: A connected component segmentation algorithm is performed on the specular highlight image to calculate the pixel area of ​​each connected component. If the area of ​​a connected component is greater than the preset upper limit of the single-spot highlight reference area, it is determined to be a multi-spot adhesion, and then the watershed algorithm based on distance transform is called to perform forced segmentation on the adhesion connected component. Then, for each segmented independent sub-region, the horizontal coordinate values ​​corresponding to all the valid pixels in the connected region are accumulated one by one to obtain the sum of the horizontal coordinates. Then, the sum of the horizontal coordinates is divided by the total number of valid pixels in the connected region, and the quotient is used as the horizontal coordinate of the extreme centroid. Similarly, the vertical coordinate of the extreme centroid is the quotient obtained by dividing the sum of the vertical coordinate values ​​of all valid pixels by the total number of valid pixels.

[0041] Extract the optical axis angle matrix pre-stored in the memory, perform spatial offset transformation calculation on the extreme centroid coordinates and the optical axis angle matrix, integrate the horizontal and vertical centroid coordinates obtained from the image to form a two-dimensional vector, perform matrix multiplication operation on the two-dimensional vector and the inverse matrix of the camera's internal parameters to obtain the spatial ray vector in the line of sight direction, and then subtract the fixed spatial translation vector derived from the geometric angle between the physical position of the light source and the principal optical axis of the camera from the spatial ray vector, calculate and output the reference coordinates of the physical sphere center in the three-dimensional world coordinate system in the memory.

[0042] The structured light texture image is subjected to contour separation operation, which outputs a fragmented contour matrix containing multiple independent fragmented arcs. Based on the principle of shortest spatial distance, each fragmented arc in the fragmented contour matrix is ​​traversed and matched to the sphere center reference coordinates with the closest Euclidean distance. The matched and bound sphere center reference coordinates are combined with the corresponding assigned fragmented contour matrix to form a joint parameter set. The selection method for high and low threshold parameters in the contour separation operation is as follows: the high threshold is selected by extracting the gradient values ​​of the top 10% of the pixel gray-level gradient histogram distribution of the entire image as the judgment criterion, and the low threshold is fixed in the program as 1 / 3 of the high threshold value.

[0043] The joint parameter set is input into the preset a priori equation for the curvature of the sphere. The logical calculation process within the a priori equation for the curvature of the sphere is as follows: using the camera's internal parameter matrix, the reference coordinates of the obtained physical sphere center in the three-dimensional world coordinate system are reprojected onto the two-dimensional image coordinate system to obtain the sphere center projection pixel coordinates. Subsequently, in the set two-dimensional coordinate system, any pixel belonging to the boundary is found, and the difference between the horizontal coordinate value of the boundary pixel and the obtained horizontal coordinate value of the sphere center projection pixel is calculated, and the square of the difference is obtained; at the same time, the difference between the vertical coordinate value of the boundary pixel and the vertical coordinate value of the sphere center is calculated, and the square of the difference is obtained; then, the two squared values ​​are added to obtain the sum of the squares of the coordinate deviations. The cross-field camera uses a telecentric lens (to achieve orthogonal projection and eliminate perspective distortion), or in the image preprocessing stage, it is limited to extracting only image data located within the effective field of view area at the center of the principal optical axis (with a projection distortion rate lower than a preset tolerance range) for subsequent a priori equation calculations. Under the premise of satisfying the above circular projection, the sum of the squares of the coordinate deviations must be numerically equivalent to the square of the set standard radius parameter of the sphere itself. The standard radius parameter is selected by dividing the known physical nominal diameter of the material by 2 to obtain the physical radius, and then dividing it by the single-pixel physical equivalent scale of the camera's field of view.

[0044] The specific generation process of the virtual completion set is as follows: extract the starting angle and ending angle of the two broken endpoints in the incomplete contour matrix relative to the reference coordinates of the sphere center; within the arc interval defined by the starting angle and the ending angle, perform a cyclic iteration operation according to the preset angle step size (set to 1 degree in this embodiment); in each iteration, call the trigonometric function calculation library, add the product of the standard radius and the cosine value of the current angle to the lateral coordinate of the sphere center to obtain the virtual lateral coordinate, add the product of the standard radius and the sine value of the current angle to the longitudinal coordinate of the sphere center to obtain the virtual longitudinal coordinate; form a virtual completion set by combining all the virtual coordinate points output by the cyclic iteration, and perform matrix merging operation with the incomplete contour matrix to output the single-view complete contour.

[0045] By performing spatial offset transformation on the angle matrix between the extreme centroid vector and the optical axis, the inherent geometric displacement error of the cross optical field of view is corrected, and the resulting sphere center reference coordinates are used to determine the spatial anchor point determined by the curve fitting operation.

[0046] Extract the center anchor point parameters within all received single-view complete contours, and perform spatial projection operations based on the center anchor point parameters to output a panoramic contour set; perform distance comparison on the panoramic contour set, remove overlapping contours, and output the non-repeating contour coordinate set; perform coordinate dimensionality reduction operations on the non-repeating contour coordinate set and import it into the two-dimensional isolated space defined by the image statistical grid; extract the spatial scalar data within the non-repeating contour coordinate set, perform boundary constraint condition comparison operations to locate and output the target grid address; perform incremental iterative operations on all acquired target grid addresses to output density count parameters; extract the depth extremum scalar within the non-repeating contour coordinate set as the corresponding grid elevation parameter; perform spatial topology reconstruction operations on the density count parameter and the grid elevation parameter to output the local surface density matrix.

[0047] Specifically, the process of generating the local surface density matrix is ​​as follows: Extract the center coordinates of the spheres bound within the complete single-view field contour as the center anchor point parameter. Perform spatial projection operation based on the center anchor point parameter. The specific process is as follows: use the external parameter translation vector and rotation matrix obtained by calibration to perform matrix multiplication and addition operations on the center anchor point parameter, and convert its original camera coordinate system coordinate values ​​into horizontal two-dimensional projection coordinates in the world coordinate system, thereby outputting the panoramic contour set.

[0048] In this embodiment, each independent contour within the panoramic contour set is traversed, and the aspect ratio parameter of its smallest bounding rectangle is extracted. This is then compared and filtered using a preset sphericity tolerance range to eliminate abnormal contours caused by irregular occlusion or jagged edges. Specifically, the coordinate points of individual contours are extracted one by one from the panoramic contour set by index. All horizontal and vertical coordinate sets contained in the current contour are traversed, and the maximum and minimum horizontal and vertical coordinates are found respectively. The pixel width of the bounding rectangle is calculated by subtracting the minimum horizontal coordinate from the maximum horizontal coordinate, and the pixel height of the bounding rectangle is calculated by subtracting the minimum vertical coordinate from the maximum vertical coordinate. The pixel width is divided by the pixel height to perform a division operation, and a floating-point aspect ratio parameter is output. A preset sphericity tolerance range is extracted, with a fixed lower threshold of 0.85 and a fixed upper threshold of 1.15. The calculated aspect ratio parameter is compared with the specified interval. If the aspect ratio parameter is greater than or equal to 0.85 and less than or equal to 1.15, the individual contour is retained and pushed into a newly created high-confidence cache stack. If it is not within this interval, the memory address occupied by the contour is directly released. After traversal, the data in the high-confidence cache stack completely overwrites the original panoramic contour set. The above process filters out non-spherical interference features and severe distortion artifacts, reducing the unnecessary computational overhead of subsequent coordinate dimensionality reduction and mesh calculation.

[0049] The non-maximum suppression (NMS) comparison operation is then performed on the panoramic contour set to remove overlapping contours. The Euclidean distance calculation in the NMS comparison operation is as follows: the square of the difference in the lateral coordinates of the two contour center anchor points is added to the square of the difference in their longitudinal coordinates, and the square root of the sum is taken to obtain the straight-line distance between the two points. The exclusivity threshold is selected based on the physical constraint that two solid spheres cannot overlap in space with a depth exceeding half their own physical radius; its parameter is set to 0.5 times the standard radius parameter value. If the distance is less than this exclusivity threshold, a removal action is triggered, and the set of coordinates of the non-repeating contours is output.

[0050] The set of unique contour coordinates is subjected to coordinate dimensionality reduction operation, discarding vertical height information, and then imported into the image statistical grid. Scalar data in the horizontal and vertical spaces are extracted and subjected to boundary constraint comparison operations. It is determined whether the extracted horizontal scalar value is greater than and equal to the coordinate value of the left vertical boundary line of the current specific statistical grid, and strictly less than the coordinate value of the right vertical boundary line of the grid; simultaneously, it is determined whether the vertical scalar value is greater than and equal to the coordinate value of the upper horizontal boundary line of the grid, and strictly less than the coordinate value of the lower horizontal boundary line of the grid. If both logical judgments are true, the target grid address is located and output.

[0051] For all target mesh addresses, perform incremental iterative operations in their memory accumulators (i.e., each time the internal count is triggered, the base number is increased by 1), and output the density count parameter; at the same time, extract the reference coordinates of the physical sphere center obtained above in the three-dimensional world coordinate system, strip the height axis component value, use it as the depth extremum scalar corresponding to the contour, and assign it to the corresponding mesh elevation parameter.

[0052] Perform a spatial topology reconstruction operation on the density counting parameter and the grid elevation parameter. Define a three-dimensional tensor region in memory. Fill the index corresponding to the first depth channel of the tensor with the two-dimensional density counting parameters of all the aforementioned grids row by row and column by column. Fill the index corresponding to the second depth channel of the tensor with the two-dimensional grid elevation parameters in the same row printing order. Combine and output a local surface density matrix.

[0053] Overlapping records of the panoramic contour are eliminated through distance comparison calculations, and the coordinates are reduced to scalar values ​​and imported into an isolated space for incremental iteration. Combined with spatial topology reconstruction calculations based on mesh elevation parameters, particle projection interference from a planar perspective is resolved with low computational overhead, and a logically clear local surface density matrix is ​​constructed.

[0054] Furthermore, a preset depth inference model is invoked to perform image integral projection on the local surface density matrix, outputting the total global material count; a side vision sensor is used to detect free-falling material in the feeding channel, outputting the falling speed and the number of particles in transit; corresponding to step S3 above; the specific implementation process includes: Extract the bottom reference elevation nested within the preset depth extrapolation model; perform dimensionality reduction and unpacking on the matrix elements within the local surface density matrix to output the surface elevation matrix; perform a subtraction comparison operation between the surface elevation matrix and the bottom reference elevation to output the effective filling thickness; input the local surface density matrix into the depth extrapolation model for forward propagation to output the algebraic descent parameter; divide the effective filling thickness by the preset slice reference thickness to output the depth cutting layer number; perform calculus accumulation operation on the local surface density matrix within the layer space defined by the depth cutting layer number according to the algebraic descent parameter to output the particle packing algebra value; perform algebraic aggregation operation on all the particle packing algebra values ​​to output the global total material count.

[0055] Specifically, the process for generating the total number of global materials is as follows: This implementation discloses the necessary network layers, hierarchical connections, and training parameters of a pre-defined deep inference model. The pre-defined deep inference model includes specific layers and connections. Its input layer receives a dual-channel local surface density matrix tensor. The subsequent feature extraction network contains cascaded two-dimensional convolutional layers. The number of convolutional kernels and neurons (e.g., 64 in the first layer, 128 in the second layer) is selected by optimizing the loss values ​​on the validation set during offline development using a hyperparameter grid search algorithm, choosing the network structure node configuration that minimizes the loss value. A linear rectified function is used as the activation function, with the following logic: determining the relationship between the input value and 0; if the input value is greater than 0, the original value is output; if the input value is less than or equal to 0, the value 0 is forcibly output. A max-pooling layer is concatenated after the convolutional layers to extract gradient distribution features layer by layer. The fully connected inference layer flattens the output of the convolutional layers before connecting it to the hidden layer. The final output layer contains a neuron array corresponding to the grid size of the local surface density matrix, used to directly output the algebraic descent parameter matrix. Regarding the specific steps and parameters for training the deep inference model, the dataset construction process is as follows: Multiple sets of stepped material filling are performed into a standard physical container according to a set step size (e.g., an increase of 100 particles per step), within the range from zero to the target full load quantity. In each step filling state, physical vibration or airflow disturbance is applied to the container to generate various random stacking patterns. Local areal density feature matrices corresponding to these various states are collected using a camera (accumulating at least N sets of samples), and the manually verified absolute true quantities are converted into corresponding labels. The original dataset is then constructed. A random shuffling operation is performed on the original dataset, and it is divided into a training set, a validation set, and a test set according to a preset ratio (e.g., 8:1:1). The training set is used to input the deep inference model to perform forward and backward propagation operations, thereby driving the gradient update of the weights and bias parameters within the network. The validation set is used to perform forward inference calculations on the model after each training cycle to obtain the generalization error, which serves as a data benchmark for assisting in adjusting the learning rate hyperparameters and triggering the early stopping mechanism. The test set serves as an independent validation sample that is completely hidden from the model throughout the entire training and parameter optimization phase. It is used to perform a final evaluation operation on the model after the early stopping condition is met and the network weights are derived.

[0056] In the hyperparameter configuration phase, an adaptive moment estimation optimizer was selected. The initial learning rate was set to 0.001. The learning rate decay strategy involved setting a monitoring window; if the error value on the validation set did not decrease for 10 consecutive training epochs, the program multiplied the current learning rate value by 0.1 to obtain a new learning rate. The batch size parameter was selected based on the maximum capacity of the computing hardware memory used for model training, choosing an integer power of 2 without causing memory overflow.

[0057] Regarding the evaluation of the loss function during training, the calculation process is described in detail as follows: First, the regression prediction coefficient value output by the current forward propagation operation of the deep model is subtracted from the true label coefficient value obtained by manual calibration to obtain the prediction bias of a single sample; then, the prediction bias is multiplied by itself to obtain the squared value of the bias; the squared bias values ​​corresponding to all samples in the current training batch are accumulated one by one to obtain the batch bias sum; finally, the batch bias sum is divided by the total number of samples in the batch, and the calculated arithmetic mean is the current batch loss value used for backpropagation. The selection method for the maximum training cycle and the early stopping mechanism time threshold is established based on mathematical statistical observation of the convergence inflection point positions of similar bulk material accumulation datasets during historical training. When the early stopping condition is met, the program terminates the iteration and exports the network weights that meet the condition as the basis for online calculation.

[0058] Dimensionality reduction and unpacking are performed on the matrix elements inside the local surface density matrix to split and output the surface elevation matrix.

[0059] The numerical values ​​of each grid element in the surface elevation matrix are compared with the extracted bottom reference elevation by subtraction. The height value recorded at a specific grid coordinate in the surface elevation matrix is ​​read, and the corresponding bottom reference elevation value at the same coordinate is retrieved from memory. The algebraic difference between the former and the latter is output as the effective filling thickness. This bottom reference elevation value is selected by performing a global static elevation scan to obtain zero-position calibration 3D data under a completely empty physical background condition during initial installation.

[0060] The local surface density matrix is ​​input into the input layer of the deep inference model. Inside the model, the data is sequentially processed by a two-dimensional convolutional layer for feature extraction, a linear rectified function for nonlinear activation, a max pooling layer for dimensionality reduction, and a fully connected inference layer to flatten the output of the convolutional layer before connecting it to the hidden layer. Finally, the output layer contains a neuron array corresponding to the grid size of the local surface density matrix, which is used to directly output the algebraic descent parameter matrix.

[0061] The calculated effective fill thickness is divided by a preset slice reference thickness. The slice reference thickness parameter is selected based on the face-centered cubic close-packed geometry theory in space crystallography, and is consistently set to 1.632 times the standard radius of a sphere. The calculation process for the number of depth cutting layers is as follows: the effective fill thickness is divided by the slice reference thickness, and the resulting floating-point quotient is rounded down, discarding all decimal places. The remaining integer value is output as the number of depth cutting layers. The decimal part of the division is retained as the residual layer coefficient. After calculus accumulation of all pure integer layers, the original two-dimensional density count value of the outermost layer is multiplied by the residual layer coefficient to obtain the top layer residual estimate value, which is then accumulated into the series sum variable according to the addition rules.

[0062] After the depth extrapolation model outputs the algebraic descent parameter, the calculated depth cutting layer number is extracted. The number 1 is divided by this depth cutting layer number to obtain the reciprocal of the physical extreme value. A threshold pruning operation is performed on the algebraic descent parameter output by the network: if the parameter is greater than the reciprocal of the physical extreme value, it is forcibly assigned the value of the reciprocal of the physical extreme value; if the parameter is less than 0, it is forcibly assigned the value of 0. This ensures that the layering attenuation factor of the bottom layer material at the maximum depth cutting layer number decays to the lower limit without algebraic negative overflow.

[0063] The local surface density matrix is ​​subjected to calculus accumulation within the layered space defined by the depth cutting layers, according to the algebraic descent parameter. For any independent two-dimensional mesh, a series sum variable is first initialized in the computational memory and assigned a value of 0. Then, a controlled loop operation is set, the total number of loops of which is equal to the depth cutting layers calculated above. In each set loop operation, the program counter value represents the current slice depth level value. The current slice depth level value is multiplied by the algebraic descent parameter output by the network inference to obtain the first product value, and then the first product value is subtracted from the fixed number 1 to obtain the slice depth level value. The attenuation factor of the layer is calculated; a lower limit safety check is performed on the attenuation factor of the layer: if the calculation result is less than 0, it is forcibly assigned to 0 to ensure that no algebraic negative value is generated after the deep physical occlusion is completely attenuated; then the original two-dimensional density count value of the outermost layer of the independent grid is extracted, and multiplied with the attenuation factor of the layer just calculated to obtain the layered particle estimation value of the specific layer; the layered particle estimation value is accumulated into the previous series sum variable according to the addition rule; after the controlled loop operation calculates all virtual slice layers from the bottom layer of the container to the top layer, the final accumulated value of the series sum variable is output as the particle packing algebra value corresponding to the grid.

[0064] An algebraic aggregation operation is performed on all the particle packing algebra values. This involves adding the corresponding particle packing algebra values ​​output by each independent grid, and the sum of all values ​​is output as the total global material count.

[0065] The surface elevation matrix is ​​compared with the bottom datum elevation by subtraction to obtain the true effective filling thickness. Algebraic descent parameter is used to perform calculus accumulation within the layered space, resolving the paradox of multiple solutions in a two-dimensional perspective.

[0066] A sliding window array is constructed to drive the lateral vision sensor to perform scanning actions on falling materials and output a continuous image stream. The continuous image stream is pushed into the sliding window array in sequence for queuing operations to separate the current extraction frame from the previous extraction frame. Pixel difference comparison is performed on the current extraction frame and the previous extraction frame to output isolated pixel sets. The centroid of the current frame and the centroid of the previous frame within each isolated pixel set are extracted and spatially differentiated to obtain a spatial displacement scalar. The spatial displacement scalar is combined with a preset fixed time interval to output the in-transit falling speed. Morphological separation is performed on all the obtained isolated pixel sets to extract target connected components with independent closed outer loops. A pixel area statistical algorithm is performed on the target connected components to output the connected component area. The connected component area is divided by the single particle reference area to output the equivalent particle multiple. A traversal accumulation calculation operation is performed on all the equivalent particle multiples to output the number of in-transit particles in the suspended state.

[0067] Specifically, the process of determining the falling speed and the number of particles en route is as follows: A sliding window array mechanism is constructed to drive a side vision sensor to perform scanning actions on falling materials and output a continuous image stream. The continuous image stream is pushed into the sliding window array according to the occurrence sequence based on the underlying millisecond-level timestamps to perform queuing operations, separating the current extraction frame from the previous extraction frame.

[0068] A pixel difference comparison is performed between the current extraction frame and the previous extraction frame. This involves traversing each pair of corresponding pixels in the extracted image that are at the same absolute two-dimensional coordinate position, obtaining the grayscale value of the target point in the current extraction frame, and simultaneously obtaining the grayscale value of the same target point in the previous extraction frame. The two grayscale values ​​are subtracted to obtain the difference data. Then, an absolute value stripping operation is performed on this difference data. If the result of the absolute value stripping operation is greater than a set background noise threshold, the pixel is marked as a valid moving pixel; otherwise, it is marked as static background. This logic filters out the set of isolated pixels. The background noise threshold parameter is selected as follows: In a static standby state with no feeding and no material falling, the side vision sensor is activated to collect the peak amplitude of the maximum random grayscale fluctuation generated by stray ambient light illuminating the transparent feed tube wall. This peak amplitude, plus a compensation constant, is used as the threshold.

[0069] In this embodiment, spatial clustering purification based on local density is performed on the isolated pixel set. The neighborhood density of each valid pixel is calculated. Pixels with a density lower than a set threshold are identified as dust noise and their coordinates are removed, resulting in a clean set of isolated pixels. Specifically, Euclidean distance retrieval is performed on all valid moving pixel coordinates recorded within the output isolated pixel set in two-dimensional space. The local neighborhood search radius is set to 3 pixels. Each central target pixel is traversed, and the straight-line distance from the remaining pixels to the central target pixel is calculated. The total number of pixels whose distance is less than or equal to the search radius is counted and used as the neighborhood density value of the central target pixel. A preset density lower threshold is extracted. This threshold is based on a single-pixel reference area and is set to 5% of the number of pixels corresponding to the single-pixel reference area (rounded to 6 pixels). All calculated neighborhood density values ​​are compared with this density lower threshold. If the neighborhood density value of a central target pixel is strictly less than 6, the valid flag of the target pixel is cleared in the memory linked list and its coordinates are removed; if it is greater than or equal to 6, it is retained. Finally, all flag bits are reassembled into valid coordinate points, and the isolated pixel set is updated and re-output. This process eliminates the interference from stray pixels caused by tiny dust particles accompanying free fall.

[0070] The centroids of the current frame and the previous frame are extracted from each set of isolated pixels and their spatial derivatives are calculated. For any centroid of the current frame in the set of isolated pixels, a nearest neighbor search using Euclidean distance is performed in the set of isolated pixels of the previous frame. A maximum matching distance threshold is set (in this embodiment, the pixel displacement is no more than 10% of the window height). If the distance to the nearest centroid of the previous frame is less than this threshold, the two are determined to be the same corresponding target centroid. The vertical coordinate values ​​of the target centroids calculated in the current frame are read, and the vertical coordinate values ​​of the same corresponding target centroids in the previous frame are read. The two are subtracted, and the absolute value is calculated to obtain the pure pixel displacement difference value. Then, the pure pixel displacement difference value is multiplied by the physical distance equivalent parameter represented by a single pixel obtained by camera calibration. The product is output as a spatial displacement scalar.

[0071] The calculation process for the on-the-way descent velocity is as follows: the obtained spatial displacement scalar value is divided by a preset fixed time interval value to calculate the instantaneous velocity value corresponding to each isolated pixel set; then, the maximum value (or arithmetic mean) among all such instantaneous velocity values ​​is extracted and output as the globally unified on-the-way descent velocity to ensure a safety margin for inertial compensation calculation. The fixed time interval value is selected by directly retrieving the sampling frequency parameter set in the tracking sensor hardware driver file, dividing the number 1 by the frequency parameter and taking its reciprocal.

[0072] Morphological separation is performed on all isolated pixel sets to extract the target connected components of independent closed outer loops. In the morphological separation operation, the size parameter of the cross-shaped structuring element is selected as follows: the smallest odd-numbered pixel matrix (e.g., a 3x3 pixel matrix) that can maintain the topological closure of the connected components of the moving object's trajectory while being able to cut off noise interference bridges connected by only a single pixel using a mathematical erosion algorithm.

[0073] A pixel area calculation algorithm is performed on the target connected component. The connected component area calculation process is as follows: First, the specific boundary pixel coordinates of the independent closed outer loop are located. Then, a polygon region filling scanning algorithm is used to traverse each pixel position completely enclosed within the boundary coordinates. Each time a non-zero pixel that meets the marking condition is traversed, the value of the area counter in the register is incremented by 1 until the pointer has traversed all the specified internal pixels. The final locked value of the area counter register is then output as the connected component area.

[0074] The calculation outputs the equivalent particle multiple. It retrieves the statistically obtained area value of the connected region and extracts the pre-saved baseline area value of a single particle. Dividing the area value by the baseline value yields a quotient containing a decimal part. Then, a rounding mathematical function is called to perform an integer operation on the quotient, rounding it to the nearest integer. The output pure integer result is the equivalent particle multiple. A traversal and accumulation calculation is performed on all detected equivalent particle multiples to output the number of particles in transit in the suspended state.

[0075] By employing pixel differentiation and spatial differentiation operations, the true in-transit falling speed is captured. Combined with a division mapping between the area of ​​the connected region and the reference area of ​​a single particle, visual deception caused by material adhesion is effectively decoupled, converting aliased pixels into multiples of equivalent particles and avoiding missed detection of suspended materials.

[0076] Further, the total number of materials, the in-transit falling speed, the number of particles in transit, and the target filling quantity are compared with a cutoff threshold calculation, and a feeding stop command is output; the feeding stop command is transmitted to the actuator, driving the actuator to cut off the material falling trajectory; this corresponds to step S4 above; see [link to relevant documentation]. Figure 3 The specific implementation process includes: A multi-channel variable area is established to import the total global material quantity, the in-transit falling speed, the number of in-transit particles, and the target filling quantity into each of the multi-channel variable areas. The inertial compensation coefficient mapped within the preset joint judgment equation is extracted. A multiplication scaling operation is performed on the in-transit falling speed and the inertial compensation coefficient to output velocity-equivalent particles. A joint accumulation operation is performed on the number of in-transit particles, the velocity-equivalent particles, and the total global material quantity to output the predicted final inventory. A difference algebra judgment operation is performed on the predicted final inventory and the target filling quantity to obtain judgment difference data. A verification operation is performed on the judgment difference data and the truncation tolerance range set in the joint judgment equation. When the judgment difference data falls within the truncation tolerance range, a threshold compliance status is output. Based on the obtained threshold compliance status, a communication message packet encapsulation operation is performed to output the feeding blocking command. The feeding blocking command is transmitted to the actuator to drive the actuator to cut off the material falling trajectory.

[0077] Specifically, the process for generating the feed interruption command is as follows: A multi-channel variable area is established, into which the total global material quantity, the in-transit falling speed, the in-transit particle count, and the target filling quantity are imported. The inertial compensation coefficient mapped internally by the joint judgment equation is extracted. The inertial compensation coefficient parameter is selected as follows: during the factory assembly and debugging phase, while sending a trigger closing level signal to the pneumatic solenoid valve control port, an external high-speed video recording and analysis device is activated to capture the absolute physical time span from the moment the level signal is emitted until the moment the pneumatic metal baffle undergoes a complete and tight mechanical closing action. This time span is extracted and solidified as the inertial compensation coefficient.

[0078] A multiplication scaling operation is performed on the in-transit falling speed and the inertia compensation coefficient. The calculated in-transit falling speed variable value is retrieved and multiplied with the set inertia compensation coefficient variable value representing the physical time span. The product value output represents the additional physical displacement of the material at the lower end of the feed port based on the current falling speed during the physical time of the mechanical actuator's lag. A pre-set feed channel cross-sectional equivalent distribution coefficient is extracted. This physical displacement value is divided by the single-particle pixel diameter derived from the square root of the single-particle reference area and multiplied by the feed channel cross-sectional equivalent distribution coefficient. The calculated product result is rounded to the nearest integer and directly output as the velocity equivalent particle estimated value. The feed channel cross-sectional equivalent distribution coefficient is determined by extracting the actual internal physical cross-sectional area value of the feed channel, dividing it by the actual physical cross-sectional area value of a single particle, and multiplying it by a preset material stacking cross-sectional utilization rate coefficient (obtained from multiple discharge experiments in this embodiment).

[0079] A joint accumulation operation is performed on the number of particles in transit, the velocity-equivalent particles, and the total global material quantity. The total global material quantity at the bottom of the filling container is extracted as the first addend. The number of particles in transit that are falling in the air in the feeding channel is extracted as the second addend. The velocity-equivalent particles that will continue to leak out based on the prediction of slow mechanical execution are extracted as the third addend. The three addends are then continuously added. The arithmetic sum of these three addends is output as the predicted final inventory.

[0080] A difference algebra judgment operation is performed on the predicted final inventory and the target loading quantity. The target loading quantity value set in the initialization stage is continuously read and used as the arithmetic minuend. The predicted final inventory value calculated in real time within the current control cycle is used as the arithmetic subtrahend. The difference result obtained by the operation is the judgment difference data.

[0081] The judgment difference data and the truncation tolerance range set in the joint judgment equation are used for verification calculation. The truncation tolerance range parameter is selected by comprehensively defining it based on the tolerance requirements for excess or insufficient packaging material quantity in the product quality inspection specifications, combined with the allowable positive and negative quantity deviation limit threshold of the downstream order, and outputting it as an upper and lower limit value range. The specific calculation process of the verification calculation is as follows: read the lower limit value and upper limit value corresponding to the truncation tolerance range; perform the first-level judgment, comparing whether the currently acquired judgment difference data is greater than and equal to the lower limit value; simultaneously perform the second-level judgment, comparing whether the judgment difference data is less than and equal to the upper limit value; when both the first-level judgment and the second-level judgment output a true signal within the same processor clock cycle, the logic judgment passes, and the output threshold compliance status is marked as true.

[0082] Based on the obtained threshold compliance status, the underlying control driver performs communication message packet encapsulation calculation and outputs the feeding blocking command. The feeding blocking command is transmitted to the signal receiving end of the actuator through a physical communication cable, driving the actuator to cut off the material falling trajectory and complete the filling control cycle.

[0083] By multiplying the falling velocity with the inertia compensation coefficient to scale the output velocity to equivalent particles, and incorporating the suspended flying material into the joint accumulation framework, combined with the inclusion verification of the truncation tolerance range, the timing deviation caused by the physical continuous falling is eliminated from the decision center.

[0084] This invention provides an image analysis-based control method for metering and filling spherical materials. It receives initial control commands and outputs a set of task parameters, establishing a foundational configuration system for the entire process. A cross-field camera performs alternating polarization illumination acquisition of the spherical materials within the receiving container. Combined with the a priori equations of the spherical curvature, edge constraint completion calculations are performed. Subsequently, a cross-sectional statistical grid and depth extrapolation model are used to map the non-repeating contour coordinate set into a local surface density matrix, thereby extrapolating the total number of materials in three-dimensional space. Simultaneously, a lateral vision sensor extracts the real-time falling velocity and number of particles of the freely falling material. These dynamic and static multi-dimensional spatial variables are combined and input into a joint judgment equation for comparison, constructing a control network with feedforward prediction capabilities. The entire data flow process mitigates the temporal deviation between the predicted output from the sensing end and the continuous physical dynamics of the falling material.

[0085] Example 2 This embodiment describes the specific implementation process of an image analysis-based spherical material metering and filling control method applied to micro-spherical pellet metering and filling floating blocks.

[0086] In the initial stage of the control cycle, the communication interface receives batch production instructions and performs transformation operations on the initial control instructions, stripping the network layer header to generate an instruction feature stream. After extracting data blocks as the task parameter set, it is segmented to generate the main path parameter set, auxiliary path parameter set, and timing configuration parameters. The values ​​in the main path parameter set are read, multiplied by the physical equivalent coefficient, and the target loading quantity is output as 2000 grains, with the single grain reference area calculated to be 120 pixels. The container size information contained in the auxiliary path parameter set is converted into an image statistical grid.

[0087] The reference clock frequency is extracted and divided by a division factor of 2. The main trigger pulse is set to 60 times per second with a duty cycle of 10%. This pulse drives a ring light source with a polarizer to perform illumination, while a cross-field camera performs an exposure capture of 1 / 1000th of a second, outputting the original image stream data. Pixel grayscale values ​​are compared with a set brightness truncation threshold of 210; values ​​greater than this threshold are set to maximum grayscale, while values ​​less than or equal to it are set to 0, removing background highlights and outputting a specular highlight image. During the low-level phase, a grid projection array is activated to project sinusoidal wavy stripes, and the camera performs a secondary exposure, outputting secondary image stream data. The Euclidean distance from the pixel to the optical center is calculated, and the offset is calculated based on radial and tangential distortion coefficients for coordinate correction. Then, spatial convolution is performed using a high-pass filter operator to output a structured light texture image.

[0088] Connected-component segmentation is performed on the highlight image. The x and y coordinates of the effective pixels within the connected components are summed and divided by the total number to obtain the extremum centroid coordinates. These are then converted to the optical axis angle matrix to extract the physical sphere center reference coordinates. Contour separation is performed on the texture image, with the top 10% of gradient values ​​used for high thresholds and 1 / 3 for low thresholds. The sphere center coordinates and the incomplete contour matrix are input into the a priori equation of the sphere curvature. Virtual coordinate points are generated based on the standard radius parameter of 15 pixels and then merged to output a complete single-viewpoint contour. After mapping to the world coordinate system, non-maximum suppression is performed, with the exclusivity threshold set to 0.5 times the radius. Overlapping coordinates are removed, and a set of non-repeating contour coordinates is output. The target mesh address is located by dimensionality reduction. The depth extremum scalar is extracted by incrementing the count in the accumulator and reconstructed into a local areal density matrix with dual depth channels.

[0089] The pre-defined depth inference network is invoked. Tensor forward propagation passes through convolutional layers, flattens, and is then connected to hidden layers, outputting an algebraic descent parameter of 0.02. The matrix is ​​unpacked, and the effective fill thickness is output by subtracting the bottom reference elevation from the grid elevation. The effective thickness is divided by 1.632 times the radius and rounded down to output the number of depth cutting layers. The single-layer attenuation factor is set to 1 minus the product of the algebraic descent parameter and the number of layers. Layer-by-layer calculus is accumulated to the total variable, and the total global material count is output after summing all grids.

[0090] A sliding window array is constructed, and the image stream from the lateral vision sensor, scanned 2000 times per second, is pushed in sequentially to separate the current frame from the previous frame. The grayscale difference at the same coordinate is compared; pixels greater than the background threshold of 25 are marked as moving pixels. The absolute difference in the vertical coordinates of the centroids of the preceding and following frames is calculated, multiplied by the physical equivalent output spatial displacement scalar, and then divided by the time interval 1 / 2000 seconds to obtain the falling velocity. A 3x3 cross-shaped dot matrix is ​​used to separate connected components, and the output area of ​​non-zero pixels within each component is counted. This area is divided by a baseline area of ​​120 and rounded off, and the total number of particles in the suspended state is accumulated.

[0091] A multi-channel variable area is established, and the inertia compensation coefficient of 0.04 seconds, solidified through factory debugging, is extracted to obtain the velocity-equivalent particle prediction value. The total number of global materials at the bottom of the container, the number of particles currently falling, and the velocity-equivalent particle prediction value are continuously added together to output the predicted final inventory. The inventory is subtracted from the target filling quantity of 2000 to obtain the judgment difference data. The lower limit of the truncation tolerance range (-1) and the upper limit (+2) are read. When the difference falls within this range, the logic judgment passes, the underlying control driver outputs a feeding blocking command, and transmits it to the signal receiver to cut off the material falling trajectory.

[0092] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for metering and filling spherical materials based on image analysis, characterized in that, include: Receive initial control commands and output task parameter sets; through configuration parsing, output target loading quantity and image statistical grid. A cross-field camera is used to perform polarization-alternating illumination acquisition on the spherical material inside the receiving container, and outputs a specular highlight image and a structured light texture image. Edge constraint completion is performed on the specular highlight image and structured light texture image using the pre-set spherical curvature a priori equation, and a complete contour in a single field of view is output. Image registration and deduplication are performed on the complete contour in the single field of view, and a set of contour coordinates without repetition is output. The set of contour coordinates without repetition is input into the image statistical grid to perform grid operation, and a local surface density matrix is ​​output. The preset depth inference model is invoked to perform image integral projection on the local surface density matrix and output the total number of global materials; the free-falling materials in the feeding channel are detected by the side vision sensor and the falling speed and the number of particles in the path are output. The total number of materials, the falling speed in transit, the number of particles in transit, and the target filling quantity are compared with the cutoff threshold to calculate the feeding interruption command; the feeding interruption command is transmitted to the actuator to drive the actuator to cut off the material falling trajectory.

2. The method for metering and filling spherical materials based on image analysis according to claim 1, characterized in that, The specific generation process of the target loading quantity and image statistical grid includes: receiving the initial control command, performing a transformation operation on the initial control command to generate an instruction feature stream; performing a field truncation operation on the instruction feature stream to extract the task parameter set; performing a separation and decomposition operation on the task parameter set to segment and generate a main path parameter set, an auxiliary path parameter set, and timing configuration parameters; performing an addressing mapping operation on the main path parameter set to output the target loading quantity and single-particle reference area; performing a splitting and unpacking operation on the auxiliary path parameter set to separate the first feature set; and converting the first feature set into a planar grid structure as the image statistical grid.

3. The method for metering and filling spherical materials based on image analysis according to claim 2, characterized in that, The specific generation process of the specular highlight image and structured light texture image includes: extracting the timing configuration parameters encapsulated in the task parameter set; performing frequency division calculations using the timing configuration parameters to output a main trigger pulse; driving a polarized illumination source to perform an illumination action in conjunction with the main trigger pulse; driving the cross-field camera to perform an exposure capture action and output original image stream data when the illumination action is enabled; performing brightness truncation filtering on the original image stream data to remove background interference information and output a set of bright pixels as the specular highlight image; performing secondary frequency division calculations using the timing configuration parameters to output a secondary trigger pulse; enabling the grid projection array to perform a projection action in conjunction with the secondary trigger pulse; driving the cross-field camera to perform a secondary exposure capture action and output secondary image stream data when the projection action is enabled; performing distortion correction filtering on the secondary image stream data to enhance edge transition coordinates and output the structured light texture image.

4. The method for metering and filling spherical materials based on image analysis according to claim 1, characterized in that, The specific construction process of the single-view complete contour includes: performing connected component segmentation on the specular highlight image to extract the extreme centroid vector; extracting the optical axis angle matrix bound to the preset spherical curvature a priori equation; performing spatial offset permutation calculation on the extreme centroid vector and the optical axis angle matrix to output the spherical center reference coordinates; performing contour separation operation on the structured light texture image to output the incomplete contour matrix; combining the spherical center reference coordinates and the incomplete contour matrix into a joint parameter set and inputting it into the spherical curvature a priori equation; extracting the standard radius parameter contained in the spherical curvature a priori equation and performing curve fitting operation on the spherical center reference coordinates to output the virtual completion set; and performing matrix merging operation on the incomplete contour matrix and the virtual completion set to output the single-view complete contour.

5. The method for metering and filling spherical materials based on image analysis according to claim 1, characterized in that, The specific generation process of the local surface density matrix includes: extracting the center anchor point parameters within all received single-view complete contours; performing spatial projection operations based on the center anchor point parameters to output a panoramic contour set; performing distance comparison on the panoramic contour set, removing overlapping contours, and outputting the non-repeating contour coordinate set; performing coordinate dimensionality reduction operations on the non-repeating contour coordinate set and importing it into the two-dimensional isolated space defined by the image statistical grid; extracting the spatial scalar data within the non-repeating contour coordinate set and performing boundary constraint condition comparison operations to locate and output the target grid address; performing incremental iterative operations on all acquired target grid addresses to output density count parameters; extracting the depth extremum scalar within the non-repeating contour coordinate set as the corresponding grid elevation parameter; and performing spatial topology reconstruction operations on the density count parameter and the grid elevation parameter to output the local surface density matrix.

6. The method for metering and filling spherical materials based on image analysis according to claim 1, characterized in that, The specific process for generating the total global material count includes: extracting the bottom reference elevation nested within a preset depth projection model; performing dimensionality reduction and unpacking on the matrix elements within the local surface density matrix to output the surface elevation matrix; performing a subtraction comparison operation between the surface elevation matrix and the bottom reference elevation to output the effective filling thickness; inputting the local surface density matrix into the depth projection model for forward propagation to output an algebraic descent parameter; performing a division operation by dividing the effective filling thickness by a preset slice reference thickness to output the number of depth cutting layers; performing calculus accumulation on the local surface density matrix within the layered space defined by the number of depth cutting layers according to the algebraic descent parameter to output the particle packing algebraic value; and performing an algebraic aggregation operation on all the particle packing algebraic values ​​to output the total global material count.

7. The method for metering and filling spherical materials based on image analysis according to claim 6, characterized in that, The specific generation process of the in-transit falling speed and the number of in-transit particles includes: constructing a sliding window array to drive the lateral vision sensor to perform scanning actions on the falling material and output a continuous image stream; pushing the continuous image stream into the sliding window array in time sequence to perform queuing operations, separating the current extraction frame and the previous extraction frame; performing pixel difference comparison on the current extraction frame and the previous extraction frame to output an isolated pixel set; extracting the centroid of the current frame and the centroid of the previous frame in each isolated pixel set and performing spatial differentiation to obtain a spatial displacement scalar; combining the spatial displacement scalar with a preset fixed time interval to output the in-transit falling speed; performing morphological separation on all the obtained isolated pixel sets to extract the target connected components of independent closed outer loops; performing a pixel area statistical algorithm on the target connected components to output the connected component area; dividing the connected component area by the single particle reference area to output the equivalent particle multiple; performing a traversal accumulation calculation operation on all the equivalent particle multiples to output the number of in-transit particles in the suspended state.

8. The method for metering and filling spherical materials based on image analysis according to claim 1, characterized in that, The specific generation process of the feeding interruption command includes: establishing a multi-channel variable area and importing the total global material count, the in-transit falling speed, the number of in-transit particles, and the target filling quantity into each of the multi-channel variable areas; extracting the inertia compensation coefficient mapped within the preset joint judgment equation; performing a multiplication scaling operation on the in-transit falling speed and the inertia compensation coefficient to output speed-equivalent particles; performing a joint accumulation operation on the number of in-transit particles, the speed-equivalent particles, and the total global material count to output the predicted final inventory; performing a difference algebra judgment operation on the predicted final inventory and the target filling quantity to obtain judgment difference data; performing a verification operation on the judgment difference data and the truncation tolerance range set in the joint judgment equation, and outputting a threshold compliance status when the judgment difference data falls within the truncation tolerance range; performing a communication message packet operation based on the obtained threshold compliance status to output the feeding interruption command; and transmitting the feeding interruption command to the actuator to drive the actuator to cut off the material falling trajectory.

Images