A method, system and device for detecting the state of a tobacco magazine

Abstract

The application discloses a kind of tobacco storage cabinet state detection method, system and device, the method comprises: at least one cabinet image corresponding to the target tobacco storage cabinet is collected;Based on cabinet image, utilize the neural radiation field modeling tool of pre-set to construct the empty cabinet environment model corresponding to the target tobacco storage cabinet;Based on the tobacco leaf parameter corresponding to the cabinet image, utilize the discrete element model of pre-training to carry out tobacco simulation, to generate the dynamic mask image corresponding to the target tobacco storage cabinet in the empty cabinet environment model;State feature detection is carried out to the dynamic mask image, obtain the characteristic parameter corresponding to the target tobacco storage cabinet, and based on the characteristic parameter, determine the material state of the target tobacco storage cabinet.Through the technical means of fusion physical simulation and visual generation, the high-precision identification and error-proof control of tobacco storage cabinet state across equipment environment are realized, and the accuracy, stability and efficiency of detecting the material state of tobacco storage cabinet are improved.

Description

Technical Field

[0001] This invention relates to the field of tobacco storage cabinet testing technology, and in particular to a method, system and apparatus for detecting the condition of tobacco storage cabinets. Background Technology

[0002] Currently, when detecting the material condition of tobacco storage cabinets, the general method is to segment and detect the collected images of the tobacco storage cabinets to determine the material condition. However, due to the randomness of the tobacco leaf stacking pattern in the tobacco storage cabinets, as well as factors such as changes in lighting and background interference, the accuracy of detecting the material condition of tobacco storage cabinets is reduced.

[0003] In addition, the feeding rate of tobacco storage cabinets also has a certain impact on the stacking pattern of tobacco leaves, which can easily lead to the misjudgment of wrinkled canvas as residual tobacco leaves. The segmentation detection model requires a large amount of labeled data during training, but the different storage cabinet models and camera positions in industrial scenarios result in large differences in data distribution, which makes the model's generalization ability poor and reduces the stability and efficiency of detecting the material state of tobacco storage cabinets. Summary of the Invention

[0004] The purpose of this invention is to address the shortcomings of existing technologies by proposing a method, system, and device for detecting the status of tobacco storage cabinets. This involves constructing an empty cabinet environment model based on an image of the corresponding cabinet, and using a discrete element method (DEM) to simulate tobacco leaves. This generates a dynamic mask image of the tobacco storage cabinet within the empty cabinet environment model. The feature parameters of the dynamic mask image are then detected to determine the material status of the tobacco storage cabinet. By integrating physical simulation and visual generation techniques, this invention achieves high-precision identification and error-proof control of the tobacco storage cabinet status across different equipment environments, improving the accuracy, stability, and efficiency of material status detection in tobacco storage cabinets.

[0005] The objective of this invention is achieved through the following technical solution: a method for detecting the status of a tobacco storage cabinet, the detection method comprising:

[0006] In response to receiving an instruction to perform status detection on the target tobacco storage cabinet, at least one cabinet image corresponding to the target tobacco storage cabinet is acquired by the image acquisition device;

[0007] Based on the cabinet image, a model of the empty cabinet environment corresponding to the target tobacco storage cabinet is constructed using a preset neural radiation field modeling tool.

[0008] Based on the tobacco leaf parameters corresponding to the cabinet image, a pre-trained discrete element model is used to simulate the tobacco leaf, and a dynamic mask image corresponding to the target tobacco storage cabinet is generated in the empty cabinet environment model.

[0009] State feature detection is performed on the dynamic mask image to obtain the feature parameters corresponding to the target tobacco storage cabinet, and the material state of the target tobacco storage cabinet is determined based on the feature parameters.

[0010] Further, the step of responding to receiving an instruction to perform status detection on the target tobacco storage cabinet and acquiring at least one cabinet image corresponding to the target tobacco storage cabinet captured by the image acquisition device includes:

[0011] In response to receiving an instruction to perform status detection on the target tobacco storage cabinet, the structural status parameters of the target tobacco storage cabinet are determined;

[0012] Based on the aforementioned structural state parameters, a pre-defined cross-domain adaptive inference network model is used to match the target tobacco storage cabinet. The network structure of the cross-domain adaptive inference network model includes at least: a backbone network that outputs global features; a feature decoupling layer including model-related branches and common-specific branches, used to output model-related features and common-specific features; and a cross-domain adapter head.

[0013] In response to the successful matching of the target tobacco storage cabinet, at least one cabinet image corresponding to the target tobacco storage cabinet is acquired by the image acquisition device.

[0014] Furthermore, the step of constructing an empty cabinet environment model corresponding to the target tobacco storage cabinet based on the cabinet image using a preset neural radiation field modeling tool includes:

[0015] The cabinet image is input into a preset neural radiation field modeling tool to obtain the density field and radiation field corresponding to the target tobacco storage cabinet output by the neural radiation field modeling tool, so as to determine the empty cabinet environment model to be processed corresponding to the target tobacco storage cabinet.

[0016] Based on the pose parameters of the image acquisition device, the static background in the empty cabinet environment model to be processed is removed to obtain the empty cabinet environment model corresponding to the target tobacco storage cabinet. Based on the absolute difference image between the current frame of the empty cabinet environment model to be processed and the rendered background, a threshold is taken for binarization processing to obtain a binarized mask image. The binarized mask image is multiplied pixel by pixel with the empty cabinet environment model to be processed to remove the static background.

[0017] Further, the step of simulating tobacco leaves using a pre-trained discrete element model based on the tobacco leaf parameters corresponding to the cabinet image, in order to generate a dynamic mask image corresponding to the target tobacco storage cabinet in the empty cabinet environment model, includes:

[0018] The tobacco leaf parameters corresponding to the cabinet image are determined, and the tobacco leaf parameters are input into a pre-trained discrete element model; the discrete element model includes a top-down particle definition layer, a contact retrieval layer, a force-displacement calculation layer, and an explicit time integration layer;

[0019] The particle definition layer is used to define the properties of tobacco leaves and output the centroid position and geometric shape of the particles.

[0020] The contact retrieval layer performs retrieval based on the centroid position and geometry of each particle, using a uniform grid and linked list retrieval method, and outputs a list of all potential contact pair IDs at the current time step.

[0021] The force-displacement calculation layer is used to calculate the contact force between particles or between particles and boundaries in the contact pair ID list, and the bonding force is superimposed when the preset moisture content is reached.

[0022] The explicit time integration layer updates the latest position after each physical time step based on the resultant force on the particle and its mass, periodically converts the particle position into a Gaussian ellipse, and outputs a rasterized rendering map.

[0023] Based on the tobacco leaf parameters, the discrete element model transforms the tobacco leaves in the target tobacco storage cabinet into non-spherical particles, and maps the non-spherical particles to Gaussian ellipsoids in the empty cabinet environment model to obtain the rendered image corresponding to the target tobacco storage cabinet.

[0024] The rendered image is aligned and registered with the cabinet image to obtain the cabinet model corresponding to the target tobacco storage cabinet;

[0025] Based on a preset height threshold, visual inspection and height threshold constraints are performed on the cabinet model to obtain a dynamic mask image corresponding to the target tobacco storage cabinet output by the discrete element model, and the dynamic mask image is generated in the empty cabinet environment model.

[0026] Furthermore, the discrete element model is pre-trained through the following steps:

[0027] Based on an empty cabinet image and an empty cabinet model corresponding to each of the multiple tobacco storage cabinets, multiple training images corresponding to each tobacco storage cabinet are generated.

[0028] The training images are used to iteratively train the discrete element model to be trained, so as to optimize the physical parameters and rendering parameters in the discrete element model to be trained. In each iteration training cycle, it is detected whether the simulation performance parameters of the optimized discrete element model to be trained are greater than a preset performance threshold. The simulation performance parameters are obtained by weighting the fill rate error, structural similarity and optical flow consistency.

[0029] When the simulation performance parameters are greater than the preset performance threshold, the discrete element model to be trained in the current iteration training cycle is determined as the discrete element model for tobacco leaf simulation.

[0030] Furthermore, based on an empty cabinet image and an empty cabinet model corresponding to each of the multiple tobacco storage cabinets, multiple training images are generated for each tobacco storage cabinet, including:

[0031] Based on an empty cabinet image and an empty cabinet model corresponding to each of the multiple tobacco storage cabinets, view images of each tobacco storage cabinet at multiple preset angles are generated.

[0032] The pre-simulated heat map of tobacco leaf stacking height is superimposed onto the view image corresponding to each preset angle to obtain the morphological view image corresponding to each view image;

[0033] Each of the aforementioned morphological viewpoint images is temporally extended to obtain a feeding image group corresponding to each morphological viewpoint image; key status tags are injected into the feeding images corresponding to preset time points in the feeding image group to ensure that the morphological gradation of the feeding images conforms to the actual stacking rules.

[0034] Each of the feeding image groups is randomly processed to obtain multiple training images corresponding to each tobacco storage cabinet.

[0035] Furthermore, the feature parameters include at least fill rate, texture complexity, optical flow direction, and contour smoothness;

[0036] Determining the material state of the target tobacco storage cabinet based on the characteristic parameters includes:

[0037] Determine whether the fill rate is less than a first fill threshold and whether the texture complexity is less than a preset complexity threshold;

[0038] When the fill rate is less than the first fill threshold and the texture complexity is less than the preset complexity threshold, the material status of the target tobacco storage cabinet is determined to be an empty cabinet.

[0039] When the fill rate is greater than or equal to the first fill threshold and / or the texture complexity is greater than or equal to the preset complexity threshold, it is determined whether the optical flow field direction matches the fill rate in terms of temporal variation;

[0040] When the direction of the optical flow field matches the filling rate in a time-series change, the material state of the target tobacco storage cabinet is determined to be in the feeding / discharging state.

[0041] When the optical flow field direction and the fill rate do not match in temporal variation, it is determined whether the fill rate is greater than or equal to the second fill threshold and whether the contour smoothness is greater than or equal to the preset smoothness threshold.

[0042] When the filling rate is greater than or equal to the second filling threshold and the contour smoothness is greater than or equal to the preset smoothness threshold, the material state of the target tobacco storage cabinet is determined to be full.

[0043] Another aspect of the present invention provides a tobacco storage cabinet status detection system, comprising:

[0044] The image acquisition module is used to acquire at least one cabinet image corresponding to the target tobacco storage cabinet in response to receiving an instruction to perform status detection on the target tobacco storage cabinet;

[0045] The model building module is used to construct an empty cabinet environment model corresponding to the target tobacco storage cabinet based on the cabinet image and using a preset neural radiation field modeling tool.

[0046] The discrete simulation module is used to simulate tobacco leaves based on the tobacco leaf parameters corresponding to the cabinet image, using a pre-trained discrete element model, so as to generate a dynamic mask image corresponding to the target tobacco storage cabinet in the empty cabinet environment model.

[0047] The state detection module is used to perform state feature detection on the dynamic mask image, obtain the feature parameters corresponding to the target tobacco storage cabinet, and determine the material state of the target tobacco storage cabinet based on the feature parameters.

[0048] This application embodiment also provides a tobacco storage cabinet status detection device, including: a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory via the bus. When the machine-readable instructions are executed by the processor, the steps of the tobacco storage cabinet status detection method described above are performed.

[0049] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of the tobacco storage cabinet status detection method described above.

[0050] The present application provides a method and apparatus for detecting the state of a tobacco storage cabinet. The detection method includes: in response to receiving an instruction to detect the state of a target tobacco storage cabinet, acquiring at least one cabinet image corresponding to the target tobacco storage cabinet acquired by an image acquisition device; constructing an empty cabinet environment model corresponding to the target tobacco storage cabinet based on the cabinet image using a preset neural radiation field modeling tool; performing tobacco leaf simulation using a pre-trained discrete element model based on the tobacco leaf parameters corresponding to the cabinet image to generate a dynamic mask image corresponding to the target tobacco storage cabinet in the empty cabinet environment model; performing state feature detection on the dynamic mask image to obtain feature parameters corresponding to the target tobacco storage cabinet, and determining the material state of the target tobacco storage cabinet based on the feature parameters.

[0051] The beneficial effects of this invention are:

[0052] Compared to existing technologies that determine the material state of tobacco storage cabinets by segmenting and detecting images of the collected cabinets, this method constructs an empty cabinet environment model based on the cabinet image and uses a discrete element model to simulate tobacco leaves, generating a dynamic mask image of the tobacco storage cabinet within the empty cabinet environment model. The feature parameters of the dynamic mask image are then detected to determine the material state of the tobacco storage cabinet. By integrating physical simulation and visual generation techniques, this method achieves high-precision identification and error-proof control of the tobacco storage cabinet state across different equipment environments, improving the accuracy, stability, and efficiency of material state detection in tobacco storage cabinets. Attached Figure Description

[0053] Figure 1 A flowchart illustrating a method for detecting the status of a tobacco storage cabinet provided in an embodiment of this application;

[0054] Figure 2 This is a schematic diagram of the structure of a tobacco storage cabinet status detection system provided in an embodiment of this application;

[0055] Figure 3 This is a schematic diagram of the structure of a tobacco storage cabinet status detection device provided in an embodiment of this application. Detailed Implementation

[0056] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of this application. Based on the embodiments of this application, every other embodiment obtained by those skilled in the art without inventive effort falls within the scope of protection of this application.

[0057] Research has found that current methods for detecting the material condition of tobacco storage cabinets generally involve segmenting and detecting images of the cabinets. However, the accuracy of this method is reduced by factors such as the randomness of the tobacco leaf stacking pattern (e.g., loose or compacted), changes in lighting (e.g., a mixture of overhead lights and natural light), and background interference (e.g., conveyor belt movement).

[0058] Furthermore, since the material state detection of tobacco storage cabinets cannot be modeled with tobacco leaf filling dynamics, and the feeding rate of tobacco storage cabinets also has a certain impact on the stacking morphology of tobacco leaves, it is easy to misjudge wrinkled canvas as residual tobacco leaves. Moreover, the segmentation detection model requires a large amount of labeled data during training, but the different storage cabinet models and camera positions in industrial scenarios lead to large differences in data distribution, resulting in poor generalization ability of the model and reducing the stability and efficiency of detecting the material state of tobacco storage cabinets.

[0059] Based on this, this application provides a method for detecting the status of tobacco storage cabinets. By constructing an empty cabinet environment model based on the cabinet image, and using a discrete element method (DEM) to simulate tobacco leaves, a dynamic mask image of the tobacco storage cabinet is generated in the empty cabinet environment model. The feature parameters of the dynamic mask image are detected to determine the material status of the tobacco storage cabinet. By integrating physical simulation and visual generation techniques, high-precision identification and error prevention control of the tobacco storage cabinet status across different equipment environments are achieved, improving the accuracy of detecting the material status of tobacco storage cabinets, and thus enhancing the stability and efficiency of the detection method.

[0060] Please see Figure 1 , Figure 1 This is a flowchart illustrating a method for detecting the status of a tobacco storage cabinet, as provided in an embodiment of this application. Figure 1 As shown in the embodiments of this application, the method for detecting the status of a tobacco storage cabinet includes:

[0061] S101. In response to receiving an instruction to perform status detection on the target tobacco storage cabinet, acquire at least one cabinet image corresponding to the target tobacco storage cabinet acquired by the image acquisition device.

[0062] It should be noted that a tobacco storage cabinet is a specialized piece of equipment used to store tobacco raw materials or finished products. It is typically equipped with automated conveying, level monitoring, and control systems to ensure the stability, safety, and efficiency of tobacco materials during storage.

[0063] In this embodiment of the application, the target tobacco storage cabinet is the tobacco storage cabinet that the user selects from multiple tobacco storage cabinets based on detection needs and is expected to have its status detected.

[0064] In one embodiment of this application, step S101 may include:

[0065] S1011. In response to receiving an instruction to perform state detection on the target tobacco storage cabinet, determine the structural state parameters of the target tobacco storage cabinet.

[0066] In this step, upon receiving external input indicating that the target tobacco storage cabinet is to be monitored for its status, the structural state parameters of the target tobacco storage cabinet are decoupled based on the instruction information, and the structural state parameters of the target tobacco storage cabinet are determined.

[0067] Specifically, the structural state parameters of the target tobacco storage cabinet are extracted using a pre-defined feature decoupling network model. The backbone network of this feature decoupling network model may include a composite scaling network model (EfficientNet), which can output multi-dimensional feature vectors. The structural state parameters of the target tobacco storage cabinet are determined by separating the feature decoupling layers of the feature decoupling network model.

[0068] Here, the structural state parameters include, but are not limited to, cabinet model-related features (e.g., model and structural dimensions) and state common features (e.g., preset fill rate).

[0069] S1012. Based on the structural state parameters, the target tobacco storage cabinet is matched using a preset cross-domain adaptive inference network model.

[0070] In the embodiments of this application, the preset cross-domain adaptive inference network model can achieve rapid adaptation across different storage cabinet models through feature decoupling and meta-learning.

[0071] Here, the network structure of the cross-domain adaptive inference network model includes at least a backbone network, a feature decoupling layer, and a cross-domain adapter head.

[0072] The backbone network can include a lightweight EfficientNet-B4 network architecture model, which can output 512-dimensional global features. The model branch of the feature decoupling layer can output model-related features including 256-dimensional specialized encoded structural dimensions and slot shapes, while the common branch of the feature decoupling layer can output state-related common features including 256-dimensional specialized encoded fill rate, texture, and illumination. The cross-domain adaptation head is followed by a gradient inversion layer (GRL) and dual discriminators (model discriminator and view discriminator) after the common branch of the feature decoupling layer. In this step, the cross-domain adaptive inference network model adjusts the matching domain parameters based on the structural state parameters of the target tobacco storage cabinet and utilizes the common features learned between tobacco storage cabinets during training to match the target tobacco storage cabinet.

[0073] S1013. In response to the successful matching of the target tobacco storage cabinet, at least one cabinet image corresponding to the target tobacco storage cabinet is acquired by the image acquisition device.

[0074] In this step, when the target tobacco storage cabinet is successfully matched, at least one cabinet image corresponding to multiple shooting angles is captured using the image acquisition device, and all cabinet images are obtained.

[0075] In this way, through feature decoupling and meta-learning, rapid adaptation across different tobacco storage cabinet models is achieved, improving the efficiency and accuracy of matching target tobacco storage cabinets. This meets the industry's requirements for rapid deployment and high accuracy, and enhances the practicality and flexibility of tobacco storage cabinet testing.

[0076] S102. Based on the cabinet image, construct an empty cabinet environment model corresponding to the target tobacco storage cabinet using a preset neural radiation field modeling tool.

[0077] In the embodiments of this application, Neural Radiance Fields (NeRF) is a 3D scene reconstruction technology based on deep learning. It is a tool that implicitly models geometric and lighting information in 3D space through multi-view 2D images and camera parameters, thereby generating high-quality new perspective rendering images.

[0078] In one embodiment of this application, step S102 may include:

[0079] S1021. Input the cabinet image into a preset neural radiation field modeling tool to obtain the density field and radiation field corresponding to the target tobacco storage cabinet output by the neural radiation field modeling tool, so as to determine the empty cabinet environment model to be processed corresponding to the target tobacco storage cabinet.

[0080] In this step, the cabinet image is input into the neural radiation field modeling tool. The neural radiation field modeling tool will locate the pixel coordinates of the target tobacco storage cabinet based on the cabinet image. Through scene reconstruction and rendering, it will output the density field and radiation field corresponding to the target tobacco storage cabinet. Then, based on the density field and radiation field, it will determine the empty cabinet environment model to be processed corresponding to the target tobacco storage cabinet.

[0081] S1022. Based on the pose parameters of the image acquisition device, the static background in the empty cabinet environment model to be processed is removed to obtain the empty cabinet environment model corresponding to the target tobacco storage cabinet.

[0082] In this step, the empty cabinet environment model to be processed is rendered based on the pose parameters of the image acquisition device, and the static background in the empty cabinet environment model to be processed is removed based on the calculated absolute difference between the current frame of the empty cabinet environment model to be processed and the rendered background, so as to obtain the empty cabinet environment model corresponding to the target tobacco storage cabinet.

[0083] Specifically, the current frame of the empty cabinet environment model to be processed is denoted as I(x, y), and the empty cabinet environment model to be processed is rendered according to the pose parameters (R, t) of the image acquisition device to obtain a targetless pure background image B(x, y). Then, the absolute difference image D(x, y) between the current frame and the rendered background is calculated by the following formula.

[0084] .

[0085] in, For the current frame; A pure background image obtained by rendering the background based on the pose parameters of the image acquisition device; This is the absolute difference image between the current frame and the rendered background. and All images are RGB three-channel images, and absolute difference can be performed pixel by pixel and channel by channel.

[0086] Furthermore, the absolute difference image between the current frame and the rendered background is thresholded by a threshold τ and binarized to obtain a binary mask image M(x, y). The representation of the binary mask image M(x, y) is shown below.

[0087] .

[0088] The threshold τ is typically set to 15 to 25 (i.e., the gray level range of 0–255 for an 8-bit image).

[0089] Furthermore, by multiplying the binary mask image pixel by pixel with the empty cabinet environment model to be processed, the static background can be removed, leaving only the target tobacco storage cabinet area, thus obtaining the final required empty cabinet environment model.

[0090] S103. Based on the tobacco leaf parameters corresponding to the cabinet image, a pre-trained discrete element model is used to simulate the tobacco leaf, so as to generate a dynamic mask image corresponding to the target tobacco storage cabinet in the empty cabinet environment model.

[0091] In the embodiments of this application, the pre-trained discrete element model uses the discrete element method (DEM) to simulate tobacco leaf dynamics and combines differentiable rendering to perform real observation of the aligned cabinet image. When training the discrete element model, a dual-loop optimization is used to improve the structural similarity between the rendered image and the real observation. This can effectively deal with problems such as the randomness of tobacco leaf stacking morphology, illumination changes and background interference, and improve the accuracy and reliability of detection.

[0092] In the discrete element model, each particle is treated as an independent object with specific mass, shape, size, and physical properties (e.g., elastic modulus and Poisson's ratio). The model defines how the contact forces between particles or between particles and boundaries are calculated. This typically involves normal and tangential forces, as well as possible adhesive or liquid bridging forces. Because the forces between particles change rapidly, very small time steps are required to accurately capture these fast dynamic processes.

[0093] Here, the calculation rules for the contact forces between particles or between particles and the boundary defined in the discrete element model may include: normal calculation, where when two particles (or a particle and a wall) overlap, an elastic repulsive force proportional to the amount of overlap is generated, along with a small damping force opposite to the direction of the relative velocity; tangential calculation, where at the same contact point, if there is a tendency for relative sliding, a frictional force proportional to the normal force is generated, the magnitude of which does not exceed the upper limit of Coulomb friction; optional adhesion calculation, where if there are adhesion marks between particles, an additional adhesion force of a fixed magnitude is superimposed in the normal direction; the wall is considered rigid, the elastic modulus is taken as the value of steel, and the coefficient of friction is given separately.

[0094] In one embodiment of this application, step S103 may include:

[0095] S1031. Determine the tobacco leaf parameters corresponding to the cabinet image, and input the tobacco leaf parameters into the pre-trained discrete element model.

[0096] In the embodiments of this application, the tobacco leaf parameters may include, but are not limited to, tobacco leaf density value, tobacco leaf friction coefficient value, and tobacco leaf particle size parameters.

[0097] S1032. Based on the tobacco leaf parameters, the discrete element model transforms the tobacco leaves in the target tobacco storage cabinet into non-spherical particles, and maps the non-spherical particles to Gaussian ellipsoids in the empty cabinet environment model to obtain the rendered image corresponding to the target tobacco storage cabinet.

[0098] In this step, the discrete element model performs discrete element simulation on the tobacco leaves in the cabinet image based on the tobacco leaf parameters and the preset real-time moisture content parameters. That is, the tobacco leaves in the target tobacco storage cabinet are transformed into non-spherical particles, and the non-spherical particles are mapped to Gaussian ellipsoids in the empty cabinet environment model. The position and attributes of the Gaussian ellipsoids are dynamically updated to obtain the rendered image corresponding to the target tobacco storage cabinet.

[0099] Here, the discrete element model adopts a "dual-loop coupling" overall structure, which is divided into four functional layers from top to bottom. During the training phase, the entire model is optimized through "backpropagation of outer layer rendering error + stochastic gradient descent of inner layer physical parameters" to finally obtain a set of general physical parameter tables applicable to all target storage tanks. Therefore, there is no need to adjust the model parameters during runtime. The specific contents of the functional layers are as follows.

[0100] The particle definition layer contains objects representing a single tobacco leaf, i.e., a non-spherical discrete element. Its attributes include mass, centroid position, velocity, moisture content, coefficient of friction, coefficient of restitution, and whether it is agglomerated. The geometry of the particle definition layer can approximate the real leaf profile by combining at least 3 to 6 spherical sub-particles through fixed hinges to form a "multi-cluster." The output of the particle definition layer includes the centroid positions and geometric shapes of all particles (the combined shape of the multi-cluster).

[0101] The contact retrieval layer employs a retrieval algorithm based on a uniform grid and linked-cell structure, with a complexity of O(N). The contact retrieval layer updates every 0.5 ms by rebuilding the nearest neighbor list. The core of the contact retrieval algorithm (uniform grid method) is based on the current position and geometry of each particle. The contact retrieval layer can quickly determine which particles are spatially close enough to potentially make contact based on the centroid positions and geometry of all particles. The contact retrieval layer outputs a list of all potential contact pair IDs for the current time step. This list is entirely dependent on the geometric information provided by the particle layer (e.g., particle A and particle B, particle C and particle D, etc.). The force-displacement calculation layer includes a normal force calculation based on a linear spring-damping model (e.g., calculating stiffness and damping); a tangential force calculation based on a linear spring-slider model (e.g., calculating stiffness and upper limit of Coulomb friction); and an adhesive force calculation based on a constant normal additional force (enabled only when moisture content > 10%). The output of the force-displacement calculation layer is the resultant force (including the vector sum of all contact forces, adhesive forces, and external forces such as gravity) and resultant torque acting on each particle. The resultant force and torque output by the force-displacement calculation layer are applied to the integrator in the explicit time integration layer, so that the integrator calculates acceleration based on the force data, and then integrates and updates the velocity and position of the particles, outputting the latest position of all particles after each physical time step.

[0102] The force-displacement calculation layer does not blindly calculate all pairs of particles, but only calculates the interaction forces (e.g., normal force, tangential force) and the adhesion force if the conditions are met for each pair of particles in the "potential contact pair list" provided by the contact retrieval layer.

[0103] An explicit time integration layer is used, with the integrator employing Velocity-Verlet, whose step size can be automatically scaled by the system's critical step size. The external differentiable rendering interface of the explicit time integration layer is configured to convert particle positions into Gaussian ellipsoids every 0.1 s, and then output a rasterized rendering image for alignment with the actual captured image. In this embodiment, the preset real-time moisture content parameter is used only as a "bonding switch" and "bonding strength scalar" in the calculation, without directly changing the particle shape or mass.

[0104] The integrator (Velocity-Verlet) in the explicit time integration layer requires the resultant force calculated by the force-displacement calculation layer. Based on the resultant force acting on each particle and its mass, the integrator calculates the acceleration, integrates it, and updates the particle's velocity and position. The explicit time integration layer outputs the latest positions of all particles after each physical time step. This allows for detailed state data of all tobacco particles at each moment (the precise position, velocity, orientation, and forces acting on each particle at each moment. This data can be used to analyze the tobacco leaf's angle of accumulation, flow velocity, density distribution, etc.) and visually observable rendered animations (continuous images output through an external rendering interface can form realistic simulation animations, intuitively demonstrating the movement of tobacco leaves in the storage container).

[0105] For example, when the moisture content is greater than or equal to 10%, the system inserts an "adhesion record" when particles come into contact with each other, and adds an additional adhesion force of a fixed size to the normal force channel. The adhesion force is a single-point value, given by the calibration experiment, and is linearly mapped to the moisture content. When the moisture content is less than 10%, the adhesion record is deleted, and the contact pair retains only the ordinary elastic-friction force.

[0106] Based on this, the real-time moisture content parameter only serves as an on / off switch and a constant value throughout the entire simulation cycle, without being involved in the real-time updates of deformation, density, or friction coefficient. This simplifies the calculation process while meeting engineering accuracy requirements.

[0107] S1033. Align and register the rendered image with the cabinet image to obtain the cabinet model corresponding to the target tobacco storage cabinet.

[0108] In this step, the consistency between the physical parameters and visual representation of the target tobacco storage cabinet and tobacco leaves was optimized by aligning and registering the rendered image with the cabinet image.

[0109] S1034. Based on a preset height threshold, visual inspection and height threshold constraint are performed on the cabinet model to obtain a dynamic mask image corresponding to the target tobacco storage cabinet output by the discrete element model, and the dynamic mask image is generated in the empty cabinet environment model.

[0110] In this step, visual inspection is first performed on the cabinet model to obtain the boundary probability map of the cabinet model. Then, using a preset height threshold and combined with preset fusion rules, the boundary probability map is constrained to obtain the dynamic mask image corresponding to the target tobacco storage cabinet output by the discrete element model. Finally, the dynamic mask image is generated in the empty cabinet environment model.

[0111] Specifically, Canny edge detection is performed on the cabinet model to obtain the boundary probability map of the cabinet model. Max pooling thresholding is applied to the boundary probability map to obtain an edge mask to eliminate artifacts caused by lighting or texture and retain only clear outlines. A height mask is generated by setting two height thresholds (upper height threshold, for example, the rated full height of the cabinet - 10 mm; lower height threshold, for example, 15 mm). The edge mask and the height mask are fused using a preset fusion rule, that is, only pixel values ​​with both channels set to 1 are recognized as "real tobacco leaves", thus obtaining the dynamic mask image corresponding to the target tobacco storage cabinet output by the discrete element model.

[0112] For example, in a dynamic mask image, a pixel value of 0 (i.e., black) represents an empty area, background, artifacts, or leaf fragments smaller than 15 mm; a pixel value of 1 (white) represents an effective tobacco leaf stacking area.

[0113] Here, when rendering the image, the discrete element model only provides information such as the position, orientation, and filling height of the target tobacco storage cabinet in the cabinet model at the current moment through simulation. The rendering engine rasterizes the target tobacco storage cabinet particles to form a rendering image consistent with the actual shooting perspective. Then, the cabinet model corresponding to the target tobacco storage cabinet is aligned and registered at the pixel level through dynamic mask post-processing. Based on the preset height threshold, the registered image is binarized. The resulting binary image is the dynamic mask image, which is used for subsequent state feature detection.

[0114] The dynamic mask image is a single-channel binary image with the same resolution as the image acquisition device's screen. Here, a pixel value of "1" indicates that "there is tobacco leaf at this location (i.e., effective filling)," and a pixel value of "0" indicates that "there is no leaf (i.e., empty area or background)." In one embodiment of this application, the step of pre-training the discrete element model in step S103 may include:

[0115] S103A: Based on an empty cabinet image and an empty cabinet model corresponding to each of the multiple tobacco storage cabinets, generate multiple training images corresponding to each tobacco storage cabinet.

[0116] In this embodiment, multiple training images with multiple perspectives and multiple lighting are generated based on a single input empty cabinet image. Through multi-view generation, temporal extension and random processing techniques, the changes in various real-world scenarios are covered, reducing the need for real data annotation and solving to some extent the problems of strong data dependence and large differences in data distribution among different cabinet models in industrial scenarios.

[0117] In one embodiment of this application, step S103A may include:

[0118] S103A1. Based on an empty cabinet image and an empty cabinet model corresponding to each of the multiple tobacco storage cabinets, generate view images of each of the tobacco storage cabinets at multiple preset angles.

[0119] In this embodiment of the application, the empty cabinet model may include a CAD model; a preset control network tool is used to generate view images of each tobacco storage cabinet at multiple preset angles; wherein, the preset angles include at least horizontal ±60° and pitch angles from 0° to 30°.

[0120] S103A2. The pre-simulated heat map of tobacco stacking height is superimposed onto the view image corresponding to each preset angle to obtain the morphological view image corresponding to each view image.

[0121] In this embodiment of the application, a pre-simulated heat map of tobacco leaf stacking height is superimposed on the view image corresponding to each preset angle to constrain the morphological rationality of the morphological view image corresponding to each view image.

[0122] S103A3. Perform time-series expansion on each of the morphological view images to obtain the feed image group corresponding to each of the morphological view images.

[0123] In this step, in specific implementation, based on the first empty cabinet image of each morphological perspective image, a feeding image group corresponding to the feeding process video is generated using a preset video diffusion model, namely, empty cabinet, half full and full cabinet; then, key status labels are injected into the feeding images corresponding to preset time points in the feeding image group to ensure that the morphological changes of the feeding images conform to the actual stacking rules.

[0124] Here, the key state labels are the three discrete categories of "empty", "half full" and "full". Their function is to label the generated feed image groups during the training phase, so that the discrete element model knows which frame corresponds to which state, and thus learns to correctly map the simulation results to the real observation.

[0125] For example, key status labels can be defined as: Empty (fill rate < 5%); Half (5% ≤ fill rate < 85%); Full (fill rate ≥ 85%). Each label is stored using integers 0, 1, and 2, and is directly written to image metadata or filename suffixes, such as frame_0072_L2.png representing "full".

[0126] Furthermore, the timing of injecting key state labels into the feed images corresponding to preset time points in the feed image group is completed in the "temporal extension" step. The key state labels are used by adding a cross-entropy term to the loss function during training, so that the network model learns to see the image prediction labels, thereby supervising the optimization of physical parameters. During inference, the key state labels no longer appear and are only used for offline training. That is, in the online detection stage, the state is directly judged based on thresholds such as fill rate and contour smoothness, without the need to look up labels again.

[0127] Therefore, key state labels only serve as "supervision signals" during the training phase, and are not present during online detection.

[0128] S103A4. Randomly process each of the feeding image groups to obtain multiple training images corresponding to each of the tobacco storage cabinets.

[0129] In the embodiments of this application, random processing may include, but is not limited to, random perturbation of lighting (e.g., color temperature, direction, and shadow), random material generation (e.g., tobacco leaf reflectivity and cabinet wear texture), and the addition of interfering negative samples (e.g., tool droppings and clothing fragments).

[0130] S103B. The training images are used to iteratively train the discrete element model to be trained in order to optimize the physical parameters and rendering parameters in the discrete element model to be trained. In each iterative training cycle, it is detected whether the simulation performance parameters of the optimized discrete element model to be trained are greater than a preset performance threshold.

[0131] Here, simulation performance parameters are a comprehensive indicator used to measure the consistency between simulation and reality of the current discrete element model after one iteration, rather than just the listed physical parameters and rendering parameters. The simulation performance parameters are derived by weighting the fill rate error, structural similarity, and optical flow consistency.

[0132] Among them, fill rate error is the absolute value of the difference between the fill rate calculated from the simulation rendering and the fill rate of the real image; structural similarity is the structural similarity between the simulation rendering and the real image in terms of texture and contour; and optical flow consistency is the root mean square of the average angle difference between the simulation optical flow and the real optical flow in adjacent frames.

[0133] Thus, the simulation performance parameters are determined by the sum of the product of fill rate error and its corresponding weight (e.g., 0.4), the product of structural similarity and its corresponding weight (e.g., 0.4), and the product of optical flow consistency and its corresponding weight (e.g., 0.2). When the simulation performance parameters are greater than a preset performance threshold (e.g., 0.7), it is considered that "the simulation performance parameters meet the standard", training is stopped and the current physical / rendering parameters are fixed.

[0134] S103C. When the simulation performance parameter is greater than the preset performance threshold, the discrete element model to be trained in the current iteration training cycle is determined as the discrete element model for tobacco leaf simulation.

[0135] Here, when the simulation performance parameters are greater than the preset performance threshold (e.g., 0.7), it indicates that the simulation performance parameters of the discrete element model to be trained meet the standard. Training is stopped and the current physics / rendering parameters are fixed. The discrete element model to be trained in the current iteration training cycle is determined as the discrete element model for tobacco leaf simulation.

[0136] S104. Perform state feature detection on the dynamic mask image to obtain the feature parameters corresponding to the target tobacco storage cabinet, and determine the material state of the target tobacco storage cabinet based on the feature parameters.

[0137] In this step, state feature detection is performed on the dynamic mask image to detect the feature parameters corresponding to the target tobacco storage cabinet; then, based on the feature parameters, the material state of the target tobacco storage cabinet is determined.

[0138] The feature parameters include at least fill rate, texture complexity, optical flow direction, and contour smoothness.

[0139] Specifically, when detecting the fill rate, the proportion of non-zero pixels in the dynamic mask image is statistically analyzed, and the fill rate is calculated based on the proportion of non-zero pixels; when detecting texture complexity, rotation-invariant features are calculated in the dynamic mask image, and the standard deviation of the rotation-invariant features is taken as the complexity value; when detecting the optical flow direction, the principal direction angle is extracted by calculating the optical flow of adjacent frames to determine the optical flow direction; when detecting contour smoothness, the contours in the dynamic mask image are extracted, and the smoothness of the contours is calculated.

[0140] In one embodiment of this application, in specific implementation, the step of determining the material state of the target tobacco storage tank based on the characteristic parameters in step S104 may include:

[0141] S1041. Determine whether the fill rate is less than a first fill threshold and whether the texture complexity is less than a preset complexity threshold.

[0142] In one possible implementation of this application, the first filling threshold may be selected as 5%.

[0143] S1042. When the filling rate is less than the first filling threshold and the texture complexity is less than the preset complexity threshold, the material status of the target tobacco storage cabinet is determined to be an empty cabinet status.

[0144] S1043. When the fill rate is greater than or equal to the first fill threshold and / or the texture complexity is greater than or equal to the preset complexity threshold, determine whether the optical flow field direction matches the fill rate in terms of temporal variation.

[0145] S1044. When the direction of the optical flow field matches the filling rate in terms of temporal variation, the material state of the target tobacco storage cabinet is determined to be in the feeding / discharging state.

[0146] S1045. When the optical flow field direction and the fill rate do not match in terms of temporal variation, determine whether the fill rate is greater than or equal to the second fill threshold and whether the contour smoothness is greater than or equal to the preset smoothness threshold.

[0147] In one possible implementation of this application, the second filling threshold may be selected as 95%.

[0148] S1046. When the filling rate is greater than or equal to the second filling threshold and the contour smoothness is greater than or equal to the preset smoothness threshold, the material state of the target tobacco storage cabinet is determined to be full.

[0149] When the material status of the target tobacco storage tank is determined to be full, a control signal is output to the target tobacco storage tank to lock the feed gate of the target tobacco storage tank.

[0150] In response to the target tobacco storage cabinet triggering an audible and visual alarm, identify and record the abnormal event that triggered the audible and visual alarm.

[0151] Specifically, when the material status of the target tobacco storage cabinet is determined to be full, the output signal locks the feed gate, triggers an abnormal event with audible and visual alarms, and records the abnormal event.

[0152] The tobacco storage cabinet status detection method provided in this application constructs an empty cabinet environment model based on the cabinet image corresponding to the tobacco storage cabinet, and uses a discrete element model to simulate tobacco leaves to generate a dynamic mask image corresponding to the tobacco storage cabinet in the empty cabinet environment model. The feature parameters of the dynamic mask image are detected to determine the material status of the tobacco storage cabinet. By integrating physical simulation and visual generation techniques, high-precision identification and error prevention control of the tobacco storage cabinet status across equipment environments are achieved, improving the accuracy of material status detection in tobacco storage cabinets, and thus improving the stability and efficiency of material status detection in tobacco storage cabinets.

[0153] Please see Figure 2 , Figure 2 This is a schematic diagram of a tobacco storage cabinet status detection system provided in an embodiment of this application. Figure 2 As shown, the detection system 200 includes:

[0154] The image acquisition module 210 is used to acquire at least one cabinet image corresponding to the target tobacco storage cabinet, which is acquired by the image acquisition device, in response to receiving an instruction information for status detection of the target tobacco storage cabinet.

[0155] The model building module 220 is used to build an empty cabinet environment model corresponding to the target tobacco storage cabinet based on the cabinet image and using a preset neural radiation field modeling tool.

[0156] The discrete simulation module 230 is used to simulate tobacco leaves based on the tobacco leaf parameters corresponding to the cabinet image and using a pre-trained discrete element model to generate a dynamic mask image corresponding to the target tobacco storage cabinet in the empty cabinet environment model.

[0157] The state detection module 240 is used to perform state feature detection on the dynamic mask image, obtain the feature parameters corresponding to the target tobacco storage cabinet, and determine the material state of the target tobacco storage cabinet based on the feature parameters.

[0158] When the image acquisition module 210 acquires at least one cabinet image corresponding to the target tobacco storage cabinet in response to receiving an instruction to perform status detection on the target tobacco storage cabinet, the image acquisition module 210 is used to:

[0159] In response to receiving an instruction to perform status detection on the target tobacco storage cabinet, the structural status parameters of the target tobacco storage cabinet are determined;

[0160] Based on the structural state parameters, the target tobacco storage cabinet is matched using a preset cross-domain adaptive inference network model;

[0161] In response to the successful matching of the target tobacco storage cabinet, at least one cabinet image corresponding to the target tobacco storage cabinet is acquired by the image acquisition device.

[0162] When the model building module 220 is used to construct an empty cabinet environment model corresponding to the target tobacco storage cabinet based on the cabinet image and using a preset neural radiation field modeling tool, the model building module 220 is used to:

[0163] The cabinet image is input into a preset neural radiation field modeling tool to obtain the density field and radiation field corresponding to the target tobacco storage cabinet output by the neural radiation field modeling tool, so as to determine the empty cabinet environment model to be processed corresponding to the target tobacco storage cabinet.

[0164] Based on the pose parameters of the image acquisition device, the static background in the empty cabinet environment model to be processed is removed to obtain the empty cabinet environment model corresponding to the target tobacco storage cabinet.

[0165] When the discrete simulation module 230 is used to perform tobacco leaf simulation based on the tobacco leaf parameters corresponding to the cabinet image using a pre-trained discrete element model, in order to generate a dynamic mask image corresponding to the target tobacco storage cabinet in the empty cabinet environment model, the discrete simulation module 230 is used to:

[0166] Determine the tobacco leaf parameters corresponding to the cabinet image, and input the tobacco leaf parameters into a pre-trained discrete element model;

[0167] Based on the tobacco leaf parameters, the discrete element model transforms the tobacco leaves in the target tobacco storage cabinet into non-spherical particles, and maps the non-spherical particles to Gaussian ellipsoids in the empty cabinet environment model to obtain the rendered image corresponding to the target tobacco storage cabinet.

[0168] The rendered image is aligned and registered with the cabinet image to obtain the cabinet model corresponding to the target tobacco storage cabinet;

[0169] Based on a preset height threshold, visual inspection and height threshold constraints are performed on the cabinet model to obtain a dynamic mask image corresponding to the target tobacco storage cabinet output by the discrete element model, and the dynamic mask image is generated in the empty cabinet environment model.

[0170] When the discrete simulation module 230 is used to pre-train the discrete element model, the discrete simulation module 230 is used for:

[0171] Based on an empty cabinet image and an empty cabinet model corresponding to each of the multiple tobacco storage cabinets, multiple training images corresponding to each tobacco storage cabinet are generated.

[0172] The training images are used to iteratively train the discrete element model to be trained in order to optimize the physical parameters and rendering parameters in the discrete element model to be trained. In each iterative training cycle, it is detected whether the simulation performance parameters of the optimized discrete element model to be trained are greater than a preset performance threshold.

[0173] When the simulation performance parameters are greater than the preset performance threshold, the discrete element model to be trained in the current iteration training cycle is determined as the discrete element model for tobacco leaf simulation.

[0174] When the discrete simulation module 230 generates multiple training images corresponding to each tobacco storage cabinet based on an empty cabinet image and an empty cabinet model corresponding to each of the multiple tobacco storage cabinets, the discrete simulation module 230 is used to:

[0175] Based on an empty cabinet image and an empty cabinet model corresponding to each of the multiple tobacco storage cabinets, view images of each tobacco storage cabinet at multiple preset angles are generated.

[0176] The pre-simulated heat map of tobacco leaf stacking height is superimposed onto the view image corresponding to each preset angle to obtain the morphological view image corresponding to each view image;

[0177] Each of the aforementioned morphological viewpoint images is temporally extended to obtain a feed image group corresponding to each of the aforementioned morphological viewpoint images;

[0178] Each of the feeding image groups is randomly processed to obtain multiple training images corresponding to each tobacco storage cabinet.

[0179] The feature parameters include at least fill rate, texture complexity, optical flow direction, and contour smoothness;

[0180] When the state detection module 240 is used to determine the material state of the target tobacco storage cabinet based on the feature parameters, the state detection module 240 is used to:

[0181] Determine whether the fill rate is less than a first fill threshold and whether the texture complexity is less than a preset complexity threshold;

[0182] When the fill rate is less than the first fill threshold and the texture complexity is less than the preset complexity threshold, the material status of the target tobacco storage cabinet is determined to be an empty cabinet.

[0183] When the fill rate is greater than or equal to the first fill threshold and / or the texture complexity is greater than or equal to the preset complexity threshold, it is determined whether the optical flow field direction matches the fill rate in terms of temporal variation;

[0184] When the direction of the optical flow field matches the filling rate in a time-series change, the material state of the target tobacco storage cabinet is determined to be in the feeding / discharging state.

[0185] When the optical flow field direction and the fill rate do not match in temporal variation, it is determined whether the fill rate is greater than or equal to the second fill threshold and whether the contour smoothness is greater than or equal to the preset smoothness threshold.

[0186] When the filling rate is greater than or equal to the second filling threshold and the contour smoothness is greater than or equal to the preset smoothness threshold, the material state of the target tobacco storage cabinet is determined to be full.

[0187] The tobacco storage cabinet status detection device provided in this application constructs an empty cabinet environment model based on the cabinet image corresponding to the tobacco storage cabinet, and uses a discrete element model to simulate tobacco leaves to generate a dynamic mask image corresponding to the tobacco storage cabinet in the empty cabinet environment model. The feature parameters of the dynamic mask image are detected to determine the material status of the tobacco storage cabinet. By integrating physical simulation and visual generation techniques, high-precision identification and error prevention control of the tobacco storage cabinet status across equipment environments are achieved, improving the accuracy of detecting the material status of the tobacco storage cabinet, and thus improving the stability and efficiency of detecting the material status of the tobacco storage cabinet.

[0188] Corresponding to the aforementioned embodiment of a method for detecting the state of a tobacco storage cabinet, the present invention also provides an embodiment of a device for detecting the state of a tobacco storage cabinet.

[0189] See Figure 3 The present invention provides a tobacco storage cabinet status detection device, which includes a memory and one or more processors. The memory stores executable code, and when the processor executes the executable code, it is used to implement a tobacco storage cabinet status detection method in the above embodiment.

[0190] An embodiment of the tobacco storage cabinet status detection device provided by this invention can be applied to any device with data processing capabilities, such as a computer. The device embodiment can be implemented through software, hardware, or a combination of both. Taking software implementation as an example, as a logical device, it is formed by the processor of any data processing device reading the corresponding computer program instructions from non-volatile memory into memory and executing them. From a hardware perspective, such as... Figure 3 The diagram shown is a hardware structure diagram of any device with data processing capabilities, including the tobacco storage cabinet status detection device provided by this invention. (Except for...) Figure 3 In addition to the processor, memory, network interface, and non-volatile memory shown, any data processing device in the embodiment may also include other hardware depending on the actual function of the data processing device, which will not be described in detail here.

[0191] The specific implementation process of the functions and roles of each unit in the above device can be found in the implementation process of the corresponding steps in the above method, and will not be repeated here.

[0192] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of the present invention according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0193] This invention also provides a computer-readable storage medium storing a program thereon, which, when executed by a processor, implements a method for detecting the status of a tobacco storage cabinet as described in the above embodiments.

[0194] The computer-readable storage medium can be an internal storage unit of any data processing device described in any of the foregoing embodiments, such as a hard disk or memory. The computer-readable storage medium can also be an external storage device of any data processing device, such as a plug-in hard disk, smart media card (SMC), SD card, flash card, etc., equipped on the device. Furthermore, the computer-readable storage medium can include both internal storage units and external storage devices of any data processing device. The computer-readable storage medium is used to store the computer program and other programs and data required by the data processing device, and can also be used to temporarily store data that has been output or will be output.

[0195] The present invention also provides a computer program product, including a computer program, which, when executed by a processor, implements the method for detecting the status of a tobacco storage cabinet.

[0196] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the disclosure herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and embodiments are to be considered exemplary only, and the true scope and spirit of this application are indicated by the claims.

[0197] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this application. This application is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A method for detecting the condition of a tobacco storage cabinet, characterized in that, The detection method includes: In response to receiving an instruction to perform status detection on the target tobacco storage cabinet, at least one cabinet image corresponding to the target tobacco storage cabinet is acquired by the image acquisition device; Based on the cabinet image, a model of the empty cabinet environment corresponding to the target tobacco storage cabinet is constructed using a preset neural radiation field modeling tool. Based on the tobacco leaf parameters corresponding to the cabinet image, a pre-trained discrete element model is used to simulate the tobacco leaf, and a dynamic mask image corresponding to the target tobacco storage cabinet is generated in the empty cabinet environment model. The step of simulating tobacco leaves using a pre-trained discrete element model based on the tobacco leaf parameters corresponding to the cabinet image, to generate a dynamic mask image corresponding to the target tobacco storage cabinet in the empty cabinet environment model, includes: Determine the tobacco leaf parameters corresponding to the cabinet image, and input the tobacco leaf parameters into a pre-trained discrete element model; Based on the tobacco leaf parameters, the discrete element model transforms the tobacco leaves in the target tobacco storage cabinet into non-spherical particles, and maps the non-spherical particles to Gaussian ellipsoids in the empty cabinet environment model to obtain the rendered image corresponding to the target tobacco storage cabinet. The rendered image is aligned and registered with the cabinet image to obtain the cabinet model corresponding to the target tobacco storage cabinet; Based on a preset height threshold, visual detection and height threshold constraints are performed on the cabinet model to obtain a dynamic mask image corresponding to the target tobacco storage cabinet output by the discrete element model, and the dynamic mask image is generated in the empty cabinet environment model; The discrete element model includes a top-down particle definition layer, a contact retrieval layer, a force-displacement calculation layer, and an explicit time integration layer. The particle definition layer is used to define the properties of tobacco leaves and output the centroid position and geometric shape of the particles. The contact retrieval layer performs retrieval based on the centroid position and geometry of each particle, using a uniform grid and linked list retrieval method, and outputs a list of all potential contact pair IDs at the current time step. The force-displacement calculation layer is used to calculate the contact force between particles or between particles and boundaries in the contact pair ID list, and the bonding force is superimposed when the preset moisture content is reached. The explicit time integration layer updates the latest position after each physical time step based on the resultant force on the particle and its mass, periodically converts the particle position into a Gaussian ellipse, and outputs a rasterized rendering map. State feature detection is performed on the dynamic mask image to obtain the feature parameters corresponding to the target tobacco storage cabinet, and the material state of the target tobacco storage cabinet is determined based on the feature parameters.

2. The method according to claim 1, characterized in that, The step of responding to receiving an instruction to perform status detection on the target tobacco storage cabinet, and acquiring at least one cabinet image corresponding to the target tobacco storage cabinet captured by the image acquisition device, includes: In response to receiving an instruction to perform status detection on the target tobacco storage cabinet, the structural status parameters of the target tobacco storage cabinet are determined; Based on the structural state parameters, a preset cross-domain adaptive inference network model is used to match the target tobacco storage cabinet. The network structure of the cross-domain adaptive inference network model includes at least: a backbone network that outputs global features, a feature decoupling layer including model branches and common branches for outputting model-related features and common state features, and a cross-domain adapter head. In response to the successful matching of the target tobacco storage cabinet, at least one cabinet image corresponding to the target tobacco storage cabinet is acquired by the image acquisition device.

3. The method according to claim 1, characterized in that, The step of constructing an empty cabinet environment model corresponding to the target tobacco storage cabinet based on the cabinet image using a preset neural radiation field modeling tool includes: The cabinet image is input into a preset neural radiation field modeling tool to obtain the density field and radiation field corresponding to the target tobacco storage cabinet output by the neural radiation field modeling tool, so as to determine the empty cabinet environment model to be processed corresponding to the target tobacco storage cabinet. Based on the pose parameters of the image acquisition device, the static background in the empty cabinet environment model to be processed is removed to obtain the empty cabinet environment model corresponding to the target tobacco storage cabinet. Based on the absolute difference image between the current frame of the empty cabinet environment model to be processed and the rendered background, a threshold is taken for binarization processing to obtain a binarized mask image. The binarized mask image is multiplied pixel by pixel with the empty cabinet environment model to be processed to remove the static background.

4. The method according to claim 1, characterized in that, The discrete element model is pre-trained using the following steps: Based on an empty cabinet image and an empty cabinet model corresponding to each of the multiple tobacco storage cabinets, multiple training images corresponding to each tobacco storage cabinet are generated. The training images are used to iteratively train the discrete element model to be trained in order to optimize the physical parameters and rendering parameters in the discrete element model to be trained. In each iterative training cycle, it is detected whether the simulation performance parameters of the optimized discrete element model to be trained are greater than a preset performance threshold. The simulation performance parameters are obtained by weighting the fill rate error, structural similarity, and optical flow consistency. When the simulation performance parameters are greater than the preset performance threshold, the discrete element model to be trained in the current iteration training cycle is determined as the discrete element model for tobacco leaf simulation.

5. The method according to claim 4, characterized in that, The process involves generating multiple training images for each tobacco storage cabinet based on an empty cabinet image and an empty cabinet model corresponding to each of the multiple tobacco storage cabinets, including: Based on an empty cabinet image and an empty cabinet model corresponding to each of the multiple tobacco storage cabinets, view images of each tobacco storage cabinet at multiple preset angles are generated. The pre-simulated heat map of tobacco leaf stacking height is superimposed onto the view image corresponding to each preset angle to obtain the morphological view image corresponding to each view image; Each of the aforementioned morphological viewpoint images is temporally extended to obtain a feeding image group corresponding to each morphological viewpoint image; key status labels are injected into the feeding images corresponding to preset time points in the feeding image group to ensure that the morphological gradient of the feeding images conforms to the actual stacking law. Each of the feeding image groups is randomly processed to obtain multiple training images corresponding to each tobacco storage cabinet.

6. The method according to claim 1, characterized in that, The feature parameters include at least fill rate, texture complexity, optical flow direction, and contour smoothness; Determining the material state of the target tobacco storage cabinet based on the characteristic parameters includes: Determine whether the fill rate is less than a first fill threshold and whether the texture complexity is less than a preset complexity threshold; When the fill rate is less than the first fill threshold and the texture complexity is less than the preset complexity threshold, the material status of the target tobacco storage cabinet is determined to be an empty cabinet. When the fill rate is greater than or equal to the first fill threshold and / or the texture complexity is greater than or equal to the preset complexity threshold, it is determined whether the optical flow field direction matches the fill rate in terms of temporal variation; When the direction of the optical flow field matches the filling rate in a time-series change, the material state of the target tobacco storage cabinet is determined to be in the feeding / discharging state. When the optical flow field direction and the fill rate do not match in temporal variation, it is determined whether the fill rate is greater than or equal to the second fill threshold and whether the contour smoothness is greater than or equal to the preset smoothness threshold. When the filling rate is greater than or equal to the second filling threshold and the contour smoothness is greater than or equal to the preset smoothness threshold, the material state of the target tobacco storage cabinet is determined to be full.

7. A system for detecting the status of a tobacco storage cabinet using the method of claim 1, characterized in that, include: The image acquisition module is used to acquire at least one cabinet image corresponding to the target tobacco storage cabinet in response to receiving an instruction to perform status detection on the target tobacco storage cabinet; The model building module is used to construct an empty cabinet environment model corresponding to the target tobacco storage cabinet based on the cabinet image and using a preset neural radiation field modeling tool. The discrete simulation module is used to simulate tobacco leaves based on the tobacco leaf parameters corresponding to the cabinet image, using a pre-trained discrete element model, so as to generate a dynamic mask image corresponding to the target tobacco storage cabinet in the empty cabinet environment model. The state detection module is used to perform state feature detection on the dynamic mask image, obtain the feature parameters corresponding to the target tobacco storage cabinet, and determine the material state of the target tobacco storage cabinet based on the feature parameters.

8. A device for detecting the status of a tobacco storage cabinet, comprising a memory and one or more processors, wherein the memory stores executable code, characterized in that, When the processor executes the executable code, it implements a method for detecting the status of a tobacco storage cabinet as described in any one of claims 1-6.

Images