A liquor solid-state brewing pit unmanned aerial vehicle inspection method and system based on multi-source vision fusion

Abstract

This invention belongs to the field of intelligent monitoring technology in baijiu brewing. It discloses a method and system for drone inspection of baijiu solid-state brewing cellars based on multi-source visual fusion, solving the problems of existing automated image monitoring technologies for cellars being unable to quantify changes in the cellar cap morphology in three dimensions, having fragmented multi-source monitoring data, and lacking safety inspection capabilities adapted to the dynamic environment of the brewing workshop. In this invention, a drone patrol mission is generated based on the production schedule and cellar distribution. The image acquisition device on the drone acquires binocular or multi-view images of the cellar cap area and performs three-dimensional reconstruction. The height and morphological parameters of the cellar cap are calculated, and the height change is characterized by temporal difference to determine fermentation fluctuations and sealing risks. At the same time, multispectral and / or thermal imaging data are acquired and fused with visible light registration to obtain the surface temperature field of the cellar, identifying hot spots and cold spots. Furthermore, the morphological changes and temperature field characteristics are coupled and analyzed to output fermentation status assessment and alarms.

Description

Technical Field

[0001] This invention belongs to the field of intelligent monitoring technology for baijiu brewing, specifically involving a method and system for unmanned aerial vehicle (UAV) inspection of baijiu solid-state brewing cellars based on multi-source visual fusion. Background Technology

[0002] Solid-state fermentation is the core of traditional Chinese brewing techniques for baijiu (Chinese liquor). The fermentation pit, as the primary container for fermentation, directly impacts the quality and yield of the liquor. During fermentation, the morphological changes, temperature distribution, and sealing condition of the pit cap (the layer of mud covering the surface of the mash) are key indicators for judging the fermentation progress and product quality.

[0003] Traditional monitoring of fermentation pit conditions mainly relies on manual inspections, which has the following problems:

[0004] (1) Low inspection efficiency: Large brewing workshops usually have hundreds or even thousands of cellars. Manual inspection of each cellar is time-consuming and labor-intensive, making it difficult to achieve high-frequency and full-coverage monitoring, and it is easy to miss abnormal cellars.

[0005] (2) Large subjective judgment error: Manual observation of the shape and temperature of the cellar cap mainly relies on experience. Different inspectors have different judgment standards and lack quantitative evaluation indicators, making it difficult to accurately capture subtle shape changes and temperature anomalies.

[0006] (3) Poor data continuity: Manual inspections are usually spaced far apart, making it impossible to form continuous monitoring records. It is difficult to trace the dynamic changes in the fermentation process, which is not conducive to process optimization and problem tracing.

[0007] (4) High safety risks: The brewing workshop has a complex environment with mobile equipment such as mash conveyor, overhead crane, and grab bucket, and manual inspection poses certain safety hazards.

[0008] In recent years, with the development of computer vision and drone technology, image-based automated monitoring solutions have been gradually applied to the field of industrial inspection. However, existing technical solutions still have shortcomings when applied to the monitoring of liquor cellars:

[0009] On the one hand, existing image monitoring solutions mostly use a single visible light camera to acquire two-dimensional images, which cannot obtain three-dimensional morphological information of the cellar cap and make it difficult to accurately quantify changes in key parameters such as the height and volume of the cellar cap.

[0010] On the other hand, temperature monitoring and morphological monitoring are independent of each other, failing to achieve the fusion analysis of multi-source data, and unable to establish a correlation model between morphological changes and temperature distribution, thus limiting the ability to comprehensively assess the fermentation state.

[0011] In addition, existing indoor drone inspection systems are mostly designed for relatively static environments such as warehousing and power supply, and lack obstacle avoidance strategies for the dynamic operating environment of brewing workshops, making it difficult to cope with interference from mobile devices such as mash transport vehicles and overhead cranes.

[0012] Therefore, there is an urgent need for an intelligent inspection system that can realize three-dimensional morphological modeling of fermentation pits, multi-source data fusion analysis, and dynamic environmental safety obstacle avoidance, so as to improve the monitoring efficiency and management level of the solid-state fermentation process of baijiu. Summary of the Invention

[0013] The technical problem to be solved by this invention is to provide a method and system for drone inspection of solid-state brewing cellars for baijiu based on multi-source visual fusion, which solves the problems of existing automated image monitoring technology for cellars being unable to quantify changes in cellar cap shape in three dimensions, having fragmented multi-source monitoring data, and lacking safety inspection capabilities adapted to the dynamic environment of brewing workshops.

[0014] The technical solution adopted by the present invention to solve the above-mentioned technical problems is as follows:

[0015] On the one hand, this invention provides a method for unmanned aerial vehicle (UAV) inspection of solid-state fermentation cellars for baijiu (Chinese liquor) based on multi-source visual fusion, including the following steps:

[0016] S1. Generate patrol missions based on production scheduling information and pit spatial distribution information;

[0017] S2. The UAV autonomously cruises within the workshop according to the cruise mission. During the cruise, it acquires the pose and motion information of the workshop's mobile equipment in real time and performs trajectory prediction. When a collision risk is detected, it performs dynamic obstacle avoidance. After flying to the collection position above the target pit, it simultaneously acquires stereoscopic visual data (binocular or multi-view visible light image) of the pit cap area, multispectral image data of the pit surface, and thermal imaging image data.

[0018] S3. Perform three-dimensional reconstruction on the stereo vision data to obtain a three-dimensional model of the cellar cap, extract the morphological parameters of the cellar cap, and perform temporal difference between the current period three-dimensional model and the reference period three-dimensional model to obtain the morphological change characterization of the cellar cap;

[0019] S4. Register and fuse the visible light image with the thermal imaging image to generate the surface temperature field distribution of the pit, and identify and extract the temperature anomaly areas;

[0020] S5. Couple the characterization of changes in the pit cap morphology with the characteristics of temperature field anomalies, and output the fermentation status assessment results and risk warning information.

[0021] This solution upgrades fermentation pit inspection from manual, experience-based monitoring to automated, digital, and intelligent full-cycle monitoring through a complete closed-loop process involving task planning, autonomous navigation and obstacle avoidance, synchronous acquisition of multi-source data, three-dimensional morphological time-series analysis, temperature field fusion identification, and morphology-temperature coupling assessment. By integrating stereoscopic vision, multispectral imaging, and thermal imaging, it achieves synchronous, accurate, and quantitative monitoring of the pit cap morphology, surface condition, and temperature distribution. This constructs a correlation analysis system between morphological changes and temperature anomalies, comprehensively improving the objectivity, accuracy, and timeliness of fermentation pit state assessment. This provides reliable data support and decision-making basis for optimizing the solid-state brewing process and quality control of baijiu.

[0022] Furthermore, in step S1, the cruise mission includes at least: a target set of the reservoir, cruise time / cycle, data collection frequency, flight channel and no-fly / altitude restrictions; when it is a multi-UAV collaborative mode, the target set of the reservoir is divided into multiple sub-tasks according to spatial partitioning, batch grouping and / or task load and assigned to multiple UAVs, while conflict resolution is performed on the corresponding flight paths of each sub-task.

[0023] In this solution, by clearly defining the constraint parameters of the patrol mission, the compliance and controllability of the inspection sequence, scope and path are ensured, and missed inspections and illegal flights are avoided. In the multi-UAV mode, the mission is split by space, batch or load and the route conflict is resolved, the workload is balanced and the cross-road collision is avoided, so as to realize the parallel and efficient inspection of large-scale pits, and take into account both inspection efficiency and flight safety.

[0024] Furthermore, in step S2, the pose and motion information of the mobile equipment in the workshop is acquired in real time during the cruise and trajectory prediction is performed. When a collision risk is detected, dynamic obstacle avoidance operation is executed, specifically including:

[0025] Real-time acquisition of position, speed, and attitude information of mobile devices within the workshop;

[0026] Based on the collected information, predict the movement trajectory of the mobile device within a preset time window and calculate the predicted minimum safe distance between the drone and the mobile device.

[0027] When the predicted minimum safe distance is less than the preset safe threshold or the predicted trajectory has intersections and overlaps, a collision risk is determined, and dynamic obstacle avoidance operations such as hovering and waiting, elevation avoidance, lateral detour, speed adjustment, or task replanning are performed according to the collision risk level.

[0028] In this solution, a closed-loop process of real-time perception, trajectory prediction, risk assessment and graded obstacle avoidance is used to identify collision risks caused by dynamic mobile devices in the workshop in advance, and adapt to dynamic interference scenarios such as cranes and conveyor vehicles in the brewing workshop; the graded obstacle avoidance strategy takes into account both safety and inspection continuity, avoids mission interruption caused by a single obstacle avoidance method, and ensures that the UAV can stably and safely complete the cruise data collection in complex dynamic environments.

[0029] Furthermore, in step S3, the morphological parameters of the cellar cap include at least one or more of the following: cellar cap height, maximum bulge height, average height, bulge volume, surface curvature, slope, flatness, crack location and crack geometric features, surface color features, wettability features, and blistering features; the crack geometric features include one or more of the following: crack length, width, orientation, number, or propagation rate.

[0030] In this scheme, morphological parameters are extracted from multiple dimensions such as macroscopic deformation, local defects, and surface condition to comprehensively depict the three-dimensional structure and surface details of the kiln cap. Height and volume reflect the overall deformation trend, crack parameters quantify the weak points of sealing, and color and wettability are related to the degree of surface contamination and dryness. This achieves refined and multi-dimensional quantification of the kiln cap condition, providing complete parameter support for the assessment of sealing risks and abnormal conditions.

[0031] Furthermore, the surface color features are extracted in the HSV color space using visible light images and / or multispectral images, including the location, area, color type, and color anomaly score of anomalous color spots. ;

[0032] The color type includes at least one or more of the following: white exudate, yellow mold spots, and black mold spots;

[0033] The detection conditions for the white precipitate are as follows: and ,in, The brightness value is in the HSV color space. This represents the saturation value. The white brightness threshold. The white saturation threshold;

[0034] The color anomaly score is calculated using the following formula:

[0035] ;

[0036] in, For the first The total area of ​​abnormal pigmentation spots; This is the hazard weighting coefficient for this type of pigmentation. This is the color rating coefficient.

[0037] In this scheme, the brightness and chromaticity information are separated by HSV space to reduce the interference of light changes and accurately identify white precipitates and various mold spots. The degree of abnormality is quantified by area, weight and coefficient, and the qualitative visual judgment is transformed into a quantitative score, which objectively reflects the risk of contamination of the cellar cap surface by miscellaneous bacteria, and realizes early warning and degree classification of contamination.

[0038] Furthermore, in step S3, the temporal difference between the current period three-dimensional model and the reference period three-dimensional model specifically includes:

[0039] Establish a unified cellar coordinate system based on the corner points, boundary outlines, or pre-set landmarks of the cellar.

[0040] The current period 3D model and the reference period 3D model are respectively transformed into this unified coordinate system to complete spatial registration;

[0041] Based on the registered 3D model, point-by-point difference calculation is performed;

[0042] The reference period is the previous cruise period of the current period or the first cruise period after entering the cellar.

[0043] In step S3, the characterization of the morphological changes of the cellar cap includes one or more of the following: a height change heat map, a volume change curve, a collapse area indication, an abnormal bulge area indication, and a sealing risk score.

[0044] In this solution, the comparison error caused by the difference in attitude and viewpoint of the UAV is eliminated by unified coordinate system registration, ensuring the spatial alignment of time series data; taking the previous cycle or the first cycle as the benchmark, the comparison of short-term dynamic changes and long-term initial state is taken into account to accurately capture deformation trends such as collapse and bulging; the change characteristics are output in the form of heat map, curve, risk score, etc., to intuitively quantify the dynamic form and provide a time series and visualization basis for the dynamic assessment of sealing risks.

[0045] Furthermore, the sealing risk score is calculated using the following formula:

[0046] ;

[0047] in, Assess the risk of sealing. Score the collapse; Score the cracks; For discontinuous scoring; , , The weighting coefficients and ;

[0048] Collapse rating ,in, This represents the maximum collapse depth of the cellar cap; This is the collapse depth threshold;

[0049] Crack rating ,in, , The first The length and width of the crack; The surface area of ​​the cellar; This is the crack scoring coefficient;

[0050] Discontinuity rating ,in, For height-varying gradient fields; This is the height gradient threshold.

[0051] In this solution, a multi-factor weighted model for collapse, cracks, and discontinuities is constructed, and the weights are matched to the impact weights of each defect on the sealing performance to avoid the one-sidedness of a single indicator. By threshold normalization, defects of different dimensions are transformed into a unified scale, quantifying the sealing risk level and realizing a comprehensive and objective assessment of the sealing status, providing a quantifiable and traceable judgment standard for sealing failure alarms.

[0052] Furthermore, in step S4, the visible light image and the thermal imaging image are registered and fused, including:

[0053] Using the boundaries, corners, or pre-defined landmarks of the cellar as reference features, a feature matching algorithm is used to align the visible light image and the thermal imaging image at the pixel level, and the temperature information of the thermal imaging image is mapped to the spatial coordinate system corresponding to the visible light image to generate a temperature field distribution image that accurately corresponds to the cellar area.

[0054] In step S4, the temperature anomaly region includes hot spot anomalies and / or cold spot anomalies, which are obtained through threshold determination, statistical anomaly determination, time-series change determination, and / or model determination.

[0055] The criteria for determining hotspot anomalies are as follows: ,in, For the region Temperature value; The average surface temperature of the pit; The hotspot temperature difference threshold;

[0056] The criteria for determining the cold spot anomaly are as follows: ;in, For the region Temperature value; The cold spot temperature difference threshold;

[0057] The comprehensive score for temperature anomalies is calculated using the following formula:

[0058] ;

[0059] in, Rate the hot topics;

[0060] Rate the cold spots;

[0061] This is the temperature rating coefficient;

[0062] This is used to score temperature gradient anomalies. For temperature gradient field, This is the temperature gradient threshold;

[0063] , , The weighting coefficients and ;

[0064] The threshold and It is adaptively updated based on historical data of the fermentation pit and / or set by process parameters.

[0065] In this scheme, pixel-level registration is achieved using the inherent characteristics of the fermentation pit, eliminating spatial misalignment between visible light and thermal imaging, and ensuring accurate correspondence between temperature information and morphological regions. Based on the average temperature dynamic threshold, cold and heat anomalies are determined, and temperature benchmarks for different fermentation stages are adapted. Local mutations are captured by combining temperature gradients, and the degree of anomaly is quantified by weighted comprehensive scoring. This objectively reflects the risks of abnormal fermentation activity, local overheating, or cold zones, providing a precise temperature dimension basis for judging the fermentation status.

[0066] Furthermore, in step S5, the coupling analysis includes: performing correlation modeling between the height change characterization and the temperature field statistical features; the correlation modeling is a rule model, a machine learning model, a deep learning model, or a combination thereof;

[0067] In step S5, the statistical characteristics of the temperature field include one or more of the following: mean, variance, gradient, hot spot / cold spot area ratio, and temperature rise / fall rate.

[0068] In step S5, the risk alarm information includes at least one or more of the following: risk of seal failure, risk of fermentation stagnation, and risk of contamination by miscellaneous bacteria.

[0069] This solution integrates morphological changes and multidimensional features of the temperature field to cover the correlation between deformation and thermal effects during fermentation. It combines rule-based models with process experience and machine learning models to uncover implicit data correlations, thereby improving the robustness of the assessment. It outputs three core risk warnings: seal failure, fermentation stagnation, and microbial contamination, accurately matching key risk points in the brewing process. This enables comprehensive and intelligent assessment of the fermentation status, providing clear early warning directions for process intervention.

[0070] On the other hand, the present invention also provides a drone inspection system for solid-state brewing cellars of baijiu based on multi-source visual fusion, for implementing the above method. The system includes:

[0071] The drone patrol terminal is used to fly autonomously according to the patrol mission. During the patrol, it acquires the position and motion information of the workshop's mobile equipment in real time and performs dynamic obstacle avoidance operations. After reaching the collection position above the target pit, it simultaneously collects stereoscopic visual data of the pit cap area, multispectral image data of the pit surface, and thermal imaging image data.

[0072] Edge computing devices are used to receive data collected by drone patrol terminals, perform three-dimensional reconstruction on stereo vision data to obtain a three-dimensional model of the pit cap, extract pit cap morphological parameters and perform temporal difference to obtain pit cap morphological change characterization, register and fuse visible light images and thermal imaging images to generate the surface temperature field distribution of the pit and identify temperature anomaly areas, couple and analyze morphological change characterization and temperature field anomaly characteristics, and output fermentation status assessment results and risk alarm information.

[0073] The mobile device positioning unit includes a UWB base station network deployed in the workshop and UWB tags installed on the mobile devices, which are used to provide real-time position and operating status information of mobile devices in the workshop.

[0074] The management platform is used to generate patrol missions based on production scheduling information and the spatial distribution information of fermentation pits, and to send them to the drone patrol terminal. It also receives and displays the fermentation status assessment results and risk alarm information output by the edge computing device.

[0075] In this solution, the UAV patrol terminal undertakes autonomous front-end patrol, dynamic obstacle avoidance, and synchronous acquisition of multi-source data, ensuring the automation and safety of front-end operations; edge computing devices centrally complete 3D reconstruction, morphological analysis, temperature field generation, and coupled evaluation, realizing intelligent analysis at the back end; mobile device positioning units provide accurate pose data to support dynamic obstacle avoidance; and the management platform coordinates task scheduling, data aggregation, and visualization, thus forming an integrated system of acquisition, transmission, analysis, and control, adapting to the full-cycle, automated, and intelligent inspection needs of large-scale cellars.

[0076] The beneficial effects of this invention are:

[0077] (1) To automate the inspection of pits and improve inspection efficiency and safety:

[0078] This invention replaces manual on-site inspections with autonomous drone patrols, enabling all-weather, high-frequency, and full-coverage inspections in the complex and dynamic environment of a brewing workshop. It effectively avoids the safety hazards caused by low efficiency, incomplete coverage, data interruption, large subjective judgment bias, and numerous on-site mobile devices in manual inspections, thereby improving inspection efficiency and operational safety.

[0079] (2) Achieve precise quantification and dynamic tracking of the three-dimensional morphology of the cellar cap:

[0080] This invention uses binocular / multi-view stereo vision to acquire three-dimensional data of the cellar cap, and achieves dynamic comparison of morphological changes through temporal difference. It can accurately quantify the changes in key parameters such as cellar cap height, volume, collapse, and cracks, transforming traditional qualitative observation into quantitative indicators. It can timely and objectively identify early signs of sealing failure, improving the accuracy and timeliness of sealing risk identification.

[0081] (3) Achieve high-precision reconstruction of the surface temperature field of the pit and accurate identification of anomalies:

[0082] This invention integrates visible light and thermal imaging images at the pixel level to generate a temperature field that precisely corresponds to the fermentation pit area. It can accurately identify local hot spots, cold spots, and abnormal temperature gradients, avoiding the problems of missed detection and insufficient accuracy in single-point temperature measurement. It can objectively reflect fermentation activity, local overheating, or abnormal cold zones, providing a reliable temperature basis for judging the fermentation status.

[0083] (4) Achieve multi-source data fusion analysis for more comprehensive and reliable fermentation status assessment:

[0084] This invention integrates multi-dimensional features such as changes in the shape of the fermentation cap, abnormal surface color, abnormal humidity, and abnormal temperature distribution to construct a morphology-temperature coupled analysis model. This model overcomes the limitations of single-dimensional judgment and can simultaneously identify three core risks: sealing failure, fermentation stagnation, and contamination by miscellaneous bacteria. It enables a comprehensive and intelligent assessment of the fermentation status, resulting in more timely warnings and more reliable conclusions.

[0085] (5) Achieve safe patrol in the dynamic operating environment of the brewing workshop:

[0086] This invention uses mobile device positioning, trajectory prediction, and hierarchical dynamic obstacle avoidance to perceive mobile devices such as vehicles and conveyors in the workshop in real time, predict collision risks in advance, and execute strategies such as hovering, elevation avoidance, and lateral detour. It effectively solves the problem of safe drone patrol under dynamic interference in the workshop and ensures the stable and continuous execution of inspection tasks.

[0087] (6) Supports efficient inspection of large-scale fermentation pit groups, adapting to the needs of large-scale brewing production management:

[0088] This invention supports multi-UAV collaborative patrols, enabling parallel inspections of large-scale fermentation pits in different zones. It achieves balanced task allocation, controllable flight path conflicts, and unified data aggregation, thereby improving inspection efficiency and forming a continuous and traceable full-cycle monitoring archive for fermentation pits. This provides data support for process optimization, quality traceability, and production management. Attached Figure Description

[0089] Figure 1 This is a flowchart of the drone inspection method for solid-state brewing cellars of baijiu based on multi-source visual fusion in Example 1.

[0090] Figure 2This is a structural diagram of the drone inspection system for solid-state brewing cellars of baijiu based on multi-source visual fusion in Example 2. Detailed Implementation

[0091] This invention aims to provide a method and system for drone inspection of solid-state brewing cellars for baijiu based on multi-source visual fusion, which solves the problems of existing automated image monitoring technology for cellars being unable to quantify changes in the shape of the cellar cap in three dimensions, having fragmented multi-source monitoring data, and lacking safety inspection capabilities adapted to the dynamic environment of the brewing workshop. The core idea of ​​this invention is as follows: This invention uses a drone as a platform for multi-source visual perception sensors, integrating stereo vision, multispectral, and thermal imaging acquisition modules. It simultaneously acquires three-dimensional structural data of the fermentation pit cap through binocular / multi-view stereo vision, and combines temporal difference technology to accurately capture the dynamic changes in morphological parameters such as height, volume, collapse, and cracks of the pit cap, quantifying sealing risks. Through pixel-level registration and fusion of visible light and thermal imaging images, it generates a high-precision temperature field on the surface of the fermentation pit and accurately identifies hot spots, cold spots, and abnormal temperature gradients. Simultaneously, it uses multispectral data to extract features such as surface color and moisture content of the pit cap. By constructing a coupled analysis model of morphological change characterization and abnormal temperature field features, and integrating rule-based models and machine learning models, it comprehensively assesses three core risks: sealing failure, fermentation stagnation, and contamination by miscellaneous bacteria. By integrating mobile device positioning, trajectory prediction, and a hierarchical dynamic obstacle avoidance mechanism, it can perceive mobile devices such as mash conveyors, cranes, and grab buckets within the workshop in real time and avoid collision risks.

[0092] The following description, in conjunction with the accompanying drawings and embodiments, further illustrates the solution of the present invention. Embodiment 1 focuses on an explicit point cloud solution based on depth estimation, which is suitable for real-time priority scenarios. Embodiment 2 focuses on a neural implicit representation and Gaussian splashing solution, which is suitable for high-fidelity reproduction and version snapshot management scenarios. The two solutions can be used independently or in combination.

[0093] Example 1:

[0094] This embodiment provides a method for UAV inspection of solid-state fermentation cellars for baijiu (Chinese liquor) based on multi-source visual fusion. It is applied to a large-scale strong-aroma baijiu production workshop with 500 fermentation cellars, using a single UAV patrol mode. See also... Figure 1 The specific implementation steps include the following:

[0095] S1. Cruise Mission Planning:

[0096] In this step, a patrol mission is generated based on production scheduling information and the spatial distribution information of the pits.

[0097] Specifically, the system obtains the daily production scheduling information from the production management system, including the entry time of each fermentation pit, the current number of fermentation days, and the planned exit time. Combined with the spatial distribution information of the fermentation pits in the workshop (including pit number, location coordinates, and size parameters) and the workshop safety constraints (including no-fly zones, height-restricted zones, and safety passages), a patrol mission is generated.

[0098] In this embodiment, the patrol mission includes the following parameters: the target set of fermentation pits is 200 pits that need to be inspected on the same day (pits from the 3rd to the 45th day of fermentation); the patrol time is once at 9:00 am and once at 3:00 pm every day; the data collection frequency is to collect one set of data for each pit; the flight channel is the main channel and branch channel preset by the workshop; the no-fly zone includes the area within 3 meters above the driving operation area; the height restriction is that the flight height does not exceed 6 meters.

[0099] S2. Cruise and collect multi-source visual data:

[0100] In this step, the UAV autonomously cruises within the workshop according to the cruise mission. During the cruise, it acquires the pose and motion information of the workshop's mobile devices in real time and performs trajectory prediction. When a collision risk is detected, it performs dynamic obstacle avoidance. After flying to the collection position above the target pit, it simultaneously acquires stereoscopic visual data of the pit cap area, multispectral image data of the pit surface, and thermal imaging image data.

[0101] Specifically, the drone takes off from the charging helipad and flies within the workshop according to the planned route of the cruise mission. The sensors carried by the drone include: a binocular vision module (baseline distance 120mm, resolution 1920×1080, frame rate 30fps), a multispectral camera module (including visible light RGB channel, near-infrared NIR channel and green light G channel), and a thermal imaging module (resolution 640×480, temperature measurement range -20℃ to 150℃, accuracy ±2℃).

[0102] The drone patrol terminal also includes an indoor positioning and autonomous navigation module (composed of a UWB positioning unit, a visual odometry unit, an IMU inertial navigation unit, and a ToF laser ranging unit, achieving centimeter-level positioning through extended Kalman filter fusion), a flight control module (supporting autonomous take-off and landing, waypoint flight, hovering, and emergency return), and an onboard computing and communication module (equipped with an embedded AI computing platform, supporting 5G / WiFi 6 dual-mode communication). When the drone reaches the preset data collection position above the target cellar (approximately 2 meters above the cellar surface), it hovers and simultaneously collects binocular image data, multispectral image data, and thermal imaging image data. After data collection, the data is transmitted back to the edge computing node in real time via a wireless link.

[0103] During cruise, the system employs a hierarchical decision-making architecture for dynamic obstacle avoidance:

[0104] (1) Perception layer: The location coordinates and movement speed of mobile devices such as the mash conveyor, crane, and AGV are obtained in real time through the UWB positioning base station deployed in the workshop (UWB tags are installed on each mobile device, with a positioning accuracy better than 30cm and an update frequency of 10Hz). At the same time, the airborne visual recognition results and the interface data of the equipment control system are integrated.

[0105] (2) Prediction layer: Based on the historical trajectory and current motion state of the mobile device, the Kalman filter algorithm is used to predict the motion trajectory within a preset time period in the future. For equipment such as AGV that moves along a fixed route, path constraint prediction is used. For the vehicle, the motion intention is predicted in combination with its current work task.

[0106] (3) Decision layer: Calculate the predicted minimum distance between the UAV and the mobile device, select the obstacle avoidance strategy according to the collision risk level, maintain the flight path and reduce speed when the risk is low, perform preventive avoidance when the risk is medium, and immediately perform emergency obstacle avoidance when the risk is high (the predicted minimum distance is less than the safety threshold or the trajectory intersects).

[0107] (4) Execution layer: Dynamic obstacle avoidance strategies include hovering and waiting (when the mobile device crosses the route laterally), elevation avoidance (rising to a safe height above the equipment), lateral detour, speed adjustment and task replanning (adjusting the order of subsequent pit inspections when the mobile device occupies the route area for a long time).

[0108] S3. Three-dimensional morphological modeling and morphological change extraction:

[0109] In this step, the stereoscopic vision data is reconstructed to obtain a three-dimensional model of the cellar cap. The morphological parameters of the cellar cap are extracted, and the current period three-dimensional model and the reference period three-dimensional model are subjected to temporal difference to obtain the characterization of the cellar cap morphological change.

[0110] Specifically, after receiving the stereo image data, the edge computing node performs the following processing:

[0111] (1) Depth estimation: The disparity map of the binocular image is calculated using the PSMNet deep learning network and converted into a depth map by combining the camera calibration parameters.

[0112] (2) 3D reconstruction: Based on the depth map, a 3D point cloud of the cellar cap region is generated. After statistical filtering for noise reduction and moving least squares smoothing, a Poisson surface reconstruction algorithm is used to construct a 3D mesh model of the cellar cap.

[0113] (3) Morphological parameter extraction: The maximum height, average height, bulge volume, surface curvature distribution, and flatness index of the cellar cap are extracted from the three-dimensional model, and the location, length, width, and orientation of cracks are detected by curvature analysis and morphological methods.

[0114] (4) Spatial registration: A unified coordinate system for the cellar is established based on the four corner points of the cellar. The currently collected 3D model is transformed to this coordinate system to eliminate the influence of the difference between the UAV's flight attitude and the collection viewpoint.

[0115] (5) Time-series difference: The three-dimensional model of the current cruise cycle is compared with the baseline model (the model collected during the first cruise after entering the cellar or the model of the previous cruise cycle) point by point to generate a height change heat map, calculate the volume change, and identify the collapse area and abnormal uplift area.

[0116] (6) Sealing risk score: The sealing risk score is calculated using a multi-factor weighted scoring model.

[0117] ;

[0118] Collapse rating ;

[0119] In the formula, This represents the maximum collapse depth of the current cellar cap relative to the baseline model (unit: cm). The collapse depth threshold is taken in this embodiment. =3cm, meaning that when the maximum collapse depth reaches 3cm, the collapse score reaches 100 points. This threshold is determined based on the sealing process requirements of strong-aroma baijiu cellars and statistical analysis of historical failure data.

[0120] Crack rating ;

[0121] In this embodiment, This is used to convert the crack area percentage into a score of 0-100.

[0122] Discontinuity rating ;

[0123] In this embodiment, The height variation gradient field (unit: cm / cm) is calculated using the Sobel operator (or the finite difference method). The maximum value of the gradient field is... The gradient threshold is taken as [value] in this embodiment. =0.3, meaning that the discontinuity score reaches 100 points when the gradient of height change reaches 0.3.

[0124] The weighting coefficients can be set to: , , The aforementioned weighting coefficients were determined based on statistical analysis of historical fermentation data from the fermentation pits and the experience of process experts. The weighting distribution reflects that collapse has the greatest impact on sealing performance, followed by cracks, and finally discontinuities. The weighting coefficients can be adaptively adjusted based on historical fermentation data from the fermentation pits and the experience of process experts.

[0125] Risk level classification: based on The numerical values ​​categorize sealing risks into four levels: low risk ( ), medium risk ( High risk Extremely high risk .

[0126] In addition, color anomaly detection on the surface of the cellar cap was performed based on visible light RGB images. After converting the RGB images to the HSV color space, abnormal color spots were detected according to the following rules:

[0127] Detection conditions for white precipitates: and ;

[0128] Detection conditions for yellow mold spots: and and ;

[0129] Detection conditions for black mold spots: And the difference in brightness with the surroundings .

[0130] Surface area of ​​cellar The area of ​​white precipitate detected Yellow mold spots area Then we have:

[0131] ;

[0132] The color abnormality score of this cellar is 6.6 points, which is within the normal range (0-20 points).

[0133] Color anomaly classification: Normal ( ), mild abnormality ( ), moderate abnormality ( ), serious abnormalities ( ).

[0134] S4. Temperature Field Generation and Temperature Anomaly Extraction:

[0135] In this step, the visible light image and the thermal imaging image are registered and fused to generate the temperature field distribution on the surface of the pit, and to identify and extract areas with abnormal temperatures.

[0136] Specifically, edge computing nodes perform the following processing on visible light images and thermal imaging images:

[0137] (1) Image registration: The registration method based on feature points is adopted (SIFT is adopted in this embodiment, ORB feature matching combined with RANSAC algorithm and other methods can also be adopted). The four corners and boundaries of the cellar are used as reference features to align the thermal imaging image with the visible light image at the pixel level, and the registration error is controlled within 3 pixels.

[0138] (2) Temperature field generation: The aligned thermal imaging data is mapped to a unified coordinate system on the surface of the pit through bilinear interpolation to generate a temperature field distribution map of the pit surface with a resolution of 1cm×1cm. .

[0139] (3) Calculation of statistical characteristics:

[0140] Calculate the following statistical characteristics of the temperature field:

[0141] Average temperature: ,variance Temperature gradient field The temperature rise / fall rate was calculated using the Sobel operator: .

[0142] (4) Abnormal area scoring:

[0143] Hotspot anomaly detection: When the pixel temperature meets the following conditions... When a hotspot is identified as abnormal, the hotspot temperature difference threshold in this embodiment is used. Furthermore, the connected area of ​​the hotspot region must be greater than 100cm² to be recorded.

[0144] Cold spot anomaly detection: When the pixel temperature meets the following conditions... When the cold spot is detected as abnormal, the cold spot temperature difference threshold in this embodiment is used. Furthermore, the connected area of ​​the cold spot region must be greater than 100cm² to be recorded.

[0145] Rate the hot topics;

[0146] This is the cold spot score; in this embodiment, we take... ;

[0147] Temperature gradient anomaly scoring The calculation method is as follows: In the formula, This represents the maximum value of the temperature gradient field (unit: °C / cm). The temperature gradient threshold is taken as [value missing] in this embodiment. That is, when the temperature difference gradient between adjacent areas reaches 2℃ / cm, the gradient anomaly score reaches 100 points.

[0148] Temperature Anomaly Comprehensive Score ;

[0149] In this embodiment, the following is taken , , The weighting reflects that hot spots and cold spots are equally important in judging the fermentation state, with the temperature gradient serving as an auxiliary criterion.

[0150] Temperature anomaly classification: based on The numerical values ​​classify temperature anomalies into four levels: normal ( ), mild abnormality ( ), moderate abnormality ( ), serious abnormalities ( ).

[0151] (5) Humidity score:

[0152] Green light reflectance of a certain cellar area Near-infrared reflectance The humidity score is then:

[0153] ;

[0154] Humidity level classification: Suitable ( ), slightly dry ( ),dry( Severe dryness ).

[0155] Therefore, the pit is severely dry and needs to be replenished with water.

[0156] S5. Status Assessment and Alarm Output:

[0157] In this step, the characterization of changes in the cellar cap morphology is coupled with the anomaly characteristics of the temperature field for analysis, and the fermentation status assessment results and risk warning information are output.

[0158] Specifically, the morphological change characterization is coupled with temperature field characteristics for analysis:

[0159] (1) Feature fusion: When using machine learning and deep learning models, the height change, volume change rate, sealing risk score and temperature mean, temperature variance, hot spot / cold spot area ratio, temperature rise rate and other features are combined into a feature vector.

[0160] (2) State evaluation model: A rule-based evaluation model combined with a machine learning model is used for judgment. Examples of rule models are shown in Table 1:

[0161] Table 1 Rule Model

[0162]

[0163] (3) Alarm output: Based on the assessment results, generate risk alarm information, including the pit number, alarm type, risk level, and recommended measures, and push it to the management platform and the mobile terminals of relevant responsible persons.

[0164] Example 2:

[0165] This embodiment provides a drone inspection system for solid-state fermentation cellars of baijiu (Chinese liquor) based on multi-source visual fusion, used to implement the detection method in Embodiment 1. (See also...) Figure 2 The system includes: a drone patrol terminal, an edge computing device, a mobile device positioning unit, and a management platform. These units work together to complete multi-source data acquisition, intelligent computation, time-series analysis, and anomaly alarms for the fermentation pits, as detailed below:

[0166] The drone patrol terminal, serving as the front-end sensing and data acquisition carrier, utilizes a 350mm wheelbase quadcopter drone platform and is equipped with multiple visual acquisition sensors to simultaneously acquire multi-source data. Among these, the visible light acquisition module features an 8000×6000 pixel onboard camera with a 1-inch image sensor and an equivalent focal length of 24mm; it also includes a 640×512 pixel uncooled thermal infrared sensor with a temperature measurement range of -20℃ to +150℃, a temperature measurement accuracy of ±2℃, and a noise equivalent temperature difference NETD ≤ 50mK, enabling simultaneous acquisition of visible light images and thermal infrared data of the pit. In this embodiment, the drone patrol terminal incorporates a route planning and flight control subsystem, achieving autonomous flight based on patrol tasks issued by the management platform; simultaneously, it collaborates with the mobile device's positioning unit to achieve obstacle perception, possessing dynamic trajectory prediction and obstacle avoidance capabilities. To expand the dimensions of data acquisition, this embodiment is compatible with fixed array camera unit for auxiliary acquisition. The fixed array includes 12 industrial area array cameras with a focal length of 8mm, equipped with a 1 / 1.2-inch sensor and a frame rate of 30fps. It is paired with a synchronous trigger controller based on the IEEE1588PTP protocol. The fixed array and the UAV acquisition unit can work in separate or collaborative modes to further improve the integrity of image data.

[0167] The mobile device positioning unit adopts a UWB positioning architecture, consisting of a UWB base station network deployed in the brewing workshop and UWB tags installed on various mobile devices in the workshop. It collects the position, speed, and attitude operation data of mobile devices such as mash conveyors, cranes, and grabs in real time, and transmits the position and attitude information back to the UAV patrol terminal in real time. This provides an accurate location data source for UAV flight trajectory prediction, collision risk assessment, and dynamic obstacle avoidance operations, ensuring the safety of UAV patrol in complex and dynamic production environments.

[0168] The edge computing device is the core data processing and analysis unit of this system, undertaking all the computational work of data calibration and registration, 3D scene construction, temporal difference analysis, and anomaly detection and alarm. The edge computing device receives multi-view image sequences and thermal infrared data transmitted from the UAV cruise terminal and the fixed array camera unit. First, it completes camera calibration and pose estimation: through the internal and external parameter calibration submodule, the fixed array joint calibration is completed by using the Zhang Zhengyou calibration method combined with a 12×9 checkerboard calibration board, with a reprojection error of less than 0.5 pixels; relying on the ORB-SLAM3 visual SLAM submodule, the image pose during the UAV flight acquisition process is restored, and RTK-GNSS information can be optionally fused, with the absolute position error of the fused trajectory being less than 3cm; at the same time, physical scale restoration is completed based on the ArUco coded scale, with a restoration accuracy better than 0.5%, and a unified three-dimensional coordinate system for the cellar is established.

[0169] Furthermore, the edge computing device incorporates a 3D reconstruction algorithm, employing a combination of neural implicit representation and explicit point cloud representation to achieve high-fidelity 3D scene reconstruction of the cellar. Specifically, the neural radiation field modeling method based on the Instant-NGP architecture achieves a baseline scene peak signal-to-noise ratio (PSNR) ≥ 30dB and structural similarity (SSIM) ≥ 0.93; the 3D Gaussian splash modeling method achieves PSNR ≥ 28dB and SSIM ≥ 0.90, with a rendering speed exceeding 100fps at 1080p resolution. This device supports incremental scene updates and local reconstruction, with a single incremental update taking approximately 3 minutes. Reconstruction is performed only on the changed areas of the model and then merged into the global model, simultaneously generating and managing PLY format scene model version snapshots.

[0170] Meanwhile, the edge computing device incorporates a time-series analysis algorithm module to perform spatial registration and temporal difference calculations on 3D scene models with different cruise cycles. It employs a combination of FPFH coarse registration and Point-to-Plane ICP fine registration to achieve model alignment with a registration accuracy of less than 1mm. A kd-tree nearest neighbor search algorithm is used to perform scene difference calculations, accurately extracting anomalous features such as crack propagation, pit cap deformation, and foreign object coverage, and outputting quantitative indicators such as anomalous spatial coordinates, deformation, crack size, and volume change. The edge computing device also performs registration and fusion of visible light and thermal infrared images to generate a high-precision pit temperature field distribution, identify temperature anomaly regions, and couple pit cap morphological changes with temperature anomaly features to complete a comprehensive evaluation.

[0171] In this embodiment, the edge computing device adopts a fusion decision strategy (fusion weight) that combines threshold rules with the XGBoost classification model. The abnormal risks are divided into three levels: general concern (score). Important Focus (Rating) ), emergency response ( The device is configured with a tiered confirmation mechanism to reduce false alarm rates. Emergency-level anomalies employ a single-frame instant warning + two-frame confirmation strategy; critical-level anomalies require confirmation from two consecutive frames of data; and general-level anomalies require confirmation from three consecutive frames of data. For scenarios with insufficient historical annotation data, the device can automatically switch to a pure rule-based judgment mode. This enables adaptive cold start judgment. The edge computing device stores the generated fermentation status assessment results, anomaly levels, and alarm types in the built-in SQLite database, realizing the association storage of alarm information, suspected anomaly data, and 3D scene snapshots.

[0172] The management platform, deployed at the back-end control end, generates drone patrol missions by combining workshop production scheduling information and fermentation pit spatial distribution information. It completes route planning, task allocation, and route conflict resolution, and distributes the patrol missions to the drone patrol terminals. At the same time, it receives fermentation pit morphology parameters, temperature field data, fermentation assessment results, and risk alarm information transmitted back from edge computing devices in real time. It visualizes the 3D model of the fermentation pit, abnormal locations, and abnormal levels, and supports retrospective query of historical monitoring data and alarm records, providing visualized data support for brewing production control, process adjustment, and risk management.

[0173] Although embodiments of the present invention have been described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the present invention, and all such changes and alterations shall not depart from the protection scope of the present invention.

Claims

1. A method for unmanned aerial vehicle (UAV) inspection of solid-state fermentation cellars for Baijiu (Chinese liquor) based on multi-source visual fusion, characterized in that, Includes the following steps: S1. Generate patrol missions based on production scheduling information and pit spatial distribution information; S2. The UAV autonomously cruises within the workshop according to the cruise mission. During the cruise, it acquires the pose and motion information of the workshop's mobile devices in real time and performs trajectory prediction. When a collision risk is detected, it performs dynamic obstacle avoidance. After flying to the collection position above the target pit, it simultaneously acquires stereoscopic visual data of the pit cap area, multispectral image data of the pit surface, and thermal imaging image data. S3. Perform three-dimensional reconstruction on the stereo vision data to obtain a three-dimensional model of the cellar cap, extract the morphological parameters of the cellar cap, and perform temporal difference between the current period three-dimensional model and the reference period three-dimensional model to obtain the morphological change characterization of the cellar cap; S4. Register and fuse the visible light image with the thermal imaging image to generate the surface temperature field distribution of the pit, and identify and extract the temperature anomaly areas; S5. Couple the characterization of changes in the pit cap morphology with the characteristics of temperature field anomalies, and output the fermentation status assessment results and risk warning information.

2. The method for UAV inspection of solid-state brewing cellars for Baijiu based on multi-source visual fusion as described in claim 1, characterized in that, In step S1, the patrol mission includes at least: a target set of the reservoir, patrol time / cycle, data collection frequency, flight channel and no-fly / altitude restrictions; when it is a multi-UAV collaborative mode, the target set of the reservoir is divided into multiple sub-tasks according to spatial partitioning, batch grouping and / or task load and assigned to multiple UAVs, while conflict resolution is performed on the corresponding flight paths of each sub-task.

3. The method for UAV inspection of solid-state brewing cellars for Baijiu based on multi-source visual fusion as described in claim 1, characterized in that, In step S2, the pose and motion information of the mobile equipment in the workshop is acquired in real time during the cruise and trajectory prediction is performed. When a collision risk is detected, dynamic obstacle avoidance is executed, specifically including: Real-time acquisition of position, speed, and attitude information of mobile devices within the workshop; Based on the collected information, predict the movement trajectory of the mobile device within a preset time window and calculate the predicted minimum safe distance between the drone and the mobile device. When the predicted minimum safe distance is less than the preset safe threshold or the predicted trajectory has intersections and overlaps, a collision risk is determined, and dynamic obstacle avoidance operations such as hovering and waiting, elevation avoidance, lateral detour, speed adjustment, or task replanning are performed according to the collision risk level.

4. The method for UAV inspection of solid-state brewing cellars for Baijiu based on multi-source visual fusion as described in claim 1, characterized in that, In step S3, the morphological parameters of the cellar cap include at least one or more of the following: cellar cap height, maximum bulge height, average height, bulge volume, surface curvature, slope, flatness, crack location and crack geometric features, surface color features, wettability features, and blistering features; the crack geometric features include one or more of the following: crack length, width, orientation, number, or propagation rate.

5. The method for UAV inspection of solid-state brewing cellars for Baijiu based on multi-source visual fusion as described in claim 4, characterized in that, The surface color features are extracted in the HSV color space using visible light images and / or multispectral images, including the location, area, color type, and color anomaly score of aberrant color spots. ; The color type includes at least one or more of the following: white exudate, yellow mold spots, and black mold spots; The detection conditions for the white precipitate are as follows: and ,in, The brightness value is in the HSV color space. This represents the saturation value. The white brightness threshold. The white saturation threshold; The color anomaly score is calculated using the following formula: ； in, For the first The total area of ​​abnormal pigmentation spots; This is the hazard weighting coefficient for this type of pigmentation. This is the color rating coefficient.

6. The method for UAV inspection of solid-state brewing cellars for Baijiu based on multi-source visual fusion as described in claim 1, characterized in that, In step S3, the temporal difference between the current period three-dimensional model and the reference period three-dimensional model specifically includes: Establish a unified cellar coordinate system based on the corner points, boundary outlines, or pre-set landmarks of the cellar. The current period 3D model and the reference period 3D model are respectively transformed into this unified coordinate system to complete spatial registration; Based on the registered 3D model, point-by-point difference calculation is performed; The reference period is the previous cruise period of the current period or the first cruise period after entering the cellar. In step S3, the characterization of the morphological changes of the cellar cap includes one or more of the following: a height change heat map, a volume change curve, a collapse area indication, an abnormal bulge area indication, and a sealing risk score.

7. The method for UAV inspection of solid-state brewing cellars for Baijiu based on multi-source visual fusion as described in claim 6, characterized in that, The sealing risk score is calculated using the following formula: ； in, Assess the risk of sealing. Score the collapse; Score the cracks; For discontinuous scoring; , , The weighting coefficients and ; Collapse rating ,in, This represents the maximum collapse depth of the cellar cap; This is the collapse depth threshold; Crack rating ,in, , The first The length and width of the crack; The surface area of ​​the cellar; This is the crack scoring coefficient; Discontinuity rating ,in, For height-varying gradient fields; This is the height gradient threshold.

8. The method for UAV inspection of solid-state brewing cellars for Baijiu based on multi-source visual fusion as described in claim 1, characterized in that, In step S4, the visible light image and the thermal imaging image are registered and fused, including: Using the boundaries, corners, or pre-defined landmarks of the cellar as reference features, a feature matching algorithm is used to align the visible light image and the thermal imaging image at the pixel level, and the temperature information of the thermal imaging image is mapped to the spatial coordinate system corresponding to the visible light image to generate a temperature field distribution image that accurately corresponds to the cellar area. In step S4, the temperature anomaly region includes hot spot anomalies and / or cold spot anomalies, which are obtained through threshold determination, statistical anomaly determination, time-series change determination, and / or model determination. The criteria for determining hotspot anomalies are as follows: ,in, For the region Temperature value; The average surface temperature of the pit; The hotspot temperature difference threshold; The criteria for determining the cold spot anomaly are as follows: ;in, For the region Temperature value; The cold spot temperature difference threshold; The comprehensive score for temperature anomalies is calculated using the following formula: ； in, Rate the hot topics; Rate the cold spots; This is the temperature rating coefficient; This is used to score temperature gradient anomalies. For temperature gradient field, This is the temperature gradient threshold; , , The weighting coefficients and ; The threshold and It is adaptively updated based on historical data of the fermentation pit and / or set by process parameters.

9. The method for UAV inspection of solid-state brewing cellars for Baijiu based on multi-source visual fusion as described in any one of claims 1 to 8, characterized in that, In step S5, the coupling analysis includes: performing correlation modeling between the height change characterization and the temperature field statistical features; the correlation modeling is a rule model, a machine learning model, a deep learning model, or a combination thereof; In step S5, the statistical characteristics of the temperature field include one or more of the following: mean, variance, gradient, hot spot / cold spot area ratio, and temperature rise / fall rate. In step S5, the risk alarm information includes at least one or more of the following: risk of seal failure, risk of fermentation stagnation, and risk of contamination by miscellaneous bacteria.

10. A drone inspection system for solid-state fermentation cellars of baijiu based on multi-source visual fusion, used to implement the drone inspection method for solid-state fermentation cellars of baijiu based on multi-source visual fusion as described in any one of claims 1 to 9, characterized in that, The system includes: The drone patrol terminal is used to fly autonomously according to the patrol mission. During the patrol, it acquires the position and motion information of the workshop's mobile equipment in real time and performs dynamic obstacle avoidance operations. After reaching the collection position above the target pit, it simultaneously collects stereoscopic visual data of the pit cap area, multispectral image data of the pit surface, and thermal imaging image data. Edge computing devices are used to receive data collected by drone patrol terminals, perform three-dimensional reconstruction on stereo vision data to obtain a three-dimensional model of the pit cap, extract pit cap morphological parameters and perform temporal difference to obtain pit cap morphological change characterization, register and fuse visible light images and thermal imaging images to generate the surface temperature field distribution of the pit and identify temperature anomaly areas, couple and analyze morphological change characterization and temperature field anomaly characteristics, and output fermentation status assessment results and risk alarm information. The mobile device positioning unit includes a UWB base station network deployed in the workshop and UWB tags installed on the mobile devices, which are used to provide real-time position and operating status information of mobile devices in the workshop. The management platform is used to generate patrol missions based on production scheduling information and the spatial distribution information of fermentation pits, and to send them to the drone patrol terminal. It also receives and displays the fermentation status assessment results and risk alarm information output by the edge computing device.

Images