Method and system for multi-source data fusion and collaborative complementation of digitalized stockyard

Abstract

The application discloses a multi-source data fusion collaborative filling method and system of a digitalized stockyard, relates to the technical field of data fusion, and comprises the following steps: collecting a multi-source data set of a target stockyard, obtaining stockyard source data through spatial three-dimensional alignment; performing neighborhood point density detection, splicing data missing boundaries, and executing triangular mesh segmentation and marking; taking multi-modal data as input, combining semantic description constraints and implicit physical constraints, and building a diffusion filling unit; importing the marked data into the unit, then taking random noise of the triangular mesh as the starting point, generating a data filling result through multi-round iteration denoising under constraint guidance, filling the triangular mesh, and determining the stockyard filling data. The application solves the technical problem that the data collected by multiple devices in the existing digitalized stockyard is mutually fragmented and there is data missing due to environmental factors, achieves the technical effect of integrating multi-source data and accurately filling the missing data, and forms complete and reliable stockyard data.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data fusion, in particular to a multi-source data fusion and collaborative filling method and system for a digitalized stockyard. BACKGROUND

[0002] A digitalized stockyard is a key scene of material management in industrial production, and its digital modeling relies on multi-source data support, and data processing and pattern recognition are core technologies to ensure modeling accuracy. In the prior art, the stockyard usually uses laser radar, video camera, thermal imaging equipment and other independent data collection devices, but the data of each device is fragmented, and there is a lack of unified spatial alignment during data processing, and it is difficult for pattern recognition to effectively associate multi-source features. In addition, due to water surface mirror reflection, shielding and other factors, laser point cloud is prone to missing, traditional data processing cannot accurately complete, and pattern recognition also has difficulty in locating the missing boundary, resulting in incomplete and inaccurate data, which cannot meet the needs of digital modeling and accurate control of the stockyard. SUMMARY

[0003] The present application provides a multi-source data fusion and collaborative filling method and system for a digitalized stockyard, which solves the technical problem of fragmented data collected by multiple devices in the existing digitalized stockyard and data missing caused by environmental factors.

[0004] In a first aspect, the present application provides a multi-source data fusion and collaborative filling method for a digitalized stockyard, which comprises: collecting a multi-source data set of a target stockyard, performing spatial three-dimensional alignment, and taking the stockyard source data; detecting the neighborhood point density of the stockyard source data, splicing the data missing boundary and performing triangular mesh segmentation, and marking in the stockyard source data; taking multi-modal data as input, taking the semantic description constraint based on the prompt engineering and the implicit physical constraint as input, building a diffusion filling unit; importing the marked stockyard source data into the diffusion filling unit, determining the generated constraint condition, taking the random noise based on the triangular mesh as the starting point, executing multi-round iteration denoising under the constraint guidance, generating the data filling result, corresponding filling the triangular mesh, and determining the stockyard filling data.

[0005] In a second aspect of the present application, a multi-source data fusion collaborative filling system for a digitalized stockyard is provided, which comprises: a stockyard source data acquisition module, configured to collect a multi-source data set of a target stockyard, perform spatial three-dimensional alignment, and serve as stockyard source data; a triangular mesh segmentation execution module, configured to perform neighborhood point density detection on the stockyard source data, splice data missing boundaries and perform triangular mesh segmentation, and mark in the stockyard source data; a diffusion filling unit construction module, configured to take multi-modal data as input, build a diffusion filling unit based on semantic description constraints and implicit physical constraints of a prompt project; and a stockyard filling data acquisition module, configured to import the marked stockyard source data into the diffusion filling unit, determine a generated constraint condition, perform multi-round iteration denoising under constraint guidance based on random noise of a triangular mesh as a starting point, generate a data filling result, perform corresponding filling on the triangular mesh, and determine stockyard filling data.

[0006] The one or more technical solutions provided in the present application have at least the following technical effects or advantages:

[0007] In the present application, fusion acquisition equipment is deployed in the entire digitalized stockyard to collect multi-source data such as laser point cloud, thermal imaging temperature measurement and images, complete stockyard data is obtained through spatial three-dimensional alignment, neighborhood point density detection, positioning of data missing boundaries, multi-round iteration denoising under constraint guidance and other processes, the filling process is adjusted in combination with implicit physical laws and local scene characteristics, the missing area of stockyard data caused by mirror reflection, shielding and the like is accurately completed, complete and reliable data support is obtained for digitalized stockyard digital modeling, the technical effect of integrating multi-source data and accurately completing missing data is achieved, and complete and reliable stockyard data is formed. BRIEF DESCRIPTION OF DRAWINGS

[0008] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0009] Figure 1 FIG. 1 is a flow diagram of a multi-source data fusion collaborative filling method for a digitalized stockyard provided by an embodiment of the present application.

[0010] Figure 2 FIG. 2 is a structural diagram of a multi-source data fusion collaborative filling system for a digitalized stockyard provided by an embodiment of the present application.

[0011] FIG. 1 is a flow diagram of a multi-source data fusion collaborative filling method for a digitalized stockyard provided by an embodiment of the present application. DETAILED DESCRIPTION

[0012] The application provides a multi-source data fusion and collaborative filling method and system for a digitized stockyard, which solves the technical problem of data missing caused by environmental factors due to the mutual fragmentation of data collected by multiple devices in the existing digitized stockyard.

[0013] The technical solutions in the embodiments of the application will be clearly and completely described below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are only part of the embodiments of the application, not all the embodiments. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the application.

[0014] It should be noted that the terms "first", "second", etc. in the specification and the above drawings of the application are used to distinguish similar objects, and do not necessarily mean a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or server including a series of steps or units does not have to be limited to those steps or units clearly listed, but can include other steps or modules that are not clearly listed or inherent to these processes, methods, products or devices.

[0015] Embodiment one, as shown in the figure, a multi-source data fusion and collaborative filling method for a digitized stockyard, wherein the method comprises: Figure 1

[0016] Collecting a multi-source data set of the target stockyard, performing spatial three-dimensional alignment, and taking the stockyard source data.

[0017] Specifically, the fusion device is composed of a laser radar, a thermal imaging lens, a rotating holder and a visible light lens, wherein the rotating holder can drive each component to realize multi-space angle rotation; during collection, the fusion device is deployed to the stockyard area, and a joint collection operation of multi-source data is performed by the fusion device to obtain the required multi-source data set, which is described in detail in the subsequent content.

[0018] Next, it is clear that the multi-source data set includes laser point cloud data, thermal imaging temperature measurement data and image data based on the coordinate system of each device component; then a unified coordinate system is determined; finally, the data coordinate conversion and phase alignment of the multi-source data set are completed through the spatial coordinate phase conversion operation of converting each component coordinate system to the unified coordinate system, thereby obtaining the stockyard source data, which is also described in detail in the subsequent content.

[0019] ​The source data of the stockyard is subjected to neighborhood point density detection, splicing of data missing boundary and triangulation segmentation, and is marked in the source data of the stockyard.

[0020] Optionally, first, neighborhood conditions are set, which include a predetermined neighborhood radius and a neighborhood data point quantity; then, for each data point in the source data of the stockyard, a neighborhood point density value based on the neighborhood conditions is calculated; subsequently, it is judged whether each neighborhood point density value meets a preset density threshold value, and the data points that do not meet the threshold value are determined as missing data, and the corresponding data points are taken as missing boundary data points; finally, the missing data boundary is spliced according to the missing boundary data points, which is described in detail in the subsequent content.

[0021] After the missing data boundary is spliced, the missing data boundary is then preprocessed to ensure the reliability of the segmentation basis. The boundary closure is checked, all vertex coordinates of the missing data boundary are traversed first, the spatial distance between the first and last vertices is calculated, and if the distance is greater than a preset small threshold value, such as 1 cm, a closed contour is formed by connecting the first and last vertices by a straight line. At the same time, a repeated point elimination strategy is adopted, the boundary point set is traversed, and the redundant points with the same coordinates are deleted to avoid calculation errors in subsequent segmentation, and finally a complete and non-redundant closed missing boundary is obtained, which clearly defines the area range for triangulation segmentation.

[0022] Next, sampling points within the missing data boundary are generated to support the construction of the triangulation, the minimum bounding box is calculated according to the vertex coordinates of the closed missing boundary, i.e., the maximum and minimum values of the boundary in the X, Y and Z axis directions are determined, forming a cubic space containing the entire missing region. Then, uniformly distributed candidate sampling points are generated in the bounding box at a fixed interval of 5 cm, and the ray method is used to determine whether each candidate point is inside the boundary: a ray is emitted from the candidate sampling point in any direction, the number of intersection points of the ray and the boundary contour is counted, and if the number of intersection points is odd, it is determined that the candidate point is an internal sampling point, and if the number of intersection points is even, it is determined that the candidate point is an external sampling point. The internal sampling points selected and the missing boundary vertices together form a complete point set required for triangulation segmentation.

[0023] After that, the triangular mesh segmentation adopts the incremental Delaunay triangulation technology, which can generate uniform triangles and is easy to implement. First, initialize the triangular mesh, and construct the initial triangle with the three non-collinear vertices of the data missing boundary as the starting point of the triangulation. Then, insert the internal sampling points in sequence one by one, and check the Delaunay condition after each insertion by the circumscribed circle detection method: calculate the circumscribed circle of the triangle formed around the newly inserted point, and if the point is inside the circumscribed circle of a certain triangle, delete the common edge of the triangle and reconnect the vertices to form a new triangle that satisfies the condition. Repeat the insertion and adjustment process until all internal sampling points are processed, and finally generate a triangular mesh that covers the entire missing area, and the mesh boundary completely coincides with the preset data missing boundary.

[0024] When marking in the stockyard source data, the method of grid ID association and attribute assignment is adopted. A unique digital identifier is assigned to each triangular mesh generated, such as a continuous integer starting from 1, recording the coordinates of the three vertices of each mesh and the corresponding spatial position information. Then, all data points in the stockyard source data are traversed, and it is judged whether the data point belongs to the triangular mesh of the missing area by matching the spatial coordinates: if the coordinates of the data point fall within the spatial range formed by the three vertices of a certain triangular mesh, then add an attribute label to the data point in the stockyard source data, indicating the triangular mesh ID to which it belongs. At the same time, establish the correspondence between the grid ID and the position of the missing area in the meta-information of the stockyard source data, and clearly indicate the specific orientation of each grid in the stockyard, thereby completing the accurate marking of the segmented area.

[0025] Through boundary closure processing, bounding box sampling and ray method screening, incremental Delaunay triangulation, and grid ID association marking, the missing area in the stockyard source data is converted into a structured triangular mesh and accurately marked.

[0026] Taking multi-modal data as input, a diffusion completion unit is built based on semantic description constraints and implicit physical constraints based on prompt engineering.

[0027] In the embodiments of the present application, prompt engineering refers to a technical method of designing and optimizing input prompts to clearly convey task requirements and constraint conditions, i.e., semantic description constraints, to AI models to guide the model to generate expected target output.

[0028] In an embodiment of the present application, first, a constraint threshold is built based on a semantic description constraint based on a prompt engineering and an implicit physical constraint, wherein the implicit physical constraint includes a statics law of a material rest angle and a heat conduction law of heat diffusion; then, a data completion layer is built through point cloud texture iterative generation under random noise completion; finally, a completion architecture is constructed according to the constraint threshold and the data completion layer, and iterative training is performed on the architecture until the training converges, thereby generating a diffusion completion unit, which is described in detail in the subsequent content.

[0029] The marked stockyard source data is imported into the diffusion completion unit to determine a generation constraint condition, to perform multi-round iterative denoising under constraint guidance based on random noise of a triangular mesh as a starting point, to generate a data completion result, and to correspondingly fill the triangular mesh to determine stockyard completion data.

[0030] Specifically, first, the marked stockyard source data is imported into the diffusion completion unit, then a first semantic term corresponding to the triangular mesh is generated based on the constraint threshold and the known information of the neighborhood of the triangular mesh, and finally the first semantic term is fused with the implicit physical constraint to form a generation constraint condition, which is described in detail in the subsequent content.

[0031] Next, for any first triangular mesh divided according to the data missing boundary, first mesh generation data is determined through random noise based on random noise of a triangular mesh as a starting point; then second mesh generation data is generated by denoising guided by the generation constraint condition; then the second mesh generation data is taken as a reference to perform multi-round iterative denoising guided by the generation constraint condition until the iterative convergence condition of smooth transition of the boundary is met to determine Nth mesh generation data; finally, the Nth mesh generation data is taken as the data completion result of the first triangular mesh, which is described in detail in the subsequent content.

[0032] Finally, a group of triangular meshes generated in one step is determined based on the triangular meshes adjacent to the data missing boundary; then, for the group of triangular meshes, both the first generation constraint condition is determined based on the constraint threshold and the diffusion iterative generation is performed based on the data completion layer to further determine a group of data completion results; then the group of data completion results is taken as known data part to update the data missing boundary, and two groups of triangular meshes generated in two steps are determined according to the updated missing boundary, and the internal diffusion type supplement is performed on the two groups of triangular meshes to further determine the stockyard completion data, which is described in detail in the subsequent content.

[0033] Further, the method provided in the embodiment of the present application comprises:

[0034] According to the fusion device, wherein the fusion device is composed of a laser radar, a thermal imaging lens, a rotating holder and a visible light lens, the rotating holder performs multi-space angle rotation of the components; the fusion device is deployed to a stockyard area to perform joint collection of multi-source data and obtain the multi-source data set.

[0035] Specifically, when collecting the multi-source data set of the target stockyard, the fusion device is first built. The laser radar uses pulse laser ranging method, emits periodic laser beams to the stockyard, receives the reflection signals of the laser after encountering the material surface, calculates the distance between the laser radar and the material point by using the time difference of signal propagation, and then generates stockyard three-dimensional laser point cloud data. The temperature measurement range of the thermal imaging lens is -20-150°C. Based on the principle of infrared thermal imaging, the infrared energy radiated by the material surface of the stockyard is detected, the infrared energy signal is converted into an electric signal, and then the thermal imaging temperature measurement data of the corresponding area is generated after signal processing, so as to capture the temperature distribution of the material. The visible light lens uses charge coupled device (CCD) or complementary metal oxide semiconductor (CMOS) imaging, converges the light of the stockyard through the lens optical system, forms an optical image on the photosensitive element, and then converts it into digital image data to record the texture details of the stockyard. The rotating holder is driven by a stepping motor, accurately adjusts the rotation angle of the motor after receiving the control signal, and drives the laser radar, thermal imaging lens and visible light lens to realize multi-space angle rotation of 360 degrees horizontally and 90 degrees vertically, so as to ensure that the device can meet the collection requirements of different directions of the stockyard.

[0036] After the fusion device is built, the joint collection of multi-source data is carried out in combination with the existing site deployment. First, the target stockyard area is surveyed by GPS positioning or laser range finder to determine the distribution of the stockpile, the key collection points, such as the top, edge and areas prone to obstruction of the stockpile, and then the fusion device is fixed at the preset collection point using a support to adjust the height and initial angle of the device to ensure the collection field of view. Then, a synchronous controller is used to send synchronous signals to the laser radar, thermal imaging lens, visible light lens and rotating holder to ensure that each component starts working at the same time. The rotating holder rotates gradually at preset angle intervals, for example, every 5 degrees is one collection interval, and it stops at each angle for a preset time. At this time, the laser radar, thermal imaging lens and visible light lens collect point cloud, thermal imaging temperature and image data at the corresponding angle synchronously. After completing the traversal collection of the entire stockyard area, all collected data is integrated to form a multi-source data set containing laser point cloud, thermal imaging temperature and image data.

[0037] By using the methods of pulse laser ranging, infrared thermal imaging, CCD / CMOS imaging, stepping motor driving and synchronous trigger control, the fusion device is built and the joint collection of multi-source data of the stockyard is completed, so as to ultimately achieve the effect of obtaining a comprehensive and time-space synchronous multi-source data set of the stockyard, which provides a basis for subsequent data processing.

[0038] Further, the method provided by the embodiments of the present application comprises:

[0039] The multi-source data set comprises laser point cloud data based on a component coordinate system, thermal imaging temperature measurement data, and image data; a unified coordinate system is determined; spatial coordinate phase conversion from the component coordinate system to the unified coordinate system is performed to perform data coordinate conversion and phase alignment on the multi-source data set as the stockyard source data.

[0040] Optionally, when performing spatial three-dimensional alignment on the multi-source data set, first, the component coordinate system parameters corresponding to each data in the multi-source data set are determined. The component coordinate systems of the laser radar, the thermal imaging lens, and the visible light lens are determined through device calibration methods. Specifically, for the laser radar, an angle reflector calibration method is used. A plurality of angle reflectors are placed in a calibration field with known three-dimensional coordinates. The laser radar collects reflector point cloud data. The origin position and X, Y, and Z axis directions of the laser radar coordinate system are determined by calculating the deviation of the point cloud from the known coordinates. For the thermal imaging lens and the visible light lens, Zhang Zhengyou calibration method is used. A plurality of groups of images with checkerboard calibration plates are collected. The pixel coordinates and actual three-dimensional coordinates of the checkerboard corner points are extracted to solve the internal parameters of the lens, including the focal length, the principal point coordinates, and the external parameters, including the installation position and angle of the lens relative to the gimbal, so as to determine the component coordinate system reference of the thermal imaging temperature measurement data and the image data, and clearly distinguish the original coordinate attribution of different data sources.

[0041] Then, when determining the unified coordinate system, reference points on the spot of the stockyard can be relied on. First, select 3 or more fixed reference points on the target stockyard, such as the top of the concrete column at the edge of the stockyard, a pre-set metal measurement marker, etc. The actual three-dimensional coordinates X, Y, and Z values of these reference points are measured using a total station. One of the reference points is used as the origin. The X axis direction is determined by the line connecting two points. The Y axis direction is determined by being perpendicular to the X axis and parallel to the ground of the stockyard. The Z axis direction is determined by being perpendicular to the ground and upward. A unified coordinate system corresponding to the actual space of the stockyard is constructed to ensure that the data after alignment can be accurately mapped to the real position of the stockyard.

[0042] Then, when performing spatial coordinate phase conversion, the homogeneous coordinate transformation method is used. According to the component coordinate system parameters and the unified coordinate system parameters obtained in the above steps, the coordinate transformation matrix corresponding to the laser radar, the thermal imaging lens, and the visible light lens is calculated. The matrix contains translation parameters, i.e., the distance from the origin of the component coordinate system to the origin of the unified coordinate system, and rotation parameters, i.e., the included angle between the axes of the component coordinate system and the axes of the unified coordinate system.

[0043] Afterwards, each group of data in the multi-source data set is converted: for laser point cloud data, the coordinates of each point are substituted into the coordinate transformation matrix of the laser radar to obtain its coordinates in the unified coordinate system; for thermal imaging temperature measurement data, the spatial position corresponding to each temperature measurement pixel is converted to the unified coordinate system according to the coordinate transformation matrix of the thermal imaging lens; for image data, the coordinates are converted through the visible light lens coordinate transformation matrix. At the same time, the time stamp during data collection is used for synchronization, and the laser point cloud, thermal imaging temperature measurement, image data at the same time stamp are screened and converted to ensure that different types of data are phase-aligned in the time dimension, and finally the source data of the stockyard in the unified coordinate system is integrated.

[0044] Through corner reflector calibration, Zhang Zhengyou calibration, reference point measurement of total station, homogeneous coordinate transformation, time stamp synchronization and other methods, the coordinate conversion and phase alignment of the multi-source data set are completed, so that the data of the stockyard of different devices and different coordinate systems are unified to the same spatial reference, and the effect of standardized stockyard source data is formed.

[0045] Further, the method provided by the embodiment of the application comprises:

[0046] The neighborhood condition is set, wherein the neighborhood condition is a pre-set neighborhood radius and a neighborhood data point number; for the source data of the stockyard, a neighborhood point density value based on the neighborhood condition is calculated for each data point; it is determined whether each neighborhood point density value meets a density threshold value, it is determined that data is missing, and the data point is taken as a missing boundary data point; and the missing boundary data point is used to form the data missing boundary.

[0047] Specifically, when the neighborhood point density of the source data of the stockyard is detected to splice the data missing boundary, first, the neighborhood condition is set as a pre-set neighborhood radius and a neighborhood data point number. The setting of the neighborhood radius and the neighborhood data point number can refer to the basic features of the source data of the stockyard, for example, if the average point spacing of the point cloud data collected by the laser radar is 5 cm, in order to ensure that enough reference points around the data point can be covered, the neighborhood radius can be pre-set to 10 cm; at the same time, in combination with the accuracy requirement of the data collection of the stockyard, if the reliability of the density calculation needs to be ensured, the neighborhood data point number can be pre-set to at least 3, which can effectively define the neighborhood range of each data point participating in the density calculation.

[0048] Next, the neighborhood point density value of each data point in the stockyard source data is calculated, and a fixed radius neighborhood search method is adopted to realize the KD tree index technology: first, all the stockyard source data points are imported into the KD tree structure, and the spatial index of the data points is quickly established through the KD tree to reduce the time cost of subsequent search. Then, each data point is traversed, and the other data points falling within the range of the preset neighborhood radius centered on the data point are quickly searched through the KD tree, and the number of these data points is counted. Finally, the number of neighborhood data points obtained by counting is divided by the volume of the neighborhood sphere, wherein the volume of the neighborhood sphere is calculated by the preset radius, to obtain the neighborhood point density value of each data point, which improves the search efficiency while avoiding complex calculation logic.

[0049] Then, it is determined whether each neighborhood point density value meets the density threshold to determine the missing boundary data point. The determination of the density threshold is specifically as follows: first, the average value of the neighborhood point density values of all data points is calculated, and then 50% of the average value is set as the density threshold. The proportion can be adjusted according to the sparsity of the stockyard source data, for example, it can be adjusted to 40% when the data is relatively sparse as a whole. Then, the neighborhood point density value of each data point is compared with the threshold value. If the neighborhood point density value of a certain data point is lower than the density threshold value, it indicates that the surrounding data of the data point is sparse, and the data point can be determined as an edge point of the data missing area and marked as a missing boundary data point, thereby quickly screening the missing boundary data points meeting the conditions.

[0050] Finally, the data missing boundary is formed according to the missing boundary data points. First, all the missing boundary data points are sorted according to the X, Y and Z values of their spatial coordinates to ensure that the points at adjacent positions can be arranged in sequence. Then, each missing boundary data point is traversed, and the spatial distance between the missing boundary data point and other missing boundary data points is calculated. Two points with a distance less than a preset threshold, such as 1.5 times the neighborhood radius, are regarded as adjacent boundary points, and are connected by a line segment. After the connection of all the missing boundary data points is completed, a continuous polyline is formed, and the polyline is the data missing boundary. The above process does not depend on feature division, but is based on the spatial position and density correlation of the data points, and the boundary trajectory is gradually spliced by point-by-point connection, to ensure the continuity and accuracy of the boundary.

[0051] By referring to the data features to set the neighborhood condition, using the KD tree search to calculate the density value, using the statistical method to determine the threshold, and using the adjacent point connection to splice the boundary, the effect of accurately detecting the missing area in the stockyard source data and forming a continuous data missing boundary is achieved.

[0052] Further, the method provided by the embodiments of the present application comprises:

[0053] A statics law of a material repose angle based on a stockpile geometry is taken as a first implicit physical constraint; a heat conduction law based on heat diffusion under a surface temperature field is taken as a second implicit physical constraint; and the first and second implicit physical constraints are added to the implicit physical constraints.

[0054] Specifically, when the first implicit physical constraint is constructed, a material repose angle range of a target stockpile is first determined. For a coal yard or the like, a stacking experiment method can be used. A material to be detected, such as different coal types, is placed on a horizontal plane, and the material is naturally stacked into a conical shape by slowly adding the material. After the material stops sliding, a protractor is used to measure the included angle between the slope of the stockpile and the horizontal plane, and the measurement is repeated 3-5 times to obtain an average value, thereby obtaining the actual repose angle range of the material. For example, the repose angle of coal is usually between 25 and 40 degrees.

[0055] Subsequently, the statics law is converted into an executable constraint condition. When data is complemented, involving reconstruction of a stockpile shape, for each candidate complement point on the surface of the stockpile, a height difference and a horizontal distance between the candidate complement point and an adjacent known data point are calculated. A slope angle of a local slope where the candidate complement point is located is obtained by trigonometric conversion. The slope angle is compared with the pre-determined repose angle range. If the slope angle exceeds the upper limit of the repose angle, it is determined that the candidate complement point violates the gravity flow law, and the coordinates of the candidate complement point need to be adjusted to meet the constraint requirement. In this way, a suspended singular shape that violates the gravity law is avoided, and it is ensured that the geometry of the stockpile conforms to the actual statics characteristics.

[0056] When the second implicit physical constraint is constructed, a Fourier heat conduction law in heat conduction theory is relied on, and constraint design is implemented in combination with thermal imaging data of a stockpile. First, temperature data of a known region of a target stockpile is obtained by a thermal imaging lens, and the spatial coordinates and corresponding temperature values of the known temperature points are determined. A thermal conductivity coefficient of a target material is queried, for example, the thermal conductivity coefficient of coal is usually between 0.2 and 0.5 watts per meter kelvin, and the coefficient can be directly obtained from a material property manual well known to those skilled in the art.

[0057] Then, when the temperature data is complemented, the known temperature points are taken as a basis, a linear interpolation method combined with a local temperature gradient calculation method is used to determine the temperature values of the complemented region according to the law that the temperature change rate is proportional to the thermal conductivity coefficient in the Fourier heat conduction law. The temperature gradient between adjacent known temperature points is calculated, and the temperature values of the complemented region are distributed according to the gradient proportion based on the distance between the complemented point and the known point, so as to ensure that the temperature change of the complemented region is continuous and smooth, and sudden temperature jumps do not occur. The above-mentioned method in combination with the thermal imaging data can ensure that the complemented data is not only complete in geometry, but also conforms to the physical law of heat conduction in temperature distribution, thereby ensuring the physical credibility of the data.

[0058] After adding the first implicit physical constraint and the second implicit physical constraint into the overall implicit physical constraint, the two physical laws are respectively converted into mathematical expressions. For example, the first constraint is converted into an inequality constraint that the slope angle of any local slope on the surface of the material pile does not exceed the upper limit of the material repose angle, and the second constraint is converted into an inequality constraint that the temperature gradient in the completed area does not exceed the maximum reasonable gradient calculated based on the thermal conductivity coefficient.

[0059] Subsequently, the two mathematical constraint expressions are integrated into the constraint term of the diffusion completion unit. Specifically, a penalty term can be added to the objective function of the unit to achieve this. When the data generated by the unit violates any constraint, the penalty term will generate a large penalty value, increasing the overall value of the objective function; on the contrary, if the data meets the constraint, the penalty term value is small. In this way, the unit automatically prioritizes meeting the above two physical constraints during training or data generation, thereby incorporating the first and second implicit physical constraints into the implicit physical constraint system.

[0060] By determining the repose angle through the stacking experiment and converting it into a slope constraint, combining the Fourier law with the thermal imaging data to construct a temperature constraint, and integrating the physical law into a mathematical constraint term, the method achieves the effect of providing implicit physical constraints that meet the laws of statics and heat conduction for data completion, ensuring that the completed data meets both the geometric shape of the material pile and the physical characteristics of the temperature field.

[0061] Further, the method provided by the embodiments of the present application comprises:

[0062] The constraint threshold is built based on the semantic description constraint of the prompting engineering and the implicit physical constraint. The data completion layer is built through point cloud texture iterative generation under random noise completion. The completion architecture is constructed according to the constraint threshold and the data completion layer, and the diffusion completion unit is generated through iterative training to convergence.

[0063] Specifically, when building the diffusion completion unit, the multi-modal input data needs to be processed first. The multi-modal data of laser point cloud, thermal imaging temperature measurement and image are standardized: the coordinate normalization method is used for laser point cloud data to map the X, Y and Z coordinates of all points to the [0, 1] interval, eliminating the scale difference caused by different collection distances; the thermal imaging temperature measurement data are linearly normalized according to the material temperature measurement range-20°C-150°C, converted into values of the same order of magnitude; the image data are adjusted to the standard range of [0, 255] by gray scale normalization, while retaining the texture details. After completing the preprocessing, the multi-modal data are associated according to the collection time stamp and spatial position to form structured input data, providing a unified data basis for subsequent constraint fusion and completion layer building.

[0064] In the process of building constraint threshold, first, the semantic description constraint is constructed based on the prompt engineering, and the rule type prompt word generation method is adopted: according to the known information of the neighborhood of the triangular mesh, such as the type of the surrounding material, the surface texture characteristics, etc., a semantic template is preset, for example, the triangular mesh is adjacent to the coal pile area, the surface needs to present the coal texture, and the slope meets the statics law, a one-to-one corresponding context semantic prompt word is generated for each triangular mesh, that is, the first semantic entry, which will be described in detail in the subsequent content.

[0065] Then, for the implicit physical constraint, the first and second implicit physical constraints constructed in the foregoing steps are converted into inequality constraints: an inequality of local inclined slope angle ≤ upper limit of angle of repose is constructed, and an inequality of completed area temperature gradient ≤ maximum reasonable temperature gradient is constructed. The semantic description constraint and the two inequality constraints are integrated, the constraint threshold is built through the logic judgment module, and it is ensured that the generated data in the subsequent process meets the semantic matching and physical law at the same time.

[0066] In the process of building the data completion layer, first, Gaussian random noise with a mean of 0 and a variance of 0.1 is generated based on the number of vertices of the triangular mesh and the spatial dimension, which is used as the initial data for point cloud texture generation. Then, the point cloud texture iterative generation is performed, and the texture characteristics of the known neighborhood grid are referred to in each iteration, including image gray distribution and laser point cloud density. The weighted average method is used to adjust the noise data: for example, if a triangular mesh is adjacent to 3 known grids, the texture parameters (such as gray mean) of these 3 grids are taken and the average value is calculated according to the distance weight, that is, the closer the distance, the greater the weight. The texture parameters of the current noise data are adjusted to make the generated texture gradually approach the actual stockpile characteristics. After each iteration, the generated texture data is matched with the semantic description constraint in the constraint threshold for matching degree detection, and the texture data with a matching degree higher than 80% is retained. After 6-8 iterations, the data completion layer that meets the semantic requirements is formed.

[0067] After that, in the process of building the completion architecture and performing iterative training, the hierarchical architecture design and gradient descent training method are adopted: the constraint threshold is used as the upper control module, and the data completion layer is used as the lower execution module to build a two-layer completion architecture. The two are connected through a data interaction interface, so that the data generated by the data completion layer can be transmitted to the constraint threshold module in real time for compliance detection.

[0068] In the process of designing the objective function, the reconstruction error term and the penalty term are combined, where the reconstruction error term is the Euclidean distance between the generated data and the known data, and the penalty term is set for the two inequality constraints in the implicit physical constraint. The basic penalty value is set to 0.5, if the generated data violates the local inclined slope angle ≤ upper limit of angle of repose, the basic penalty value is increased by 10 times; if it violates the temperature gradient ≤ maximum reasonable temperature gradient, the basic penalty value is increased by 8 times; if both constraints are met, the basic penalty value is increased by 0.1 times.

[0069] The gradient descent method is used in the training process, the target function value is calculated in each iteration, the weight coefficient of the data completion layer is adjusted through back propagation, when the change amount of the target function value of continuous 10 iterations is less than 0.001, it is determined that the training converges, the iteration is stopped and the final diffusion completion unit is generated.

[0070] Through the existing simple methods such as multi-modal data standardization, rule-based semantic prompt generation, physical constraint inequality transformation, Gaussian noise iterative optimization and gradient descent training with penalty term, the effect of constructing a diffusion completion unit that meets the semantic description and physical law and can stably generate reliable completion data is achieved.

[0071] Further, the method provided by the embodiment of the application comprises:

[0072] The marked stockyard source data is imported into the diffusion completion unit, context semantic prompt words are generated according to the known information of the neighborhood of the triangular mesh according to the constraint threshold, and the first semantic term is determined, wherein the first semantic term corresponds to the triangular mesh one by one; and the first semantic term and the implicit physical constraint are fused as a generation constraint condition.

[0073] In one embodiment, the marked stockyard source data is first imported into the diffusion completion unit. The marked stockyard source data is first subjected to format unification processing: the laser point cloud data retains its three-dimensional coordinate information and is converted into a floating point number format, the image data extracts texture features of the corresponding triangular mesh region including the gray mean value and texture direction and stores them in the form of a numerical array, and the thermal imaging temperature measurement data retains its temperature value. Subsequently, the three types of data are associated according to the unique triangular mesh ID of the triangular mesh and integrated into structured data in JSON format. The structured data is imported through the standard data interface of the diffusion completion unit to ensure that the diffusion completion unit can accurately identify the corresponding marked information and neighborhood data of each triangular mesh.

[0074] Then, according to the preset semantic description rule in the constraint threshold, a rule-based prompt word generation method is used to generate context semantic prompt words based on the known information of the neighborhood of the triangular mesh. First, the neighborhood known information is obtained through a basic feature extraction algorithm: the neighborhood point cloud data is used to calculate geometric parameters such as the slope and height difference of the adjacent mesh, the image texture similarity is determined by comparing the gray histogram, and it is judged whether there is a temperature mutation by counting the neighborhood temperature data. Subsequently, a preset structured text template is called, and an example of the template format is: fill in a slope region that is continuous with the left coal pile surface, similar in texture and has no sharp temperature mutation. Then, the extracted geometric, texture and temperature information is filled into the template one by one to generate the first semantic term for the triangular mesh. To ensure that the first semantic term corresponds to the triangular mesh one by one, the corresponding triangular mesh ID is added to each term during generation and stored in the semantic database of the unit.

[0075] After the first semantic term is generated, the implicit physical constraint is converted into a quantifiable conditional expression, that is, the inequality of the constructed local slope angle ≤ the upper limit of the angle of repose and the inequality of the completed regional temperature gradient ≤ the maximum reasonable temperature gradient. Then, the first semantic term is integrated with the two quantified physical constraint conditions by using a logical splicing method, for example, a slope region that is continuous with the left coal pile surface, similar in texture, and has no sharp temperature mutation is spliced with the local slope angle ≤ the upper limit of the angle of repose and the completed regional temperature gradient ≤ the maximum reasonable temperature gradient to form the final generation constraint condition of the triangular mesh, and the triangular mesh ID is associated to ensure that the constraint condition is accurately matched with the mesh.

[0076] Through the steps of data format standardization and interface import, rule-based prompt word generation and feature extraction, physical constraint quantization and logical splicing, the generation constraint condition with semantic guidance and physical rationality for each triangular mesh is generated, and the effect of subsequent data completion conforming to the actual scene characteristics of the stockyard is ensured.

[0077] Further, the method provided in the embodiments of the present application comprises:

[0078] For the first triangular mesh, first grid generation data is determined by using random noise, wherein the first triangular mesh is any triangular mesh divided based on a data missing boundary; second grid generation data is determined by denoising generation of the first grid generation data guided by the generation constraint condition; and Nth grid generation data is determined by denoising generation guided by the generation constraint condition based on the second grid generation data, and multiple iterations are performed until the iteration converges based on boundary smooth transition; and the Nth grid generation data is taken as the data completion result of the first triangular mesh.

[0079] Optionally, first, one of the generated triangular meshes is selected as the first triangular mesh for subsequent processing according to the triangular mesh constructed in the foregoing steps. For the determined first triangular mesh, first grid generation data is determined by using a Gaussian random noise generation method. First, the number of vertices of the first triangular mesh is counted. If the triangular mesh contains 3 vertices, 3 sets of coordinate noise are generated. The mean value of the Gaussian random noise parameter is set to 0, and the variance is set to 0.1. Three-dimensional coordinate noise data matching the number of vertices are generated by using a random number generator, and each vertex corresponds to a set of noise coordinates. These noise coordinates are combined to form the first grid generation data, which is used as the initial input for subsequent denoising iteration.

[0080] Next, the key constraint terms in the generated constraint conditions are extracted, such as the local inclined plane slope angle ≤ the upper limit of the angle of repose, the temperature gradient of the completed area ≤ the maximum reasonable temperature gradient, and the texture is similar to that of the left coal pile. Each vertex coordinate of the first mesh generation data is compared with the constraint terms one by one. For the vertices that violate the constraints, the vertex coordinates of the known mesh in the neighborhood are used as the reference, and the self-noise coordinates are given a lower weight. The adjusted vertex coordinates are calculated by the weighted average formula: reference coordinate × weight + noise coordinate × weight. For the vertices that meet the constraints, only the weight proportion is slightly adjusted to reduce unnecessary modifications. All the adjusted vertex coordinates are integrated to obtain the second mesh generation data.

[0081] In the subsequent multiple rounds of iteration denoising based on the second mesh generation data, the weighted average denoising method of the above steps is used in each round of iteration, and the determination accuracy of the constraint terms is gradually improved. For example, the first round of iteration only determines whether the constraint is seriously violated, such as a slope exceeding 40°. The subsequent iterations gradually reduce the determination threshold, such as determining whether the slope exceeds 36° in the third round, whether it exceeds 35° in the fifth round, and so on. At the same time, the distance between the boundary vertices of the current mesh and the known mesh in the neighborhood is determined after each iteration. If all the boundary vertex distances are less than a preset threshold, such as 0.5 cm, it is determined that the boundary is smoothly transitioned, meeting the iteration convergence condition, stopping the iteration, and determining the Nth mesh generation data. If not, repeat the steps of weighted average denoising based on the current mesh data until convergence.

[0082] Finally, the Nth mesh generation data is subjected to a final check by comparing its compliance with the generated constraint conditions, i.e., the local inclined plane slope angle ≤ the upper limit of the angle of repose, the temperature gradient of the completed area ≤ the maximum reasonable temperature gradient, and the texture is consistent with the neighborhood. After confirming that there is no violation of the constraints, the data completion result of the first triangular mesh is formally determined.

[0083] Through the steps of generating initial data with Gaussian random noise, weighted average denoising combined with constraint determination, and boundary distance judgment for convergence, the reliable data completion result of the first triangular mesh based on the data missing boundary division that meets the constraint conditions is achieved.

[0084] Further, the method provided by the embodiments of the present application comprises:

[0085] Based on the triangular mesh adjacent to the data missing boundary, a set of triangular meshes generated in one step is determined. The first generation constraint condition determination based on the constraint threshold and the diffusion iterative generation based on the data completion layer are performed to determine a set of data completion results. The data missing boundary is updated based on the set of data completion results as known data, and two sets of triangular mesh parts generated in two steps are determined according to the updated data missing boundary, and the internal shrinkage diffusion type supplement is performed.

[0086] In one embodiment, a set of triangular mesh parts generated in one step is first determined, the data missing boundary marked in the foregoing step is extracted, and the vertex coordinates of all triangular meshes on the boundary are obtained. Then, the shortest distance of each triangular mesh to the data missing boundary is calculated, and the triangular mesh with a distance of zero, i.e., the vertex coincides with the vertex of the data missing boundary, is screened out to determine the triangular mesh adjacent to the data missing boundary. Then, according to the spatial distribution of the triangular mesh, the adjacent meshes are grouped in a clockwise or counterclockwise order to form a set of triangular mesh parts generated in one step, ensuring that the meshes in the set are all the outermost meshes of the missing area, laying a foundation for subsequent progressive filling from the outermost circle to the innermost circle.

[0087] Next, the triangular mesh is executed to generate the constraint condition determination and diffusion iteration generation. First, according to the constraint threshold, the known information of the neighborhood of the mesh is extracted, such as the geometric parameters, texture features, and temperature data of the surrounding completed meshes, and the corresponding first semantic term is generated for each mesh through the rule-based prompt word generation method, and then the implicit physical constraint is fused to determine the first generation constraint condition of each mesh. Then, the diffusion iteration generation function of the data filling layer is called to start with the random noise of the triangular mesh, and perform multiple rounds of weighted average denoising according to the generation constraint condition. After each iteration, the boundary smoothness of the mesh and the neighborhood known data is detected until the convergence condition is met, and finally the data filling result of the set of triangular meshes is determined.

[0088] Then, a set of data filling results is updated as known data part of the missing boundary. First, the mesh data of the generated data filling result is marked as known data and included in the known database of the stockyard source data, and then the neighborhood point density detection is performed on the updated stockyard source data. The same neighborhood radius and neighborhood data point number as in the initial detection in the foregoing step are set, the neighborhood point density values of the surrounding data points of the remaining unfilled area are calculated, and the data points with density values lower than the density threshold are screened out to form a new data missing boundary, and the boundary update is completed. The same spatial distance judgment method as in the first step is used to determine two sets of triangular mesh parts generated in two steps from the adjacent meshes of the updated missing boundary. The part meshes are closer to the inside of the missing area than the set of meshes in the first step, and the filling range is realized to be shrunk inward.

[0089] Subsequently, for the inward-expanding supplementation of the second and subsequent sets of triangular meshes, the above constraint determination and diffusion iteration generation steps are repeated: for each new set of inner triangular meshes, specific constraints are first generated based on the known data of its neighborhood (including the mesh data already supplemented in the previous step), and then the supplementation result is generated iteratively through the data supplementation layer. After that, the missing boundary is updated and the next set of inner meshes is determined. This process is repeated until all triangular meshes in the missing area are supplemented, that is, the data supplementation result generated by the last set of triangular meshes achieves a smooth boundary transition with all surrounding known data, and there are no remaining unsupplemented meshes, thus completing the supplementation of the entire missing area. By filling all triangular meshes accordingly, the supplementary data for the material yard is determined.

[0090] By employing existing simple methods such as spatial adjacency detection grouping, regular constraint generation and weighted average denoising, neighborhood point density detection to update boundaries, and cyclic shrinking supplementation, the system achieves the effect of progressively filling large missing regions from the outer circle to the inner circle, ensuring that the data is complete and meets the constraints after all triangular meshes are stitched together.

[0091] In summary, the multi-source data fusion and collaborative supplementation method for intelligent material yards provided in this application has the following technical effects:

[0092] This application utilizes a fusion device consisting of LiDAR, thermal imaging lens, rotating gimbal, and visible light lens deployed in a digitally intelligent material yard area to collect multi-source datasets including laser point clouds, thermal imaging temperature measurement, and images. Through spatial 3D alignment, neighborhood point density detection to locate missing data boundaries, and triangular mesh segmentation and marking, the application obtains the material yard source data and missing area marking information. Using multimodal data as input, a diffusion-based completion unit is constructed by combining semantic description constraints and implicit physical constraints based on cueing engineering. The marked material yard source data is imported into the unit to determine the generation constraints. Starting with random noise from the triangular mesh, a multi-round iterative denoising process guided by constraints is executed. Combined with an inward-diffusion-style completion adjustment process, this effectively solves the problem of missing laser measurement data caused by water surface reflection, dead-end pits, or object obstruction in the material yard. This provides reliable data support for the digital modeling of smart material yards, making the multi-source data fusion completion results of the digitally intelligent material yard more accurate and reliable. It achieves the technical effect of integrating multi-source data and accurately completing missing data, forming complete and reliable material yard data.

[0093] Example 2, as Figure 2 As shown, based on the same inventive concept as in Embodiment 1 above, this application provides a multi-source data fusion and collaborative supplementation system for intelligent material yards, the system comprising:

[0094] Material yard source data acquisition module 1 is used to collect multi-source datasets of the target material yard, perform spatial three-dimensional alignment, and use them as material yard source data.

[0095] The triangular mesh segmentation execution module 2 is configured to perform the following steps:

[0096] The diffusion inpainting unit construction module 3 is configured to perform the following steps:

[0097] The stockyard inpainting data acquisition module 4 is configured to perform the following steps:

[0098] Further, the triangular mesh segmentation execution module 2 is configured to perform the following steps:

[0099] Further, the triangular mesh segmentation execution module 2 is configured to perform the following steps:

[0100] Further, the diffusion inpainting unit construction module 3 is configured to perform the following steps:

[0101] Further, the diffusion inpainting unit construction module 3 is configured to perform the following steps:

[0102] Further, the diffusion inpainting unit construction module 3 is configured to perform the following steps:

[0103] Further, the diffusion inpainting unit construction module 3 is configured to perform the following steps:

[0104] Further, the stockyard inpainting data acquisition module 4 is configured to perform the following steps:

[0105] The labeled stockyard source data is imported into the diffusion completion unit, and according to the constraint threshold, the context semantic prompt word is generated based on the known information of the triangular mesh neighborhood, and the first semantic word is determined, wherein the first semantic word corresponds to the triangular mesh one by one; the first semantic word and the implicit physical constraint are fused as a generated constraint condition.

[0106] Further, the stockyard completion data acquisition module 4 is used to execute the following steps:

[0107] For the first triangular mesh, a first grid generation data is determined based on random noise, wherein the first triangular mesh is any triangular mesh based on data missing boundary division; the first grid generation data is denoised based on the generated constraint condition, and a second grid generation data is determined; based on the second grid generation data, denoising generation guided by the generated constraint condition is performed, and multiple iterations are performed until the Nth grid generation data is determined, wherein the boundary smooth transition is used as the iteration convergence condition; the Nth grid generation data is used as the data completion result of the first triangular mesh.

[0108] Further, the stockyard completion data acquisition module 4 is used to execute the following steps:

[0109] Based on the triangular mesh adjacent to the data missing boundary, a set of triangular mesh parts generated in one step is determined; based on the constraint threshold, a first generated constraint condition is determined for the set of triangular meshes, and based on the diffusion iterative generation of the data completion layer, a set of data completion results is determined; the set of data completion results is used as known data part, and the data missing boundary is updated, and based on the updated data missing boundary, two sets of triangular mesh parts generated in two steps are determined, and the internal shrinkage diffusion type supplement is performed.

[0110] Further, the stockyard source data acquisition module 1 is used to execute the following steps:

[0111] According to the fusion device, wherein the fusion device is composed of a laser radar, a thermal imaging lens, a rotating cloud platform and a visible light lens, and the rotating cloud platform performs multi-space angle rotation of the components; the fusion device is deployed to the stockyard area to perform multi-source data joint acquisition, and the multi-source data set is obtained.

[0112] Further, the stockyard source data acquisition module 1 is used to execute the following steps:

[0113] The multi-source data set includes laser point cloud data, thermal imaging temperature measurement data and image data based on the component coordinate system; a unified coordinate system is determined; the multi-source data set is subjected to data coordinate conversion and phase alignment based on the space coordinate phase conversion from the component coordinate system to the unified coordinate system, and the stockyard source data is obtained.

[0114] The multi-source data fusion collaborative supplement system of the digitized stockyard provided by the embodiments of the present application can execute the multi-source data fusion collaborative supplement method of the digitized stockyard provided by any of the embodiments of the present application, and has the function modules and beneficial effects corresponding to the execution method.

[0115] Although the present application makes various references to certain modules in the system according to the embodiments of the present application, however, any number of different modules can be used and run on the user terminal and / or server, and each unit and module included is only divided according to the functional logic, but is not limited to the above division, as long as the corresponding functions can be realized; in addition, the specific names of each functional unit are only for the convenience of mutual differentiation, and do not limit the protection scope of the present application.

[0116] The above specific embodiments do not constitute a limitation on the protection scope of the present application. Those skilled in the art should understand that various modifications, combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present application shall be included in the protection scope of the present application. In some cases, the actions or steps recorded in the present application can be executed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multi-task processing and parallel processing are possible or can be advantageous.

Claims

1. A method for multi-source data fusion and collaborative supplementation in a digitalized material yard, characterized in that, The method includes: Collect multi-source datasets of the target material yard, perform spatial 3D alignment, and use them as the source data for the material yard; The material yard source data is subjected to neighborhood point density detection, missing data boundaries are spliced ​​and triangular mesh segmentation is performed, and the material yard source data is marked. Using multimodal data as input, and semantic description constraints and implicit physical constraints based on prompting engineering, a diffusion completion unit is constructed. The marked material yard source data is imported into the diffusion completion unit, the generation constraints are determined, and multi-round iterative denoising under constraint guidance is performed starting with random noise based on triangular mesh to generate data completion results. The triangular mesh is filled accordingly to determine the material yard completion data. The implicit physical constraints include: The static law of the material repose angle based on the geometry of the stockpile is used as the first implicit physical constraint; The heat conduction law based on thermal diffusion under the surface temperature field is used as the second implicit physical constraint; Add the first implicit physical constraint and the second implicit physical constraint into the implicit physical constraint; The construction of the diffusion completion unit includes: Constraint thresholds are constructed using semantic description constraints and implicit physical constraints based on prompting engineering. A data completion layer is constructed by iteratively generating point cloud textures under random noise completion; Based on the constraint threshold and data completion layer, a completion architecture is constructed, and the diffusion completion unit is generated through iterative training until convergence.

2. The multi-source data fusion and collaborative supplementation method for intelligent data storage yards as described in claim 1, characterized in that, The neighborhood point density of the material yard source data is detected, and missing data boundaries are stitched together, including: Set neighborhood conditions, wherein the neighborhood conditions are a pre-defined neighborhood radius and the number of neighborhood data points; For the aforementioned material yard source data, calculate the neighborhood point density value for each data point based on the neighborhood conditions; Determine whether the density value of each neighboring point meets the density threshold, and if so, determine that the data point is missing and use it as the missing boundary data point. The missing data boundaries are formed based on the missing boundary data points.

3. The multi-source data fusion and collaborative supplementation method for intelligent data storage yards as described in claim 1, characterized in that, Determine the generation constraints, including: The marked material source data is imported into the diffusion completion unit. Based on the constraint threshold, contextual semantic prompts are generated using the known neighborhood information of the triangular mesh to determine the first semantic term, wherein the first semantic term corresponds one-to-one with the triangular mesh. The first semantic term and implicit physical constraints are combined to form the generation constraint conditions.

4. The multi-source data fusion and collaborative supplementation method for intelligent data storage yards as described in claim 3, characterized in that, Starting with random noise based on triangular meshes, a multi-round iterative denoising process guided by constraints is performed to generate data completion results, including: For the first triangular mesh, random noise is used to determine the data generated by the first mesh, wherein the first triangular mesh is any triangular mesh divided based on the data missing boundary; Guided by the aforementioned generation constraints, the first grid generation data is denoised and generated to determine the second grid generation data; Based on the data generated by the second grid, a denoising generation guided by the generation constraints is performed, and multiple iterations are carried out until the data generated by the Nth grid is determined, wherein the smooth transition of the boundary is used as the convergence condition of the iteration. The data generated by the Nth grid is used as the data completion result of the first triangular grid.

5. The multi-source data fusion and collaborative supplementation method for intelligent data storage yards as described in claim 4, characterized in that, Generate data completion results, including: A set of triangular meshes generated in one step is determined based on the triangular meshes adjacent to the missing data boundaries; The first generation constraint condition based on the constraint threshold is determined for the set of triangular meshes, and a set of data completion results are determined by diffusion iteration based on the data completion layer. The data completion result is used as the known data part. The missing data boundary is updated. Based on the updated missing data boundary, the two sets of triangular mesh parts generated in the second step are determined, and inward diffusion-type completion is performed.

6. The multi-source data fusion and collaborative supplementation method for intelligent data storage yards as described in claim 1, characterized in that, Collect multi-source datasets from the target material yard, including: According to the fusion device, the fusion device consists of a lidar, a thermal imaging lens, a rotating gimbal, and a visible light lens, wherein the rotating gimbal performs multi-spatial angle rotation of the components; The fusion device is deployed to the material yard area to perform multi-source data joint acquisition and obtain the multi-source dataset.

7. The multi-source data fusion and collaborative supplementation method for intelligent material yards as described in claim 6, characterized in that, Perform spatial 3D alignment as source data for the material yard, including: The multi-source dataset includes laser point cloud data, thermal imaging temperature measurement data, and image data based on the component coordinate system; Establish a unified coordinate system; The multi-source dataset is transformed and phase-aligned by spatial coordinate phase transformation from the component coordinate system to the unified coordinate system, and is used as the material yard source data.

8. A multi-source data fusion and collaborative supplementation system for intelligent material yards, characterized in that, The system is used to implement the multi-source data fusion and collaborative supplementation method for intelligent material yards according to any one of claims 1-7, the system comprising: The material yard source data acquisition module is used to collect multi-source datasets of the target material yard, perform spatial three-dimensional alignment, and use them as material yard source data. The triangular mesh segmentation execution module is used to perform neighborhood point density detection on the material yard source data, splice missing data boundaries and perform triangular mesh segmentation, and mark the material yard source data. The diffusion completion unit construction module is used to build diffusion completion units by taking multimodal data as input and using semantic description constraints and implicit physical constraints based on prompting engineering. The material yard completion data acquisition module is used to import the marked material yard source data into the diffusion completion unit, determine the generation constraints, start with random noise based on triangular mesh, perform multi-round iterative denoising under constraint guidance, generate data completion results, fill the triangular mesh accordingly, and determine the material yard completion data.

Images