An industrial pile identification method, system, device and storage medium

Abstract

The present application relates to the technical field of computer vision, and relates to an industrial material pile identification method, system, device and storage medium; wherein the industrial material pile identification method comprises: obtaining material pile data to be identified; inputting the material pile data into a material pile type judgment model, the material pile type judgment model processing the material pile data according to a preset rule, and outputting a material pile type identification result; wherein the processing steps of the material pile type judgment model on the material pile data are: performing feature extraction on radar point cloud data to obtain point cloud features; performing feature extraction on pile state text description data to obtain semantic features; fusing the point cloud features and the semantic features to obtain fused features; analyzing environment data to determine an environment influence weight, and adjusting the fused features according to the environment influence weight to obtain environment perception fused features; and determining the material pile type identification result according to the environment perception fused features. The present application can improve the accuracy of the material pile type identification result.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision, and particularly relates to an industrial pile identification method, system, device and storage medium. BACKGROUND

[0002] In the industrial scenes of metallurgy, mining, building materials, etc., the dynamic management of piles (such as ores, coals, sandstones, etc.) faces many challenges. First, if the parameters such as the stacking height and the inclination angle of the piles exceed the specified range, serious collapse accidents may be caused, bringing huge safety hazards. Second, the traditional manual inspection method is difficult to realize the real-time acquisition of the three-dimensional form parameters (such as the volume, density distribution, etc.) of the piles, thereby causing the efficiency bottleneck, especially in large-scale industrial production, it is difficult to ensure timely and accurate state feedback, and it is difficult to form an automated industrial operation process. In addition, the environmental interference factors of the industrial site, such as light change, dust shielding, etc., often greatly affect the data reliability of a single sensor, further aggravating the management and monitoring difficulty.

[0003] Currently, stockpile recognition technology mainly relies on three types of solutions: point cloud-based solutions, vision-based solutions, and fusion solutions of point clouds and images. However, these three types of technology routes have certain limitations, and a complete framework for industrial practical applications has not yet been established. The first type is a point cloud-based processing solution that relies on three-dimensional sensors such as radars or depth cameras, and uses point cloud processing algorithms such as PointNet++ to obtain the three-dimensional geometric structure features of the stockpile. Although it has advantages in spatial modeling, it lacks high-level semantic modeling capabilities and cannot effectively recognize semantic layer information such as material type and stacking state. The second type is a two-dimensional image-based vision solution, which typically includes the application of Convolutional Neural Network (CNN) in image recognition tasks. This solution can identify the appearance and surface texture of the stockpile, but is limited by camera angles, lighting conditions, and occlusion factors, making it difficult to accurately estimate three-dimensional indicators such as stockpile volume and spatial distribution. The structure of the fuel pile is often three-dimensional, and the stacking form is complex and diverse. Traditional image processing methods cannot fully express these three-dimensional structures, especially when dealing with point cloud data and complex spatial relationships. This method ignores the three-dimensional geometric information of the stockpile, making the model difficult to maintain high accuracy and stability in complex environments. The third type is a fusion solution of point clouds and images, which attempts to use sensor registration and multi-channel feature splicing to jointly model the stockpile state. Although this method improves the recognition accuracy to some extent, it is limited by the alignment accuracy of heterogeneous data, the limitations of feature fusion strategies, and the lack of task-level semantic guidance, making its performance unstable in industrial dynamic scenarios. In addition, many existing stockpile stacking judgment methods rely on manual feature extraction and traditional machine learning algorithms such as Support Vector Machine (SVM) and decision trees. These methods often require manual analysis and feature design of the stockpile form, but the extraction of manual features is not only time-consuming but also difficult to capture complex patterns in the data. Especially when faced with a large amount of high-dimensional data, traditional machine learning methods are insufficient in terms of accuracy and adaptability. As the amount of data increases, traditional methods often struggle to handle more complex stacking classification tasks, limiting their effectiveness and efficiency in large-scale practical applications.

[0004] More critically, existing technologies generally lack the introduction of Large Language Models (LLMs) in multi-modal processing architectures, failing to support natural language-based human-machine interactive task feedback and responses. Especially in industrial complex scenarios that require integrated decision-making based on structure recognition and state understanding, the lack of explainable and schedulable language modeling methods severely hinders the extension of stockpile recognition technology towards "intelligent control". SUMMARY

[0005] The present application aims to at least solve the technical problems existing in the prior art, and provides an industrial pile identification method, system, device and storage medium.

[0006] In a first aspect, the present application provides an industrial pile identification method, comprising:

[0007] Obtaining pile data to be identified, the pile data comprising radar point cloud data of the pile, text description data of a stacking state and environment data;

[0008] Inputting the pile data into a pile type judgment model, the pile type judgment model processing the pile data according to a preset rule and outputting a pile type identification result;

[0009] The processing steps of the pile type judgment model on the pile data are:

[0010] Performing feature extraction on the radar point cloud data to obtain point cloud features;

[0011] Performing feature extraction on the text description data of the stacking state to obtain semantic features;

[0012] Fusing the point cloud features and the semantic features to obtain fused features;

[0013] Analyzing the environment data to determine an environment influence weight, and adjusting the fused features according to the environment influence weight to obtain environment perception fused features;

[0014] Determining the pile type identification result according to the environment perception fused features.

[0015] In a second aspect, the present application provides an industrial pile identification system, comprising:

[0016] An acquisition module, configured to acquire pile data to be identified, the pile data comprising radar point cloud data of the pile, text description data of a stacking state and environment data;

[0017] A processing module, configured to input the pile data into a pile type judgment model, the pile type judgment model processing the pile data according to a preset rule and outputting a pile type identification result;

[0018] The processing steps of the pile type judgment model on the pile data are:

[0019] Performing feature extraction on the radar point cloud data to obtain point cloud features;

[0020] Performing feature extraction on the text description data of the stacking state to obtain semantic features;

[0021] Fusing the point cloud features and the semantic features to obtain fused features;

[0022] The environmental data is analyzed to determine an environmental influence weight, and a fusion feature is adjusted according to the environmental influence weight, to obtain an environmental perception fusion feature;

[0023] A stockpile type identification result is determined according to the environmental perception fusion feature.

[0024] In a third aspect, the present application provides an electronic device, which comprises:

[0025] at least one processor; and,

[0026] a memory connected to the at least one processor in communication; wherein,

[0027] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to execute the industrial stockpile identification method described above.

[0028] In a fourth aspect, the present application further provides a computer readable storage medium, which stores at least one computer program, and the at least one computer program is executed by a processor in an electronic device to implement the industrial stockpile identification method described above.

[0029] In summary, the present application has the following beneficial technical effects:

[0030] The stockpile data is identified and judged by the stockpile type judgment model, the stockpile type judgment model comprehensively considers the point cloud feature, the semantic feature and the environmental data of the stockpile data, determines the environmental influence weight according to the environmental data, adjusts the fusion proportion of the point cloud feature and the semantic feature according to the environmental influence weight, the fusion feature includes the influence information of the environmental change on the stability of the stockpile, and the adaptability of the stockpile type judgment model in the complex changing environment can be enhanced, and the accuracy of the stockpile type judgment can be improved. BRIEF DESCRIPTION OF DRAWINGS

[0031] Figure 1 A flowchart of an industrial stockpile identification method provided by an embodiment of the present application is shown;

[0032] Figure 2 A module diagram of a stockpile type judgment model provided by an embodiment of the present application is shown;

[0033] Figure 3 An internal processing flowchart of an environmental driving feature regulation module provided by an embodiment of the present application is shown;

[0034] Figure 4 A structural diagram of an electronic device for implementing the industrial stockpile identification method provided by an embodiment of the present application is shown.

[0035] Reference signs: 10, processor; 11, memory; 12, communication bus; 13, communication interface.

[0036] The implementation, functional features and advantages of the present application will be further described with reference to the embodiments, with reference to the accompanying drawings. DETAILED DESCRIPTION

[0037] Embodiments of the present application are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference signs represent the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be understood as a limitation of the present application.

[0038] In the description of the present application, it should be understood that the terms "longitudinal", "transverse", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer" and the like indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and therefore cannot be understood as indicating or implying that the device or element referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation of the present application.

[0039] In the description of the present application, unless otherwise specified and limited, it should be noted that the terms "mounting", "connection", "connection" should be understood broadly, for example, it can be mechanical connection or electrical connection, it can be the communication between the two elements, it can be direct connection, or indirect connection through intermediate medium, and the specific meaning of the above terms can be understood by those skilled in the art according to the specific circumstances.

[0040] Reference Figure 1 As shown in the flowchart of the industrial pile identification method provided by an embodiment of the present application. In this embodiment, the industrial pile identification method comprises:

[0041] S1, obtaining pile data to be identified.

[0042] In an industrial scene, the industrial pile includes ore, coal, sand, metallurgical slag, fuel slag, glass slag, papermaking slag and ceramic slag, etc. The pile data includes radar point cloud data, stacking state text description data and environment data of the pile.

[0043] The radar point cloud data of the material pile is collected by a laser radar device (LiDAR), which can provide high-precision three-dimensional spatial point cloud data. Specifically, the point cloud data collected by the laser radar device is an N x 3 matrix, where N is the number of points, and each row represents the three-dimensional coordinate (x, y, z) data format of a point. In addition, the laser radar device may also be accompanied by intensity or point cloud color information, which helps to further analyze the quality of the point cloud data and the material characteristics of the object.

[0044] In order to remove irrelevant data and noise, a Voxel Grid Filter is used to downsample the point cloud data; this method can effectively reduce the amount of data and preserve the geometric shape features of the point cloud. After that, the downsampled point cloud data is standardized to obtain the final radar point cloud data. The specific method of standardizing the downsampled point cloud data is: the three-dimensional coordinates of each point are subtracted by the mean value and divided by the standard deviation to ensure that the data is within a unified scale, avoiding the influence of large-scale coordinate differences on subsequent processing. The final radar point cloud data is obtained through the point cloud data.

[0045] The stacking state text description data is used to represent the natural language description of the stacking state of the material pile; the stacking state text description data includes at least one of the type of the material pile, the stacking requirements, the environmental conditions, etc. The stacking state text description data can be derived from the standard documents of the material pile, the stacking requirements, the stacking safety specifications, etc.

[0046] After collecting the text description corresponding to the stacking state text description data, first, use a natural language processing toolkit (such as NLTK or SpaCy) to perform word segmentation on the collected text description, remove stop words (such as meaningless words such as "of", "and", etc.), and other preprocessing. The English full name of the natural language processing toolkit NLTK is Natural Language Toolkit, LTK is a widely used Python library that provides a wealth of natural language processing tools and resources, including part-of-speech tagging, syntax analysis, semantic analysis, etc. Its advantages are rich in functions, detailed documentation, and good community support; the natural language processing toolkit SpaCy is a powerful NLP library (Natural Language Processing), which provides high-performance natural language processing tools and models, including tokenization, word vector representation, named entity recognition, etc.

[0047] Then, the preprocessed text description is encoded using a BERT (Bidirectional Encoder Representations from Transformers) pre-training model. The BERT model can generate fixed-dimension vector representations that can capture semantic information in the description as input for subsequent processing. The embedding vectors of the description data are processed by the BERT to obtain fixed-dimension vectors. To maintain the uniformity of the data, L2 normalization is performed on these fixed-dimension vectors to obtain the final stacked state text description data, ensuring that the text descriptions in the stacked state text description data are on the same scale.

[0048] The environmental data includes at least one of temperature data, humidity data, and vibration data, and the environmental data is collected by a plurality of sensors (such as temperature sensors, humidity sensors, vibration sensors, etc.) in real time to obtain the environmental data of the material pile storage area; the influence of the industrial material pile storage environment on the material pile shape is determined according to the data of the temperature sensor, the humidity sensor, and / or the vibration sensor, and the influence of the environmental data on the material pile shape is fully considered in the subsequent identification of the material pile shape, so that the subsequent material pile shape identification result is more accurate.

[0049] S2, input the material pile data into the material pile shape judgment model, and the material pile shape judgment model processes the material pile data according to a preset rule to output a material pile shape identification result.

[0050] In this embodiment, the material pile shape judgment model is based on the architecture of the GPT4Point model for innovation and improvement. The material pile shape judgment model is a multi-modal learning model used for material pile shape judgment tasks. GPT4Point is a point cloud multi-modal large model. This model can complete 3D object recognition, understanding, and question answering tasks by only inputting point clouds without the aid of pictures. It combines the understanding and generation of point clouds, enabling multi-modal large models to have control capabilities for 3D generation.

[0051] Referring to Figure 2 , the material pile shape judgment model includes a point cloud feature extraction module, a description feature extraction module, a multi-modal fusion module, and an environmental driving feature regulation module. The point cloud feature extraction module extracts the geometric features of the point cloud data through a Transformer network and a multi-scale convolutional network to obtain point cloud features. The description feature extraction module processes the description information through a BERT model to obtain semantic features. The multi-modal fusion module is used to fuse the point cloud features and the semantic features to obtain multi-modal fusion features. In this way, the model can consider both the point cloud features and the description information when judging the material pile shape. Finally, the environmental driving feature regulation module dynamically adjusts the point cloud features by collecting environmental data to adapt to the influence of environmental changes on the material pile judgment.

[0052] Specifically, the processing steps of the stockpile shape judgment model on the stockpile data are as follows:

[0053] S21, feature extraction is performed on the radar point cloud data to obtain point cloud features.

[0054] The point cloud feature extraction module mainly includes a multi-layer Transformer encoder and a multi-scale convolutional network. In the preferred embodiment of the present embodiment, the specific processing manner of step S21 is as follows:

[0055] S211, input the point cloud data into the multi-layer Transformer encoder to obtain the geometric feature representation of the point cloud data.

[0056] In the present embodiment, the point cloud feature extraction module uses an improved Transformer architecture, which can capture the relationship between points through a self-attention mechanism and process the geometric information of the stockpile. The dimension of the input radar point cloud data is N x 3, where N is the number of points and 3 is the three-dimensional spatial coordinates (x, y, z) of each point. On this basis, a multi-layer Transformer encoder is used to process the point cloud data to generate a geometric feature representation. Specifically:

[0057] The radar point cloud data is first processed by the Transformer encoder. Each layer of the Transformer processes the input point cloud data through a self-attention mechanism to capture the relationship between points, and the calculation formula is as follows:

[0058] F Point =TransformerEncoder(X)

[0059] where X is the input point cloud data, F Point is the point cloud feature representation after Transformer encoding, and TransformerEncoder(·) represents the Transformer encoding operation.

[0060] S212, the multi-scale convolutional network is used to process the geometric feature representation to obtain the local geometric features and the global geometric features of the stockpile.

[0061] In order to further enhance the point cloud feature extraction capability, a multi-scale convolutional network (MS-CNN) is introduced. The network extracts local geometric features and global geometric features of the stockpile through convolution operations at different scales. In the multi-scale convolutional network, local features are mainly extracted in the shallow convolutional layer. The shallow convolutional layer performs convolution operation on the input image through multiple convolution kernels, and each convolution kernel extracts local geometric features such as edges and textures in the image. Global features are mainly extracted in the deep convolutional layer. With the deepening of the convolutional layer, the local geometric features extracted in the shallow layer are combined and abstracted into higher-level features, and finally form global geometric features.

[0062] The processing formula of the multi-scale convolutional network for geometric feature representation is as follows:

[0063] F i =Covn i (F Point ),i∈{1,2,3}

[0064] Where F i is the output of the i-th convolution operation, i represents the index of the convolutional layer of the multi-scale convolutional network; F Point is the radar point cloud feature after Transformer encoding.

[0065] S213, determining the point cloud feature of the stockpile according to the local geometric feature and the global geometric feature.

[0066] S22, performing feature extraction on the stacking state text description data to obtain semantic features.

[0067] The stacking state text description data is processed by the BERT model. The BERT model can generate high-quality embedding vectors that can capture feature information in the stacking state text description. After processing by the BERT model, each feature description is converted into a fixed-dimensional vector representation as input for subsequent processing, and its core formula is as follows:

[0068] F text =BERT pooled (T)=CLS_token(Transformer L (T embed ))

[0069] Where T represents the input text description sequence, consisting of n tokens:

[0070] T=[t1,t2,…,t n ]

[0071] T embed= E token + E position + E segment

[0072] T embed is the embedding representation of input text. Transformer L represents the BERT backbone network composed of L layers of Transformer encoder. CLS_token is the first special token (classification mark) of BERT output, which is used for overall semantic representation.

[0073] BERT pooled (T) here represents the pooling processing of BERT output, the most commonly used way is to extract the CLS_token vector at the first position, which is a special symbol added at the beginning of the input sequence by BERT, used to aggregate the semantic information of the whole text, especially suitable for tasks such as text classification that require overall representation. In addition, strategies such as average pooling, maximum pooling or multi-layer feature fusion can also be selected, but in this model we choose CLS_token as the default.

[0074] The input embedding layer can be represented as

[0075] T embed = E token (t i ) + E position (i) + E segment (s i )

[0076] where E token (t i ) represents the word vector embedding of token (t i ), E position (i) represents the position encoding of position i, using the sine-cosine function:

[0077]

[0078] where i represents the absolute position index of token in the sequence, j represents the corresponding frequency order, d model is the hidden dimension size of embedding vector, and the constant 10000 is used to expand the frequency range in the exponential scale, so as to cover the multi-scale position information from high to low. Exponential decay makes the first few dimensions correspond to short-period sine waves (high frequency, sensitive to absolute / local position), and the last few dimensions correspond to long-period sine waves (low frequency, can encode global / relative distance). E pos (i,2j) and E pos(i,2j+1) respectively correspond to the calculation of different dimensions of the position encoding vector, j is used to traverse the dimension index, and is used to determine whether the sine or cosine function is used to calculate the corresponding vector in the calculation of the position encoding.

[0079] E segment (s i ) represents the segmentation encoding, which is used to distinguish different sentences.

[0080] The BERT output sequence [C, h1, h2, …, h n ], wherein C = CLS_token.

[0081] C (the length of C is d model ) is the context embedding of the first CLS token in the sequence, which is used for overall semantic representation and contains the global semantic information of the entire input text (or text pair), and can be regarded as an aggregated expression of the overall meaning of the input text. Each h (of the same length) corresponds to the context embedding of the i-th actual token, which is the output vector of the BERT model after processing each token in the input text, and contains the semantic information of the corresponding position token, and also integrates the influence of other tokens in the context on it, which can be used for token-level tasks. n

[0082] S23, fuse the point cloud feature and the semantic feature to obtain a fused feature.

[0083] In order to facilitate the fusion of the point cloud feature and the semantic feature, the semantic feature is aligned with the point cloud feature.

[0084] In order to align with the point cloud feature, a linear projection layer F' is added: text = LayerNorm (W proj F text +b proj )

[0085] Wherein d BERT is the dimension of the BERT feature, and d point is the dimension of the point cloud feature. W proj is the weight matrix of the linear projection layer. It is a matrix with a spatial dimension of , which is responsible for mapping and converting the feature dimension d BERT output by BERT to the point cloud feature dimension d point , so that the text feature and the point cloud feature can be aligned in dimension, and subsequent operations such as fusion can be facilitated.

[0086] ​The point cloud feature and the semantic feature are fused through a multi-modal alignment mechanism. The alignment process calculates the similarity between the point cloud feature and the description feature using a self-attention mechanism, thereby realizing the fusion of the two, and finally generating a unified feature representation. The alignment formula is as follows:

[0087] F aligned =Aliend(F Point ,F’ text )

[0088] Wherein, F aligned is the fused multi-modal feature representation; F Point is the feature obtained by the point cloud encoding network Transformer, and F’ text is the description text feature output by the text encoder BERT and enhanced to the same dimension after linear projection;

[0089] Aliend(·) is a multi-modal alignment function, and Aliend(·) calculates and weights the fusion of two modal features to generate a unified cross-modal representation F aligned , which not only retains the spatial information of the point cloud but also integrates the semantic information of the text. The alignment process is performed by calculating the similarity between the point cloud feature and the point cloud description feature. The present patent adopts the cosine similarity method.

[0090] First, calculate the cosine similarity of the point cloud feature and the semantic feature:

[0091]

[0092] Wherein, ||·|| represents the modulus length;

[0093] Then, weighted fusion is performed:

[0094] F aligned =β·F Point +(1-β)·F' text

[0095] Wherein β represents

[0096] ∈ is a small constant to prevent the denominator from being 0.

[0097] S24, analyze the environmental data to determine the environmental impact weight, and adjust the fused feature according to the environmental impact weight to obtain the environmental perception fused feature.

[0098] In the preferred mode of the present embodiment, by collecting environmental data (such as temperature, humidity, vibration, etc.) in real time, automatically calculating the environmental impact weight, adjusting the fusion features according to the changes of environmental data, so that the model can maintain high accuracy and robustness in dynamic environment. The workflow of this module includes environmental data collection, influence function design, feature adjustment calculation and final weight application.

[0099] Referring to Figure 3 , the environmental data includes temperature data, humidity data and vibration data, wherein the temperature influence E temp on the environment is represented by the temperature of the area where the stockpile is located, and the temperature may affect the physical properties of the stockpile; the humidity influence E humidity on the environment is represented by the environmental humidity, which may change the stability of the stockpile and the material; the vibration influence E vibration on the environment is represented by the vibration sensor monitoring the vibration changes of the environment where the stockpile is located, and the vibration may cause the displacement or instability of the stockpile.

[0100] In the preferred embodiment of the present embodiment, the specific processing mode of step S23 is:

[0101] determining the environmental impact weight according to at least one of the temperature influence factor, the humidity influence factor and the vibration influence factor, and determining the fusion ratio of the fusion features and the semantic features according to the environmental impact weight;

[0102] determining the temperature influence factor of the environment on the stockpile type according to the temperature data, determining the humidity influence factor of the environment on the stockpile type according to the humidity data, and determining the vibration influence factor of the environment on the stockpile type according to the vibration data.

[0103] In the environment-driven feature regulation module, the core task is to calculate the influence of each environmental factor on the fusion features according to the environmental data. This process is completed through the influence function . Each environmental data source (temperature, humidity, vibration) has a corresponding influence function, and through the output of these functions, the model can dynamically adjust the weight of the fusion features. The present application designs different influence functions for each environmental data source, and the specific method is as follows:

[0104] The influence of temperature change on the stockpile is usually nonlinear, especially in high temperature environment, the stability of the stockpile may decrease. Therefore, the influence function of temperature needs to enhance the attention to the stability of the stockpile in the case of large temperature fluctuation. Specifically, the sigmoid function is used to represent the influence of temperature on the fusion features. The sigmoid function can smoothly adjust the influence degree according to the range of temperature change, so as to appropriately weight the fusion features. The calculation formula of temperature influence factor is

[0105]

[0106] wherein, represents the temperature influence factor; k temp is the sensitivity coefficient of temperature, used to represent the sensitivity of temperature change to the environmental influence weight; E temp represents the current temperature value, and T0 is the reference temperature value;

[0107] The influence of humidity on the pile is similar to that of temperature, and generally in extreme humidity conditions, the stability of the pile will change. The humidity influence function needs to appropriately increase the model's attention to the stability of the pile type in the case of large humidity changes. The ReLU (Rectified Linear Unit) function of the present application is used to represent the influence of humidity on the fused features. The ReLU function can quickly enhance the weight of the features when the humidity exceeds a certain threshold. The calculation formula of the humidity influence factor is

[0108]

[0109] wherein, represents the humidity influence factor; k humidity is the sensitivity coefficient of humidity, used to represent the influence of humidity on the environmental influence weight; E humidity is the current humidity value, and H0 is the reference humidity value; max(·) represents the maximum value operation;

[0110] The influence of vibration on the pile is very direct, especially in the environment with strong vibration, the stability of the pile will decrease significantly. Therefore, the influence function of vibration needs to enhance the influence on the pile type judgment when the vibration is strong. The present application adopts the combination form of sigmoid function and ReLU, so as to strongly affect the weight adjustment of the model in the case of large vibration amplitude. The calculation formula of the vibration influence factor is

[0111]

[0112] wherein, represents the vibration influence factor; k vibration is the sensitivity coefficient of vibration, E vibration is the current vibration amplitude, V0 is the reference vibration value, and V threshold is the vibration threshold value.

[0113] Each environment data source has a corresponding influence function, and the final environmental weight EnvWeigh t(E) is obtained by weighting and combining the influences of all environment data sources. The present application calculates the environmental weight by weighting and summing the influence functions of each environment data source, and the calculation formula of the environmental influence weight is

[0114]

[0115] wherein EnvWeight(E) represents the environmental impact weight, w temp represents the weight of the temperature impact factor, w vibration represents the weight of the humidity impact factor, w humidit represents the weight of the vibration impact factor.

[0116] Specifically, according to the environmental impact weight EnvWeight(E), the fused features are dynamically adjusted to ensure that under different environmental conditions, the model can automatically enhance the attention to the most important features for judging the stockpile shape. The adjustment formula for dynamically adjusting the original fusion weight of the point cloud features according to the environmental impact weight EnvWeight(E) is as follows:

[0117] F adjusted = F aligned × EnvWeight(E)

[0118] wherein F adjusted is the fused feature after environmental perception adjustment (i.e., the environmental perception fused feature), F aligned is the fused feature, and EnvWeight(E) is the weight calculated according to the environmental data.

[0119] S25, determining the stockpile shape recognition result according to the environmental perception fused feature.

[0120] The task of the stockpile shape judgment model is to further use the fused features after feature extraction, fusion, and environmental driving feature regulation for accurate classification and description generation of the stockpile shape. According to the environmental perception fused feature, a detailed description of the stockpile is generated, including information such as the shape type, stability, and accumulation mode of the industrial stockpile.

[0121] The description generation of the stockpile shape is realized by a GPT-4 model (Generative Pre-trained Transformer 4), which is a pre-trained large generative Transformer model specifically designed for description generation tasks. GPT-4 generates a natural language description of the stockpile using the input features. This process utilizes the powerful autoregressive generation capabilities of GPT-4 to generate detailed shape descriptions from feature vectors through step-by-step generation.

[0122] The input of the GPT-4 model is the fused feature F adjusted , which is a feature vector obtained by regulating and fusing point cloud data, description information, and environmental features. This vector contains multi-dimensional information of the stockpile, including its geometric shape, semantic information, and environmental adaptability.

[0123] The GPT-4 model uses a self-recurrent generation mechanism for description generation. The characteristic of the self-recurrent model is that each time a new word is generated, it relies on the previously generated words. The generation process of the model can be represented as:

[0124] y t =GPT-4(F adjusted ,y1,y2,…y t-1 )

[0125] where y t is the output of the current time step (the tth word), F adjusted is the input environmental perception fusion feature vector, which contains all the necessary point cloud, description features and environmental information, y1,y2,...,y t-1 is the generated first t-1 words. In this way, GPT-4 uses the self-recurrent mechanism to rely on the input features and the previously generated words to generate the entire description step by step until the predetermined end marker is reached or the complete description is generated.

[0126] The generated description will accurately reflect the type of pile shape, the way of stacking, the stability and the environmental adaptability of the pile. The following is a specific example of the generated description:

[0127] (1) "The pile presents a three-dimensional stacking mode, the bottom is stable but the upper part has a risk of tilting, and is suitable for stacking in a relatively stable environment."

[0128] (2) "The pile is flat and has high overall stability, but may deform under high humidity."

[0129] (3) "The pile is conical in structure, suitable for long-term stacking, but may be unstable in a frequently vibrating environment."

[0130] (4) "The pile shape is irregular, showing a scattered stacking, with poor overall stability, and the safety monitoring of the stacking area needs to be enhanced."

[0131] The final generated description output not only provides the classification information of the pile, but also provides rich environmental adaptability information for subsequent management, storage and safety evaluation.

[0132] In another embodiment of the present application, the industrial pile recognition method further comprises:

[0133] S3, obtaining a training data set and training a pile type judgment model using the training data set.

[0134] The training data set includes stockpile sample data and true information corresponding to the stockpile shape of the stockpile sample data. The training process of the stockpile shape judgment and description generation model aims to optimize the GPT-4 model so that it can generate accurate stockpile descriptions after receiving input features and capture key information such as shape type and stability during the generation process. The training process focuses on improving the description generation ability of the model by minimizing the description generation loss.

[0135] The training steps of the stockpile shape judgment model include:

[0136] S31, construct the network structure of the initial stockpile shape judgment model.

[0137] S32, train the network of the initial stockpile shape judgment model using the training data set. In each training, input the stockpile sample data into the initial stockpile shape judgment model, and obtain the prediction result of the stockpile shape of the stockpile sample data by the initial stockpile shape judgment model.

[0138] The stockpile sample data includes point cloud data, description data and environment data. The point cloud data is processed by a feature extraction module to generate point cloud features, which are used to describe the geometric information of the stockpile. The description data is converted into description features by a BERT model, providing semantic support. Temperature, humidity, vibration and other environmental factors are smoothed by a Kalman filter to generate environmental impact weights.

[0139] The point cloud features, description features and environmental impact weights are input into the GPT-4 model, and the model is optimized by minimizing the generation loss. GPT-4 generates the description of the stockpile according to the input F adjusted features. The training goal is to optimize the description generation ability of the model by minimizing the generation loss.

[0140] S33, calculate the negative log-likelihood loss between the true information and the prediction result, and optimize the network parameters of the initial stockpile shape judgment model according to the negative log-likelihood loss, to obtain the final stockpile shape judgment model.

[0141] Since the task of stockpile shape judgment and description generation depends on GPT-4 to generate point cloud descriptions, the loss function mainly focuses on the quality of description generation. Therefore, the negative log-likelihood loss (NLLLoss) is used, which is used to measure the difference between the generated description and the true description. The specific formula is as follows:

[0142]

[0143] where T is the length of the generated description, y t is the word generated at the t-th time step, and finally P(y t |y1,y2…y t-1F adjusted ) is GPT-4 according to F adjusted The feature vector and the preceding content generate the probability of the current word.

[0144] The training uses the Adam optimizer, in order to prevent the learning rate from being too large to cause unstable training or too small to cause slow convergence speed, a learning rate decay strategy is adopted, the initial learning rate is set to 0.0001, the batch size is set to 32, and a total of 20 cycles are trained, and L2 regularization is added in the training process, and the regularization coefficient is set to 0.0001.

[0145] By optimizing the description generation loss, the training process will continuously improve the accuracy and fluency of the generated description, and ultimately achieve the purpose of accurately describing the material pile.

[0146] Based on the same inventive concept, an embodiment of the present application provides an industrial material pile identification system.

[0147] The industrial material pile identification system can be loaded in an electronic device. According to the functions implemented, the industrial material pile identification system includes an acquisition module and a processing module,

[0148] The acquisition module can acquire material pile data to be identified, and the material pile data includes radar point cloud data, stacking state text description data and environment data of the material pile; the processing module can input the material pile data into a material pile type judgment model, and the material pile type judgment model processes the material pile data according to a preset rule and outputs a material pile type identification result;

[0149] The processing steps of the material pile type judgment model on the material pile data are:

[0150] The radar point cloud data is subjected to feature extraction to obtain point cloud features;

[0151] The stacking state text description data is subjected to feature extraction to obtain semantic features;

[0152] The point cloud features and the semantic features are fused to obtain fusion features;

[0153] The environment data is analyzed to determine an environment influence weight, and the fusion features are adjusted according to the environment influence weight to obtain environment perception fusion features;

[0154] The environment perception fusion features are used to determine the material pile type identification result.

[0155] The modules described in the present application can also be referred to as units, which refer to a series of computer program segments that can be executed by an electronic device processor and can complete fixed functions, and are stored in the memory of the electronic device.

[0156] The various changes and specific examples of the industrial pile identification method provided in the above embodiments are also applicable to the industrial pile identification system of the present embodiment. The skilled in the art can clearly understand the implementation method of the industrial pile identification system in the present embodiment through the foregoing detailed description of the industrial pile identification method. For the sake of brevity of the specification, the implementation method of the industrial pile identification system in the present embodiment will not be described in detail here.

[0157] The present application also discloses an electronic device, such as Figure 4 As shown in FIG. 1, it is a structural schematic diagram of an electronic device for an industrial pile identification method according to an embodiment of the present application. The electronic device can include at least one processor 10, a memory 11 in communication with the at least one processor, a communication bus 12, and a communication interface 13, and can further include a computer program stored in the memory 11 and executable on the processor 10, such as a method program for industrial pile identification.

[0158] In some embodiments, the processor 10 can be composed of integrated circuits, for example, can be composed of a single packaged integrated circuit, or can be composed of multiple packaged integrated circuits with the same function or different functions, including one or more combinations of central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and various control chips. The processor 10 is the control unit of the electronic device, which connects all components of the electronic device through various interfaces and lines, and executes or runs programs or modules stored in the memory 11 (such as a method for industrial pile identification, etc.), and calls data stored in the memory 11, to perform various functions of the electronic device and process data.

[0159] The memory 11 includes at least one type of readable storage medium, including flash memory, mobile hard disk, multimedia card, card-type memory (such as SD or DX memory, etc.), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 11 can be an internal storage unit of the electronic device, such as a mobile hard disk of the electronic device. In other embodiments, the memory 11 can also be an external storage device of the electronic device, such as a plug-in mobile hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device. Further, the memory 11 can include both the internal storage unit and the external storage device of the electronic device. The memory 11 can be used not only to store application software and various data installed on the electronic device, such as the code of the method program for industrial pile identification, but also to temporarily store data that has been output or will be output.

[0160] The communication bus 12 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. The bus is configured to enable connection and communication between the memory 11, the at least one processor 10, etc.

[0161] The communication interface 13 is configured to enable communication between the electronic device and other devices, including a network interface and a user interface. Optionally, the network interface can include a wired interface and / or a wireless interface (e.g., a WI-FI interface, a Bluetooth interface, etc.), and is typically configured to establish a communication connection between the electronic device and other electronic devices. The user interface can be a display, an input unit (e.g., a keyboard), and optionally, the user interface can also be a standard wired interface, a wireless interface. Optionally, in some embodiments, the display can be an LED display, a liquid crystal display, a touch liquid crystal display, an OLED (Organic Light-Emitting Diode) touch screen, etc. The display can also be referred to as a display screen or a display unit, and is configured to display information processed in the electronic device and to display a visualized user interface.

[0162] Figure 4 Only the electronic device with components is shown, and those skilled in the art can understand that, Figure 4 The structure shown does not limit the electronic device, and the electronic device can include fewer or more components than shown, or combine certain components, or have different component arrangements. For example, although not shown, the electronic device can also include a power supply (e.g., a battery) to supply power to each component. Preferably, the power supply can be logically connected to the at least one processor 10 through a power management device, so that the power management device can implement functions such as charge management, discharge management, and power consumption management. The power supply can also include one or more direct current or alternating current power supplies, a recharging device, a power supply fault detection circuit, a power supply converter or inverter, a power supply status indicator, etc. The electronic device can also include various sensors, a Bluetooth module, a Wi-Fi module, etc., which are not described here.

[0163] It should be understood that the embodiments are for illustration only, and the scope of the patent application is not limited by the structure.

[0164] Further, the modules / units integrated in the electronic device, if realized in the form of software function units and sold or used as independent products, can be stored in a computer readable storage medium. The computer readable storage medium can be volatile or non-volatile.

[0165] The embodiment of the present application provides a computer readable storage medium, for example, including any entity or device capable of carrying the computer program code, a recording medium, a U disk, a mobile hard disk, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM). The computer readable storage medium stores the computer program capable of being loaded by the processor and executing the industrial stockpile identification method of the above-mentioned embodiment.

[0166] In the description of the present specification, the description of the terms "one embodiment", "some embodiments", "an example", "a specific example", "an implementation", "a preferred implementation" or "some examples" and the like means that the specific features, structures, materials or characteristics described in combination with the embodiment or example are included in at least one embodiment or example of the present application. In the present specification, the illustrative description of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0167] Although the embodiments of the present application have been shown and described, those of ordinary skill in the art can understand that various changes, modifications, replacements and variations can be made to the embodiments without departing from the principles and spirit of the present application, and the scope of the present application is defined by the claims and their equivalents.

Claims

1. A method for identifying industrial stockpiles, characterized in that, The method includes: Acquire the data of the material pile to be identified, which includes radar point cloud data of the material pile, text description data of the stacking status, and environmental data; Input the stockpile data into the stockpile type judgment model. The stockpile type judgment model processes the stockpile data according to preset rules and outputs the stockpile type recognition result. The processing steps for the stockpile data in the stockpile type determination model are as follows: Feature extraction is performed on radar point cloud data to obtain point cloud features; Semantic features are obtained by extracting features from the text description data of the stacking status; By fusing point cloud features and semantic features, a fused feature is obtained; The environmental data is analyzed to determine the environmental impact weights, and the fusion features are adjusted according to the environmental impact weights to obtain the environmental perception fusion features; The material pile type identification result is determined based on the environmental perception fusion characteristics; The stacked text description data is processed by the BERT model. After processing by the BERT model, each feature description is transformed into a fixed-dimensional vector representation, thus obtaining semantic features.

2. The industrial stockpile identification method as described in claim 1, characterized in that, The step of extracting features from radar point cloud data to obtain point cloud features includes: The point cloud data is input into a multi-layer Transformer encoder to obtain the geometric feature representation of the point cloud data; Multi-scale convolutional networks are used to process geometric feature representations to obtain local and global geometric features of the material pile; The point cloud features of the material pile are determined based on local and global geometric features.

3. The industrial stockpile identification method as described in claim 1, characterized in that, Environmental data includes at least one of temperature data, humidity data, and vibration data, and the environmental impact weight is determined based on at least one of temperature impact factor, humidity impact factor, and vibration impact factor. Determine the temperature influence factors of the environment on the stockpile type based on temperature data; Determine the humidity influence factors of the environment on the material pile type based on humidity data; The vibration data is used to determine the vibration influence factors of the environment on the stockpile shape.

4. The industrial stockpile identification method as described in claim 3, characterized in that, The formula for calculating the temperature influence factor is as follows: in, Indicates the temperature influence factor; This is the temperature sensitivity coefficient, used to represent the sensitivity of the environmental impact weight of temperature changes; This indicates the current temperature value. The reference temperature value; The formula for calculating the humidity impact factor is: in, Indicates the humidity influencing factor; This is the humidity sensitivity coefficient, used to influence the weighting of humidity's impact on the environment; This is the current humidity value. The baseline humidity value; This indicates the operation of retrieving the maximum value; The formula for calculating the vibration influence factor is as follows: in, Indicates the vibration influence factor; The sensitivity coefficient for vibration. This represents the current vibration amplitude. As the reference vibration value, This is the vibration threshold.

5. The industrial stockpile identification method as described in claim 4, characterized in that, The formula for calculating the environmental impact weight is: in, Indicates the weight of environmental impact. This indicates the weight of the temperature-related factors. This indicates the weight of the humidity-related factors. This indicates the weight of the vibration influence factor.

6. The industrial stockpile identification method according to any one of claims 1 to 5, characterized in that, The method further includes: acquiring a training dataset and using the training dataset to train a material pile type judgment model.

7. The industrial stockpile identification method as described in claim 6, characterized in that, The training dataset includes stockpile sample data and real information on the stockpile shape corresponding to the stockpile sample data. The training steps for the stockpile shape judgment model include: Construct the network structure for the initial material pile type judgment model; The network of the initial material pile type judgment model is trained using the training dataset. In each training session, the material pile sample data is input into the initial material pile type judgment model, and the prediction result of the initial material pile type judgment model on the material pile sample data is obtained. The negative log-likelihood loss between the actual information and the predicted result is calculated, and the network parameters of the initial stockpile type judgment model are optimized based on the negative log-likelihood loss to obtain the final stockpile type judgment model.

8. An industrial stockpile identification system, used to implement the industrial stockpile identification method according to any one of claims 1 to 7, characterized in that, include: The acquisition module is used to acquire the data of the material pile to be identified. The material pile data includes radar point cloud data of the material pile, text description data of the stacking status, and environmental data. The processing module is used to input the stockpile data into the stockpile type judgment model. The stockpile type judgment model processes the stockpile data according to preset rules and outputs the stockpile type recognition result. The processing steps for the stockpile data in the stockpile type determination model are as follows: Feature extraction is performed on radar point cloud data to obtain point cloud features; Semantic features are obtained by extracting features from the text description data of the stacking status; By fusing point cloud features and semantic features, a fused feature is obtained; The environmental data is analyzed to determine the environmental impact weights, and the fusion features are adjusted according to the environmental impact weights to obtain the environmental perception fusion features; The material pile type identification result is determined based on the environmental perception fusion characteristics.

9. An electronic device, characterized in that, The electronic device includes: At least one processor (10); and, A memory (11) communicatively connected to the at least one processor (10); The memory (11) stores a computer program that can be executed by the at least one processor (10) to enable the at least one processor (10) to perform the industrial stockpile identification method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program; when the computer program is executed by a processor, it implements the industrial stockpile identification method as described in any one of claims 1 to 7.

Images