Intelligent automatic identification method and system for transfer machine

Abstract

The invention provides an intelligent automatic identification method and system for a transfer machine. The method comprises the following steps: installing various sensors on the transfer machine to collect multi-modal data; segmenting a single voxel into a plurality of object areas to obtain voxel sets of different objects; performing stability and state analysis on the object according to the enhanced environment model to form an object state characteristic vector, and integrating the object state characteristic vector into the enhanced environment model; generating a preliminary carrying strategy suitable for each object according to the enhanced state model; a sensor on the transfer machine is used for collecting state data of an object in real time, a feedback vector is generated, and carrying parameters are adjusted and finely adjusted according to feedback information; various parameters and adjustment data in the carrying process are fed back to the small sample learning model, and category data and stability factors in the carrying data are fed back to the small sample learning model, and feature description of categories is updated. According to the invention, the limitation of a traditional transfer machine in a changeable environment is overcome, and the challenge of carrying diversified objects can be coped with.

Description

Technical Field

[0001] The present invention belongs to the field of intelligent automatic identification, and in particular relates to an intelligent automatic identification and analysis method and system for a transfer machine. Background Art

[0002] In modern industrial and logistics automation, transfer machines (or automatic handling robots) have become the core equipment for material handling and are widely used in warehouses, production workshops and logistics centers. These automated devices usually rely on sensors, cameras and algorithms to identify and move objects to improve production and transportation efficiency. However, existing transfer machine systems still face many technical bottlenecks in practical applications, especially when dealing with complex scenes and diverse objects, showing a lack of adaptability.

[0003] In current technology, the object recognition function of most transfer machines relies on single or a few sensor data (such as cameras or lidar), which is only applicable to the handling of fixed or stable objects. Under such a technical model, the transfer machine system usually pre-sets the recognition and handling parameters and generates fixed grasping and handling strategies for known objects. However, when the environment is complex and changeable, and the objects are of various types and shapes, the transfer machine system often finds it difficult to quickly identify the characteristics of new objects, resulting in the handling strategy not being consistent with the actual situation, thereby reducing the efficiency and stability of the system. In addition, when facing a dynamically changing environment, traditional transfer machines cannot flexibly adjust the handling strategy in real time. For example, the object being transported may be offset due to vibration during transportation, or obstacles may suddenly appear around it, which will lead to handling failure or even damage to the object. Even if some high-end transfer machines integrate multiple sensors, they are still limited by the lack of effective multimodal data fusion and adaptive algorithms, and the system still cannot fully utilize multi-dimensional data for object state analysis and real-time strategy optimization.

[0004] In addition to the flexibility problem of recognition and handling strategies, current transfer machine systems have another significant flaw, which is that they usually require a large amount of labeled data to train and recognize new objects. When encountering new objects that have never been seen before, it is difficult for the system to generate reliable recognition and handling strategies in a short period of time, resulting in a significant reduction in work efficiency. In practical applications, especially when faced with frequent changes and diverse handling needs in complex environments, traditional transfer machine systems lack the ability to self-learn and quickly adapt to new objects. Obtaining sufficient labeled data is not only time-consuming, but also not cost-effective. Therefore, how to develop an adaptive transfer machine system that can quickly identify new objects and flexibly adjust when the environment changes has become an important challenge in current intelligent handling technology. Summary of the invention

[0005] The purpose of the present invention is to propose an intelligent automatic recognition and analysis method and system for a transfer machine, which overcomes the limitations of traditional transfer machines in changing environments and meets the challenges of handling diverse objects through adaptive scene perception, multi-object state evaluation and dynamic handling algorithm of small sample learning.

[0006] In order to achieve the above object, a first aspect of the present invention provides an intelligent automatic identification method for a transfer machine, the method comprising:

[0007] S1. Install multiple sensors on the transfer machine to collect multimodal data, and generate an environmental model M in the form of 3D voxels based on the multimodal data, where M = {v i},v i Represents a single voxel, and optimizes and segmented the environment model M to obtain the segmented sub-regions, and then extracts regional features from each sub-region to construct the regional model M j ; Wherein, the multimodal data includes RGB image data, point cloud data of laser radar and temperature data of infrared sensor, the single voxel includes RGB color value, point cloud depth value and temperature value, and the features include shape feature, texture feature and temperature feature;

[0008] S2, split a single voxel into multiple object regions to obtain a set of voxels of different objects, denoted as Each of these sets Represents a target object, and then the voxel set Each voxel v in i Perform weighted feature calculation to obtain the object The comprehensive feature vector x j , then the comprehensive feature vector x j Perform small sample learning to build a small sample learning model, and use the small sample learning model to calculate the central feature vector μ of each category c , when an unknown object is detected When , extract its feature vector And with each category center μ c Calculate the distance and place the unknown object Classify it into the category c with the smallest distance * , and finally each identified object category label c j Assign its eigenvector x j , and update the environment model to generate an enhanced environment model M containing the category label C , that is, M C = {v i ,c i}, where c i is the category label of the object corresponding to the voxel;

[0009] S3, according to the enhanced environment model M C The stability and state analysis of the object is performed, and the object state characteristic vector is formed according to the results of the stability and state analysis, and then the object state characteristic vector is integrated into the enhanced environment model M C In the enhanced state model M CS ; Wherein, the results of the stability and state analysis include the stability factor S j , dynamic inertia factor Γ j , the enhanced state model M CS Indicated as M CS = {v i ,c i ,S i ,Γ i};

[0010] S4, according to the enhanced state model M CS Generate a preliminary handling strategy suitable for each object, wherein the preliminary handling strategy includes the grasping force F j , transport speed v j , maximum acceleration a j and the final path

[0011] S5. Use the state data of the object collected in real time by the sensor on the transfer machine to generate a feedback vector. At the same time, during the handling process, the state changes and environmental factors of the object are monitored in real time through the sensor. The handling parameters are adjusted and fine-tuned according to the feedback information, and updated in real time during the next handling process of the transfer machine. When the environment or the state of the object changes further, the transfer machine will repeat the above real-time feedback adjustment steps to form a closed-loop control mechanism.

[0012] S6. Summarize various parameters and adjustment data during the handling process to generate a data set Will transport data The category data and stability factor in the sample learning model are fed back to update the feature description of the category and update the enhanced state model M. CS Get the updated state model M opt , and finally according to the updated state model M opt Further update the object handling strategy.

[0013] Further, in the S1, for the RGB image data I RGB , using image enhancement and segmentation methods to remove background impurities and improve the clarity of the edge of the target object; Lidar Sparse processing is performed to remove background noise points and retain key points on the surface of the object; for infrared temperature data T IR, perform normalization processing based on temperature threshold, unify the temperature range, and reduce the data distribution differences caused by different sensors; finally, fuse the preprocessed data into a multimodal point set D f .

[0014] 3. The intelligent automatic identification method for a transfer machine according to claim 1 is characterized in that the optimization and regional segmentation of the environmental model M are performed to obtain the segmented sub-regions, and then the regional features of each sub-region are extracted to construct the regional model M. j , including:

[0015] An adaptive voxel optimization algorithm is used to downsample high-density areas:

[0016] At each voxel v i , calculate its local density ρ i , defined as the number of surrounding voxels with a density greater than a set threshold ρ threshold Downsample the voxel groups to improve computational efficiency;

[0017] The environmental model M is divided into independent regions using an adaptive segmentation method based on spatial connectivity and temperature distribution, and each segmented sub-region M is obtained. j ;

[0018] For each segmented sub-region M j , extract regional features; the shape feature is calculated based on the point cloud contour and surface curvature, the texture feature is based on the color gradient and edge directionality analysis, and the temperature feature is calculated based on the thermal distribution pattern of infrared data. Combining the above features, the update model of each region is represented as M j ={φ shape (M j ),φ texture (M j ),φ temp (M j )}, where the shape feature, texture feature and temperature feature are φ shape (M j ),φ texture (M j ) and φ temp (M j ).

[0019] Furthermore, the voxel set Each voxel v in i Perform weighted feature calculation to obtain the object The comprehensive feature vector x j , calculated as follows:

[0020]

[0021] Among them, x j Representing objects The comprehensive feature vector includes the weighted mean of multimodal features such as RGB color, depth, and temperature; Indicates the object area The number of voxels in the mean value, w RGB 、w Depth and w Temp Respectively represent the weight parameters of RGB, depth and temperature modal features, which are used to adjust the influence of different modal features in the fusion feature; I RGB (v i ), P Lidar (v i ) and T IR (v i ) represent voxel v i RGB color value, LiDAR depth value and temperature;

[0022] The comprehensive feature vector x j Perform small sample learning to build a small sample learning model, and use the small sample learning model to calculate the central feature vector μ of each category c , calculated as follows:

[0023]

[0024] Among them, μ c Represents the central feature vector of category c, which is used to represent the comprehensive features of objects of this type; Represents a sample set of category c, including all feature vectors belonging to category c; Representing a collection The number of samples in the function used to calculate the mean; Ω c represents the regularization term, which is used to limit the deviation of the category center and reduce overfitting; λ represents the regularization coefficient, which controls the regularization term Ω c The size of the impact.

[0025] Furthermore, when an unknown object is detected When , the category of the new object is matched and calculated as follows:

[0026]

[0027] Among them, d(x new ,μ c ) represents the new object x new and the central feature vector μ of category c c The matching distance between them is used to determine the category of the new object; new,j and μ c,j Represent the new object features xnew and category center μ c The component on the jth feature dimension; σ c Represents the discrimination parameter of category c, which is obtained by calculating the average distance between the center vectors of different categories to prevent category confusion and improve the discrimination between categories;

[0028] The unknown object Classify it into the category c with the smallest distance * , that is, satisfying c * =argmin c d(x new ,μ c )’s category label.

[0029] Further, the enhanced environment model M C The stability and state analysis of the object includes the following steps:

[0030] Split the voxels into i For multiple object areas Each area Represents object j, for each object Extract geometric, class and quality characteristics and construct a preliminary characteristic vector z j ={x j ,m j ,c j}, where x j For Objects The geometric center coordinates are denoted by x j =[x j1 ,x j2 ,x j3 ],m j is the mass of the object, and the volume V is calculated by j and volume density ρ j Get, that is, m j =ρ j ·V j , c j is the category label of the object;

[0031] Introduce the object's center of gravity offset Δ COM,j and contact angle θ j Constructing stability factor S as a stability influencing factor j , and adjust its contribution according to the category weight, and define the dynamic inertia factor Γ according to the inertia of the object during transportation j To quantify the inertia factor Γ j The characteristics of the object are combined with the mass and class inertia coefficient to calculate, which is expressed as follows:

[0032]

[0033] Among them, S j Representing objects The stability factor is in the range of [0,1], with the largest value being the most stable; Indicates category c j The adjustment parameters of Δ COM,j and θ j The weight of COM,j Represents the object's center of gravity offset, defined as the geometric center x j With the center of mass x COM,j The distance, Δ COM,j =∥x COM,j -x j ∥;θ j Represents the inclination angle of the object, the angle between the object and the ground contact surface; ∈ represents a small positive value to prevent the denominator from being zero; Γ j Representing objects The dynamic inertia factor describes its inertia; γ cj Indicates category c j The inertia coefficient is adjusted according to the object type; m j Represents the mass of an object, represented by its volume V j and volume density ρ j Calculate; j Represents the object density coefficient, which is calculated based on voxel data and reflects the overall density of the object.

[0034] Furthermore, the gripping force F j The calculation formula is as follows:

[0035]

[0036] Among them, F j Representing objects The gripping force, Represents the object category c j The grasping coefficient controls the grasping strength of different types of objects; m j Indicates the mass of an object; S j Represents the stability factor of the object; ∈ represents a small positive value that prevents the denominator from being zero; Ω j represents the enhanced regularization term, reflecting the morphological complexity of the contact surface of the object; η represents the regularization coefficient, adjusting the enhanced regularization term Ω j The impact of

[0037] The transport speed v j and the maximum acceleration a j , calculated as follows:

[0038]

[0039] Among them, v j Representing objects The handling speed; a j Indicates the acceleration of an object, which determines the smoothness of transport, starting and braking; and They are the adjustment coefficients for speed and acceleration, respectively, which are determined by the object category;

[0040] The final path The calculation is as follows:

[0041]

[0042] Among them, P j Represents object O j The final transport path adapts to the dynamic characteristics; represents the preliminary path planning; Indicates category c j The path adjustment coefficient reflects the path adjustment requirements of different objects; Δθ represents the change in the object's inclination angle, based on the object's stability S j and the tilt protection item generated by the current path angle.

[0043] Furthermore, the state data of the object collected in real time by the sensor on the transfer machine includes the object The real-time center of gravity offset d COM,j 、The real-time tilt angle θ of the object j and the speed v of obstacles detected in the environment obs,j , then the feedback vector is f j ={d COM,j ,θ j ,v obs,j};

[0044] According to the center of gravity offset d COM,j and the real-time tilt angle θ j Adjust gripping force F j , which is expressed as follows:

[0045] F′ j =F j +λ d ·d COM,j +λ θ ·sin(θ j )

[0046] Among them, F′ j Represents the adjusted grasping force, which is used to correct the grasper's strength in real time; d and λ θ are the adjustment coefficients of the center of gravity offset and the tilt angle, respectively, which are used to control the influence of the center of gravity offset and the tilt on the grasping force; sin(θ j) represents the sine term of the tilt angle. When the object is tilted, sin(θ j ) will increase, thereby increasing the gripping force to prevent slipping;

[0047] According to the obstacle speed v obs,j To adjust the transport speed v j At the same time, the transport path is fine-tuned through the real-time feedback of the inclination angle and path curvature, as shown below:

[0048] v′ j =v j ·exp(-γ·v obs,j )

[0049] Among them, v′ j It represents the adjusted handling speed, ensuring that the object slows down when approaching an obstacle; γ represents the speed adjustment coefficient, which is used to adjust the degree of influence of the obstacle speed on the handling speed;

[0050] According to the inclination angle θ of the object j Real-time fine-tuning of the transport path P j , so that the path avoids sudden sharp turns and unnecessary direction changes, is calculated as follows:

[0051] P′ j =P j +κ·θ j

[0052] Among them, P′ j represents the adjusted transport path; κ represents the path adjustment coefficient, which is used to control the influence of the inclination angle on the path smoothness.

[0053] Furthermore, the data set for in For the The feedback data vector of the transport is expressed as Among them, each Including real-time adjusted grip Adjusted transport speed v′ j , actual acceleration a′ j , Actual transport path The stability factor S of an object j and the object's dynamic inertia factor Γ j .

[0054] In another aspect of the present invention, an intelligent automatic identification system for a transfer machine is provided, the system comprising:

[0055] The multimodal data acquisition unit is used to install multiple sensors on the transfer machine to collect multimodal data, and generate an environmental model M in the form of 3D voxels based on the multimodal data, where M = {v i},v i Represents a single voxel, and optimizes and segmented the environment model M to obtain the segmented sub-regions, and then extracts regional features from each sub-region to construct the regional model M j ; Wherein, the multimodal data includes RGB image data, point cloud data of laser radar and temperature data of infrared sensor, the single voxel includes RGB color value, point cloud depth value and temperature value, and the features include shape feature, texture feature and temperature feature;

[0056] A single voxel segmentation unit is used to segment a single voxel into multiple object regions to obtain a set of voxels of different objects, denoted as Each of these sets Represents a target object, and then the voxel set Each voxel v in i Perform weighted feature calculation to obtain the object The comprehensive feature vector x j , then the comprehensive feature vector x j Perform small sample learning to build a small sample learning model, and use the small sample learning model to calculate the central feature vector μ of each category c , when an unknown object is detected When , extract its feature vector And with each category center μ c Calculate the distance and place the unknown object Classify it into the category c with the smallest distance * , and finally each identified object category label c j Assign its eigenvector x j , and update the environment model to generate an enhanced environment model M containing the category label C , that is, M C = {v i ,c i}, where c i is the category label of the object corresponding to the voxel;

[0057] The environment enhancement unit is used to enhance the environment model M C The stability and state analysis of the object is performed, and the object state characteristic vector is formed according to the results of the stability and state analysis, and then the object state characteristic vector is integrated into the enhanced environment model M C In the enhanced state model M CS ; Wherein, the results of the stability and state analysis include the stability factor S j , dynamic inertia factor Γj , the enhanced state model M CS Indicated as M CS = {v i ,c i ,S i ,Γ i};

[0058] The preliminary handling strategy generation unit is used to generate a preliminary handling strategy according to the enhanced state model M CS Generate a preliminary handling strategy suitable for each object, wherein the preliminary handling strategy includes the grasping force F j , transport speed v j , maximum acceleration a j and the final path

[0059] The real-time data acquisition unit is used to use the state data of the object collected in real time by the sensors on the transfer machine to generate a feedback vector. At the same time, the state changes and environmental factors of the object are monitored in real time through the sensors during the handling process. The handling parameters are adjusted and fine-tuned according to the feedback information, and updated in real time during the next handling process of the transfer machine. When the environment or the state of the object changes further, the transfer machine will repeat the above real-time feedback adjustment steps to form a closed-loop control mechanism.

[0060] The handling strategy optimization unit is used to aggregate various parameters and adjustment data during the handling process to generate a data set Will transport data The category data and stability factor in the sample learning model are fed back to update the feature description of the category and update the enhanced state model M. CS Get the updated state model M opt , and finally according to the updated state model M opt Further update the object handling strategy.

[0061] The beneficial technical effects of the present invention are at least as follows:

[0062] (1) The present invention innovatively integrates multimodal sensing data, including inputs from cameras, lidars, and force sensors, and uses adaptive algorithms to analyze changes in objects and the surrounding environment in real time. Through the scene perception module, the transfer machine can build an accurate three-dimensional environmental model and identify the spatial distribution and dynamic state of objects. Even if new obstacles are added or the position of objects changes in a complex environment, the system can quickly update the recognition results and handling strategies. This adaptive scene perception solves the defects of traditional systems that are limited by a single sensor and insensitive to dynamic changes, allowing the transfer machine to respond more flexibly to changes in the handling environment.

[0063] (2) The present invention introduces multi-object state assessment technology to perform a comprehensive analysis of the geometric shape, placement, center of gravity and stability of the object. For different categories of objects (such as fragile, sliding or irregular objects), the system will automatically generate corresponding clamping force and handling paths. For unstable objects, the system adopts protective clamping and avoids sharp turns and paths with excessive vibrations during path planning. Through dynamic strategy generation, the system can automatically adjust the clamping force and handling route according to real-time feedback during the handling process, thereby preventing objects from falling or being damaged. This innovation effectively solves the defect that traditional transfer machines are difficult to stably carry irregular objects in a changing environment.

[0064] (3) The present invention innovatively introduces a small sample learning module. With a small number of training samples, the system can quickly learn and classify new objects, generate adaptive handling strategies, and operate efficiently even when faced with new object categories. Combined with fast re-identification technology, the system can re-identify and adapt to the state changes of objects in dynamic environments. For example, if an object moves or changes its posture during transportation, the system can update the strategy in real time to ensure the stability and safety of transportation. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] The present invention is further described using the accompanying drawings, but the embodiments in the accompanying drawings do not constitute any limitation to the present invention. A person skilled in the art can obtain other drawings based on the following drawings without creative work.

[0066] Figure 1 The present invention is a flowchart of an intelligent automatic identification method for a transfer machine according to an embodiment of the present invention.

[0067] Figure 2 The present invention is a framework diagram of an intelligent automatic identification system for a transfer machine according to an embodiment of the present invention. DETAILED DESCRIPTION

[0068] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.

[0069] like Figure 1 As shown, an embodiment of the present invention provides an intelligent automatic identification method for a transfer machine, the method comprising the following steps S1-S6:

[0070] S1. Install multiple sensors on the transfer machine to collect multimodal data, and generate an environmental model M in the form of 3D voxels based on the multimodal data, where M = {v i},v iRepresents a single voxel, and optimizes and segmented the environment model M to obtain the segmented sub-regions, and then extracts regional features from each sub-region to construct the regional model M j ; Wherein, the multimodal data includes RGB image data, point cloud data of lidar and temperature data of infrared sensor, the single voxel contains RGB color value, point cloud depth value and temperature value, and the features include shape features, texture features and temperature features.

[0071] Specifically, first, multiple sensors (such as RGB cameras, laser radars, and infrared sensors) installed on the transfer machine are used to scan the current working area and collect multimodal data. Let the data collected by the sensor be D, where D = {I RGB ,P Lidar ,T IR}, representing RGB image data, LiDAR point cloud data, and infrared sensor temperature data respectively.

[0072] For RGB image data I RGB , use image enhancement and segmentation methods to remove background impurities and improve the clarity of the edge of the target object to facilitate subsequent accurate recognition. Lidar Sparse processing is performed to remove background noise points and retain key points on the surface of the object to ensure data quality. IR , perform normalization based on temperature threshold, unify the temperature range, and reduce the data distribution differences caused by different sensors. Finally, the preprocessed data is fused into a multimodal point set D f , for the next step.

[0073] Furthermore, the generated multimodal data D f Used to construct the environment model M. All sensor data are projected into a unified coordinate system to generate the environment model M in the form of 3D voxels, where M = {v i},v i represents a single voxel. For each voxel v i Assign the following feature attributes: RGB color value, point cloud depth value and temperature value.

[0074] In order to improve modeling efficiency and reduce redundant information, the characteristic function g(v i ) performs weighted calculation on each voxel:

[0075] g(v i )=w RGB I RGB (v i )+w Depth ·P Lidar (v i)+w Temp ·T IR (v i )

[0076] Among them, w RGB 、w Depth and w Temp are the weight coefficients of each mode, which are used to adjust the importance of different modes. The output of this function is used as the overall eigenvalue of the voxel in the environment model M.

[0077] Furthermore, in order to improve the accuracy of subsequent object recognition, the feature data in the model M is optimized and the region is segmented. An adaptive voxel optimization algorithm is used to downsample the high-density area to reduce the amount of calculation. i , calculate its local density ρ i , defined as the number of surrounding voxels. threshold The voxel groups are downsampled to improve computational efficiency.

[0078] According to the optimized environmental model, the environmental model M is divided into independent regions using an adaptive segmentation method based on spatial connectivity and temperature distribution. In this way, high-temperature objects or special-shaped regions can be segmented into independent subsets M. j , each subset contains information about various target objects, providing accurate data for subsequent object recognition.

[0079] Furthermore, for each segmented sub-region M j , regional feature extraction is performed. Three main features are extracted for each region, including shape features, texture features, and temperature features, to form a more refined regional model. Shape features can be calculated based on point cloud contours and surface curvature, texture features are based on color gradient and edge directionality analysis, and temperature features are calculated based on the thermal distribution pattern of infrared data. Let shape features, texture features, and temperature features be φ shape (M j ),φ texture (M j ) and φ temp (M j ).

[0080] Combining the above features, the update model of each region is represented as M j ={φ shape (M j ),φ texture (M j ),φ temp (M j )}, which contains three types of feature data. This updated model provides accurate multimodal regional features for the next step of object recognition.

[0081] The final output generated by the above steps is the multimodal fusion environment model M, which will become the basic input for object recognition and state analysis in subsequent steps, providing accurate and efficient basic data for the overall process of this patent.

[0082] S2, split a single voxel into multiple object regions to obtain a set of voxels of different objects, denoted as Each of these sets Represents a target object, and then the voxel set Each voxel v in i Perform weighted feature calculation to obtain the object The comprehensive feature vector x j , then the comprehensive feature vector x j Perform small sample learning to build a small sample learning model, and use the small sample learning model to calculate the central feature vector μ of each category c , when an unknown object is detected When , extract its feature vector And with each category center μ c Calculate the distance and place the unknown object Classify it into the category c with the smallest distance * , and finally each identified object category label c j Assign its eigenvector x j , and update the environment model to generate an enhanced environment model M containing the category label C , that is, M C = {v i ,c i}, where c i is the category label of the object corresponding to the voxel.

[0083] Specifically, the input is the multimodal environment model M output in step 1 = {v i}, where v i is a single voxel, which contains the geometric position, RGB color, point cloud depth and temperature characteristics of the object. First, each voxel is divided into multiple object regions to obtain a set of voxels of different objects, recorded as Each of these sets Represents a target object.

[0084] In order to extract the comprehensive features of each object, Each voxel v in i Perform weighted feature calculation. Define regional feature extraction function for:

[0085]

[0086] Among them, xj Representing objects The comprehensive feature vector includes the weighted mean of multimodal features such as RGB color, depth, and temperature. Indicates the object area The number of voxels in w used for mean calculation. RGB 、w Depth and w Temp Represents the weight parameters of RGB, depth and temperature modal features, which are used to adjust the influence of different modal features in the fusion feature. RGB (v i ), P Lidar (v i ) and T IR (v i ) represents the voxel v i RGB color value, lidar depth value and temperature value.

[0087] Through feature extraction function Get the object The comprehensive feature vector x j , used for subsequent category recalculation and matching.

[0088] Furthermore, this step is based on small sample learning to calculate the central features of each type of object. i ,y i}(where y i is the category label), calculate the central feature vector μ of each category c The calculation formula of the category center vector is:

[0089]

[0090] Among them, μ c Represents the central feature vector of category c, which is used to represent the comprehensive features of this type of object. Represents a sample set of category c, containing all feature vectors belonging to category c. Representing a collection The number of samples in Ω used to calculate the mean. c Represents a regular term, which is used to limit the deviation of the category center and reduce overfitting. Ω c It can be set as the variance of each category feature to smooth the category center and increase generalization ability. λ represents the regularization coefficient, which controls the regularization term Ω c The calculated category center μ c It is used to describe the average characteristics of each object category to facilitate similarity matching in subsequent new object recognition.

[0091] Furthermore, for the unknown objects newly detected in the environment model M Extract its feature vector And with each category center μ c Perform distance calculation. Define the improved similarity measurement function d(x new ,μ c ), used to match the category of new objects:

[0092]

[0093] Among them, d(x new ,μ c ) represents the new object x new and the central feature vector μ of category c c The matching distance between them is used to determine the category of the new object. new,j and μ c,j Denotes the new object features x new and category center μ c The component on the jth feature dimension. σ c Represents the discrimination parameter of category c, which is obtained by calculating the average distance between the center vectors of different categories. It is used to prevent category confusion and improve the discrimination between categories.

[0094] The system will new objects Classify it into the category c with the smallest distance * , that is, satisfying c * =argmin c d(x new ,μ c )’s category label.

[0095] Finally, each identified object category label c j Assign its eigenvector x j , and update the environment model to generate an enhanced environment model M containing the category label C , that is, M C = {v i ,c i}, where c i is the category label of the object corresponding to the voxel. Environment model M C It will serve as the basic input for the next step of object state evaluation.

[0096] In this scheme, by introducing regularization terms and category discrimination parameters, the system achieves efficient and accurate object recognition under small sample conditions, making the updated environment model M C It can accurately characterize different objects in the environment, providing reliable support for state analysis and handling strategies in subsequent steps.

[0097] S3, according to the enhanced environment model M CThe stability and state analysis of the object is performed, and the object state characteristic vector is formed according to the results of the stability and state analysis, and then the object state characteristic vector is integrated into the enhanced environment model M C In the enhanced state model M CS ; Wherein, the results of the stability and state analysis include the stability factor S j , dynamic inertia factor Γ j , the enhanced state model M CS Indicated as M CS = {v i ,c i ,S i ,Γ i}.

[0098] Specifically, the output of step 2 is used to enhance the environment model M C = {v i ,c i} as input, where v i represents a single voxel, c i is the category label of the object to which the voxel belongs. Divide the voxel into multiple object regions Each area represents object j. For each object Extract geometric, class and quality characteristics and construct a preliminary characteristic vector z j ={x j ,m j ,c j}, where x j For Objects The geometric center coordinates are denoted by x j =[x j1 ,x j2 ,x j3 ]; m j is the mass of the object, and the volume V is calculated by j and volume density ρ j Get, that is, m j =ρ j ·V j ;c j is the category label of the object, and the subsequent calculation parameters are adjusted according to the object category.

[0099] Furthermore, the stability factor S j Used to quantify the balance of an object during transport. Introduce the object's center of gravity offset Δ COM,j and contact angle θ j As a stability factor, its contribution is adjusted according to the category weight. Stability factor S j The calculation is:

[0100]

[0101] Among them, S j Representing objects The stability factor is in the range of [0,1], the larger the value, the more stable it is; Indicates category c j The adjustment parameters of Δ COM,j and θ j The weight of COM,j Represents the object's center of gravity offset, defined as the geometric center x j With the center of mass x COM,j The distance, Δ COM,j =∥x COM,j -x j ∥;θ j It represents the inclination angle of the object and the angle between it and the ground; ∈ represents a small positive value to prevent the denominator from being zero.

[0102] Furthermore, during the handling process, the inertial response of the object will significantly affect its dynamic stability. Define the dynamic inertia factor Γ j To quantify this characteristic, and combine the mass of the object and the class inertia coefficient for calculation, the formula is as follows:

[0103]

[0104] Among them, Γ j Representing objects The dynamic inertia factor describes its inertia; Indicates category c j The inertia coefficient is adjusted according to the object type; m j Represents the mass of an object, represented by its volume V j and volume density ρ j Calculate; j Represents the object density coefficient, which is calculated based on voxel data and reflects the overall density of the object.

[0105] Furthermore, according to the calculated S j and Γ j , which is combined into the object state characteristic vector s j ={S j ,Γ j ,m j ,c j}. Integrate the state characteristics into the voxel set of each object to obtain the updated environment model M CS = {v i ,c i ,S i ,Γ i}, where S i and Γ iEach voxel corresponds to the state characteristics of the object. Model M CS Provide input for the next step of handling strategy generation.

[0106] Furthermore, the output environment model M CS Contains the category, stability, and dynamic inertia information of each object for the next step of generating the transport path and strategy. j and Γ j Through joint evaluation, the system can generate personalized handling solutions based on the object category and dynamic characteristics to ensure stability and safety during the handling process.

[0107] This step is done by the stability factor S j and the dynamic inertia factor Γ j Comprehensive analysis ensures that the transfer machine can respond to the different characteristics of the object, and the generated model M CS It provides reliable data support for the generation of handling strategies.

[0108] S4, according to the enhanced state model M CS Generate a preliminary handling strategy suitable for each object, wherein the preliminary handling strategy includes the grasping force F j , transport speed v j , maximum acceleration a j and the final path

[0109] Specifically, the input of this step is the enhanced state model M output from step 3 CS = {v i ,c i ,S i ,Γ i}, where v i is a voxel, c i is the category label, S i is the stability factor of the voxel, Γ i is the dynamic inertia factor of the voxel. The model is decomposed into multiple object regions Each object area Represents object j. The system extracts the overall characteristics of object j, including stability S j , inertia factor Γ j 、Mass m j Category c j , used for subsequent strategy generation.

[0110] Furthermore, the gripping force F j An accurate calculation is required based on the type, stability factor and mass of the object to ensure that the object will not slip or be damaged during handling due to improper gripping force. j The calculation formula is as follows:

[0111]

[0112] Among them, F j Representing objects The gripping power. Represents the object category c j The gripping coefficient controls the gripping strength of different types of objects. For example, the gripping coefficient of fragile objects is small, while that of heavy objects is large. j Indicates the mass of an object. j Indicates the stability factor of the object. The smaller the value, the more unstable the object is. Therefore, the grasping force F j Need to be increased appropriately. ∈ represents a small positive value to prevent the denominator from being zero. Ω j represents the enhanced regularization term, reflecting the morphological complexity of the contact surface of the object. If the surface of the object is smooth and the grasping points are limited, Ω j The value of will be larger to increase the accuracy of the grasping force. η represents the regularization coefficient, which adjusts the enhanced regularization term Ω j impact.

[0113] Enhanced regularization term Ω in grasping force calculation j Adjustments are made based on the smoothness and morphological complexity of the object’s surface, ensuring that even objects with irregular surfaces or insufficient contact can be firmly grasped.

[0114] Furthermore, according to the dynamic inertia factor Γ of the object j To determine the transport speed and acceleration. Objects with greater inertia require lower acceleration during transport to avoid unnecessary shaking. Define the transport speed v of the object j and the maximum acceleration a j ,in:

[0115]

[0116] Among them, v j Representing objects The handling speed should be kept low when the inertia is large. j Indicates the acceleration of an object and determines the smoothness of transportation, starting and braking. and They are the adjustment coefficients of speed and acceleration, respectively, which are determined by the type of object. The coefficients of fragile objects are smaller to maintain smooth handling. j Indicates the dynamic inertia factor, which indicates the inertial characteristics of an object. The larger the inertia factor, the faster the speed v j and acceleration a j Proper lowering is required to prevent the object from becoming unstable during acceleration and cornering.

[0117] Among them, the generated speed v j and acceleration aj It will serve as the basic parameter for path planning to ensure a smooth handling process, especially when handling fragile or high-inertia objects, helping to avoid unnecessary shaking or damage.

[0118] Furthermore, path planning is to reduce the shaking of objects caused by turns and obstacles during transportation. Especially for objects with high inertia or low stability, it is more necessary to optimize the path to maintain stability. Suppose the initial path is Define the final path of the object as follows:

[0119]

[0120] in, Representing objects The final transport path adapts to the dynamic characteristics. Represents a preliminary path plan (e.g., a straight line or shortest path). Indicates category c j The path adjustment coefficient reflects the path adjustment requirements of different objects. The path adjustment coefficient of fragile or heavy objects is larger. Δθ represents the change in the object's inclination angle, based on the object's stability S j The tilt protection item generated by the current path angle will prevent sharp turns and vibrations when the tilt is large.

[0121] The system dynamically calculates the path during transportation and automatically adapts to turning angles and obstacle distribution to ensure that the object will not tilt or shake due to unstable path during transportation.

[0122] Furthermore, the force F j , transport speed v j , maximum acceleration a j and the final path These parameters are adaptively adjusted according to the type and state of the object to ensure stability and safety during the handling process. The output handling strategy serves as the input basis for the next step of real-time feedback adjustment, providing a basis for further adjustments according to environmental changes during the handling process.

[0123] In this step, by introducing the grasping force, speed and path planning strategy based on the state model, the system can generate the optimal handling parameters according to the dynamic characteristics of different objects, ensuring the handling efficiency while greatly improving the stability and safety of the objects. CS and handling strategies as input for real-time feedback adjustment in the next step.

[0124] S5. Use the state data of the object collected in real time by the sensors on the transfer machine to generate a feedback vector. At the same time, during the transportation process, the state changes and environmental factors of the object are monitored in real time through the sensors. The transportation parameters are adjusted and fine-tuned according to the feedback information, and updated in real time during the next transportation process of the transfer machine. When the environment or the state of the object changes further, the transfer machine will repeat the above real-time feedback adjustment steps to form a closed-loop control mechanism.

[0125] Specifically, sensors (such as accelerometers, gyroscopes, and force sensors) installed on the transfer machine are used to collect the state data of the object in real time and form a feedback vector f j ={d COM,j ,θ j ,v obs,j}, where d COM,j Representing objects The real-time center of gravity offset indicates the real-time offset between the object's mass center and its geometric center. j Indicates the real-time tilt angle of the object, indicating the tilt of the object during transportation. obs,j Indicates the speed of obstacles detected in the environment, and is used to adjust the handling speed to avoid collisions with moving obstacles.

[0126] Feedback vector f j It will be used as the input of the fine-tuning algorithm and the initial transport strategy parameters Combined, it is used to generate real-time adjusted parameters.

[0127] Furthermore, according to the center of gravity offset d COM,j and the real-time tilt angle θ j Adjust gripping force F j , in order to improve the stability of the object during handling. The dynamic adjustment formula for grasping force is defined as follows:

[0128] F′ j =F j +λ d ·d COM,j +λ θ ·sin(θ j )

[0129] Among them, F′ j Indicates the adjusted gripping force, which is used to correct the gripper's strength in real time. j represents the initial grasping force. d and λ θ are the adjustment coefficients of the center of gravity offset and the tilt angle, respectively, which are used to control the influence of the center of gravity offset and the tilt on the grasping force. j ) represents the sine term of the tilt angle. When the object is tilted, sin(θ j) will increase, thereby increasing the gripping force to prevent slipping.

[0130] By adjusting the gripping force F′ in real time j , the system can better cope with instabilities caused by center of gravity shift or tilt during handling, preventing objects from slipping out of the gripper or accidentally tipping over.

[0131] Furthermore, if a dynamic obstacle (such as a forklift or other mobile equipment) is detected in the environment during the transport process, the system will obs,j To adjust the transport speed v j At the same time, the transport path is fine-tuned through real-time feedback of the tilt angle and path curvature to ensure smooth transport of the object. The adjustment formula for defining the transport speed is as follows:

[0132] v′ j =v j ·exp(-γ·v obs,j )

[0133] Among them, v′ j Indicates the adjusted handling speed to ensure that the object slows down when approaching obstacles. j represents the initial transport speed. γ represents the speed adjustment coefficient, which is used to adjust the influence of obstacle speed on transport speed. obs,j Indicates the speed of the obstacle. When the obstacle approaches, the handling speed will gradually decrease.

[0134] In addition, according to the inclination angle θ of the object j Fine-tune the transport path in real time The path avoids sudden sharp turns and unnecessary direction changes. The final path fine-tuning formula is defined as:

[0135]

[0136] in, Indicates the transport path after adjustment. represents the initial path. κ represents the path adjustment coefficient, which is used to control the influence of the inclination angle on the path smoothness.

[0137] By adjusting the handling speed and path, the system can effectively respond to environmental changes while reducing vibration and instability of objects during handling.

[0138] Furthermore, after dynamic fine-tuning, the handling parameters The feedback is fed back to the handling control module and updated in real time during the next handling process. When the environment or object status changes further, the system will repeat the above real-time feedback adjustment steps to form a closed-loop control mechanism to ensure that the entire handling process is always in a stable state. Finally, the fine-tuned handling strategy is output to ensure the efficient completion of the handling task.

[0139] In this step, through the real-time feedback mechanism, the system can dynamically fine-tune the gripping force, handling speed and path to respond to changes in the object state and environment, ensuring the continuous stability and efficiency of the handling process. The output fine-tuning strategy provides effective support for the closed-loop control of the entire handling process.

[0140] S6. Summarize various parameters and adjustment data during the handling process to generate a data set Will transport data The category data and stability factor in the sample learning model are fed back to update the feature description of the category and update the enhanced state model M. CS Get the updated state model M opt , and finally according to the updated state model M opt Further update the object handling strategy.

[0141] Specifically, collect various parameters and adjustment data during the handling process, and summarize them to generate a data set where d k is the feedback data vector of the k-th transport, expressed as Each d k Contains the following: F′ j Represents the grasping force after real-time adjustment, indicating the final actual force of the grasper. v′ j Indicates the adjusted transport speed, reflecting the speed adjustment in the actual environment. a′ j Indicates the actual acceleration, indicating the actual feedback during acceleration and deceleration. Indicates the actual transport path, combined with path fine-tuning to avoid obstacles. j Represents the stability factor of an object, which is used to reflect the state of the object. Γ j It represents the dynamic inertia factor of an object and the sensitivity of the object to dynamic response.

[0142] These feedback data will provide statistical basis in future tasks and provide data support for optimizing model parameters and generating more accurate strategies.

[0143] Furthermore, the object classification and recognition module will transport data The category data and stability factor in are fed back to the small sample learning model. By calculating the new category center vector μ′c , the model can gradually update the feature description of each category. The category center vector update formula is defined as follows:

[0144]

[0145] Among them, μ′ c Represents the updated category center vector, which is used for category determination in subsequent tasks. α represents the update coefficient, which controls the impact of new data on the category center. It usually takes a small value (such as α = 0.1) so that the model gradually adapts to new samples. The feedback sample set representing category c is composed of the object features collected in the handling task. This update formula allows the category center to gradually adapt to new data, especially when the environment or object characteristics change, and can automatically adjust the generalization ability of the category classification model.

[0146] Furthermore, by analyzing the stability factor S during transportation j and dynamic inertia factor Γ j relationship, using historical data To update the weight parameters in the model and (Stability and inertia related parameters respectively). Define the parameter update formula as:

[0147]

[0148] in, and Represents the updated stability and inertia related parameters. η represents the learning rate, which controls the speed of parameter update and usually takes a small value (e.g. η=0.01). S k and Γ k represents the stability and inertia feedback value in the kth task. n represents the cumulative number of tasks.

[0149] With these updates, the system can automatically adjust the parameters of the state model, allowing for more accurate assessment of object states in future handling tasks, thereby improving the stability and reliability of the entire system.

[0150] Furthermore, after each transport task is completed, the feedback path adjustment parameter Δθ is k The data of speed, acceleration, etc. are used as experience accumulation to optimize the handling strategy. Define the fine-tuning strategy of the optimized handling path as follows:

[0151]

[0152] in, Represents the optimized transport path, which automatically tends to a smooth path in future tasks.k It represents the path tilt adjustment in the historical tasks, and the adaptive changes in the transport path are obtained through accumulation.

[0153] By superimposing the mean of all path-adjusted data in the historical data on the initial path In fact, the system can gradually optimize the adaptability of the path and reduce the need for additional path adjustments during the transportation process.

[0154] Furthermore, the updated model M opt Will replace the original state model M CS , including optimized category centers, parameter updates, and policy path optimization. The model is used as input for object recognition, state evaluation, and handling strategy generation in the next task, achieving dynamic learning and self-optimization capabilities.

[0155] By adjusting the system parameters and models after the task is completed, this step provides a more reliable and intelligent decision-making basis for future handling tasks. The self-optimization mechanism of the model can improve the adaptability of the transfer machine in a dynamic environment, gradually achieving a dual improvement in handling efficiency and stability.

[0156] like Figure 2 As shown, in another embodiment of the present invention, an intelligent automatic identification system for a transfer machine is provided, the system comprising:

[0157] The multimodal data acquisition unit 1011 is used to install multiple sensors on the transfer machine to collect multimodal data, and generate an environment model M in the form of 3D voxels according to the multimodal data, where M = {v i},v i Represents a single voxel, and optimizes and segmented the environment model M to obtain the segmented sub-regions, and then extracts regional features from each sub-region to construct the regional model M j ; Wherein, the multimodal data includes RGB image data, point cloud data of laser radar and temperature data of infrared sensor, the single voxel includes RGB color value, point cloud depth value and temperature value, and the features include shape feature, texture feature and temperature feature;

[0158] The single voxel segmentation unit 1012 is used to segment a single voxel into multiple object regions to obtain a set of voxels of different objects, denoted as Each of these sets Represents a target object, and then the voxel set Each voxel v in i Perform weighted feature calculation to obtain the object The comprehensive feature vector x j , then the comprehensive feature vector x jPerform small sample learning to build a small sample learning model, and use the small sample learning model to calculate the central feature vector μ of each category c , when an unknown object is detected When , extract its feature vector And with each category center μ c Calculate the distance and place the unknown object Classify it into the category c with the smallest distance * , and finally each identified object category label c j Assign its eigenvector x j , and update the environment model to generate an enhanced environment model M containing the category label C , that is, M C = {v i ,c i}, where c i is the category label of the object corresponding to the voxel;

[0159] The environment enhancement unit 1013 is used to enhance the environment model M C The stability and state analysis of the object is performed, and the object state characteristic vector is formed according to the results of the stability and state analysis, and then the object state characteristic vector is integrated into the enhanced environment model M C In the enhanced state model M CS ; Wherein, the results of the stability and state analysis include the stability factor S j , dynamic inertia factor Γ j , the enhanced state model M CS Indicated as M CS = {v i ,c i ,S i ,Γ i};

[0160] The preliminary handling strategy generating unit 1014 is used to generate a preliminary handling strategy according to the enhanced state model M CS Generate a preliminary handling strategy suitable for each object, wherein the preliminary handling strategy includes the grasping force F j , transport speed v j , maximum acceleration a j and the final path

[0161] The real-time data acquisition unit 1015 is used to use the state data of the object collected in real time by the sensors on the transfer machine to generate a feedback vector. At the same time, during the handling process, the sensors are used to monitor the state changes and environmental factors of the object in real time, and the handling parameters are adjusted and fine-tuned according to the feedback information. The data are updated in real time during the next handling process of the transfer machine. When the environment or the state of the object changes further, the transfer machine will repeat the above-mentioned real-time feedback adjustment steps to form a closed-loop control mechanism.

[0162] The transport strategy optimization unit 1016 is used to aggregate various parameters and adjustment data during the transport process to generate a data set Will transport data The category data and stability factor in the sample learning model are fed back to update the feature description of the category and update the enhanced state model M. CS Get the updated state model M opt , and finally according to the updated state model M opt Further update the object handling strategy.

[0163] It should be noted that the workflow described above is merely illustrative and does not limit the scope of protection of the present invention. In practical applications, technicians in this field can select part or all of them according to actual needs to achieve the purpose of the present embodiment, and no limitation is made here.

[0164] In addition, for technical details that are not described in detail in this embodiment, reference can be made to the parameter operation method provided in any embodiment of the present invention, and will not be repeated here.

[0165] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0166] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0167] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as a read-only memory / random access memory, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0168] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. An intelligent automatic identification method for a transfer machine, characterized in that: The method comprises: S1. Install multiple sensors on the transfer machine to collect multimodal data, and generate an environmental model M in the form of 3D voxels based on the multimodal data, where M = {v i },v i Represents a single voxel, and optimizes and segmented the environment model M to obtain the segmented sub-regions, and then extracts regional features from each sub-region to construct the regional model M j ; Wherein, the multimodal data includes RGB image data, point cloud data of laser radar and temperature data of infrared sensor, the single voxel includes RGB color value, point cloud depth value and temperature value, and the features include shape feature, texture feature and temperature feature; S2, split a single voxel into multiple object regions to obtain a set of voxels of different objects, denoted as Each of these sets Represents a target object, and then the voxel set Each voxel v in i Perform weighted feature calculation to obtain the object The comprehensive feature vector x j , then the comprehensive feature vector x j Perform small sample learning to build a small sample learning model, and use the small sample learning model to calculate the central feature vector μ of each category c , when an unknown object is detected When , extract its feature vector And with each category center μ c Calculate the distance and place the unknown object Classify it into the category c with the smallest distance * , and finally each identified object category label c j Assign its eigenvector x j , and update the environment model to generate an enhanced environment model M containing category labels C , that is, M C = {v i ,c i }, where c i is the category label of the object corresponding to the voxel; S3, according to the enhanced environment model M C The stability and state analysis of the object is performed, and the object state characteristic vector is formed according to the results of the stability and state analysis, and then the object state characteristic vector is integrated into the enhanced environment model M C In the enhanced state model M CS ; Wherein, the results of the stability and state analysis include the stability factor S j , dynamic inertia factor Γ j , the enhanced state model M CS Indicated as M CS = {v i ,c i ,S i ,Γ i }; S4, according to the enhanced state model M CS Generate a preliminary handling strategy suitable for each object, wherein the preliminary handling strategy includes the grasping force F j , transport speed v j , maximum acceleration a j and the final path S5. Use the state data of the object collected in real time by the sensor on the transfer machine to generate a feedback vector. At the same time, during the handling process, the state changes and environmental factors of the object are monitored in real time through the sensor. The handling parameters are adjusted and fine-tuned according to the feedback information, and updated in real time during the next handling process of the transfer machine. When the environment or the state of the object changes further, the transfer machine will repeat the above real-time feedback adjustment steps to form a closed-loop control mechanism. S6. Summarize various parameters and adjustment data during the handling process to generate a data set Will transport data The category data and stability factor in the sample learning model are fed back to update the feature description of the category and update the enhanced state model M. CS Get the updated state model M opt , and finally according to the updated state model M opt Further update the object handling strategy.

2. The intelligent automatic identification method for a transfer machine according to claim 1, characterized in that: In the S1, for the RGB image data I RGB , using image enhancement and segmentation methods to remove background impurities and improve the clarity of the edge of the target object; Lidar Sparse processing is performed to remove background noise points and retain key points on the surface of the object; for infrared temperature data T IR , perform normalization processing based on temperature threshold, unify the temperature range, and reduce the data distribution differences caused by different sensors; finally, fuse the preprocessed data into a multimodal point set D f .

3. The intelligent automatic identification method for a transfer machine according to claim 1, characterized in that: The optimization and regional segmentation of the environment model M are performed to obtain the segmented sub-regions, and then the regional features of each sub-region are extracted to construct the regional model M. j , specifically including: An adaptive voxel optimization algorithm is used to downsample high-density areas: At each voxel v i , calculate its local density ρ i , defined as the number of surrounding voxels with a density greater than a set threshold ρ threshold Downsample the voxel groups to improve computational efficiency; The environmental model M is divided into independent regions using an adaptive segmentation method based on spatial connectivity and temperature distribution, and each segmented sub-region M is obtained. j ; For each segmented sub-region M j , extract regional features; the shape feature is calculated based on the point cloud contour and surface curvature, the texture feature is based on the color gradient and edge directionality analysis, and the temperature feature is calculated based on the thermal distribution pattern of infrared data. Combining the above features, the update model of each region is represented as M j ={φ shape (M j ),φ texture (M j ),φ temp (M j )}, where the shape feature, texture feature and temperature feature are φ shape (M j ),φ texture (M j ) and φ temp (M j ).

4. The intelligent automatic identification method for a transfer machine according to claim 1, characterized in that: The voxel set Each voxel v in i Perform weighted feature calculation to obtain the object The comprehensive feature vector x j , calculated as follows: Among them, x j Representing objects The comprehensive feature vector includes the weighted mean of multimodal features such as RGB color, depth, and temperature; Indicates the object area The number of voxels in the mean value, w RGB 、w Depth and w Temp Respectively represent the weight parameters of RGB, depth and temperature modal features, which are used to adjust the influence of different modal features in the fusion feature; I RGB (v i ), P Lidar (v i ) and T IR (v i ) represent voxel v i RGB color value, LiDAR depth value and temperature; The comprehensive feature vector x j Perform small sample learning to build a small sample learning model, and use the small sample learning model to calculate the central feature vector μ of each category c , calculated as follows: Among them, μ c Represents the central feature vector of category c, which is used to represent the comprehensive features of objects of this type; Represents a sample set of category c, including all feature vectors belonging to category c; Representing a collection The number of samples in the function used to calculate the mean; Ω c represents the regularization term, which is used to limit the deviation of the category center and reduce overfitting; λ represents the regularization coefficient, which controls the regularization term Ω c The size of the impact.

5. The intelligent automatic identification method for a transfer machine according to claim 4, characterized in that: When an unknown object is detected When , the category of the new object is matched and calculated as follows: Among them, d(x new ,μ c ) represents the new object x new and the central feature vector μ of category c c The matching distance between them is used to determine the category of the new object; new,j and μ c,j Represent the new object features x new and category center μ c The component on the jth feature dimension; σ c Represents the discrimination parameter of category c, which is obtained by calculating the average distance between the center vectors of different categories to prevent category confusion and improve the discrimination between categories; The unknown object Classify it into the category c with the smallest distance * , that is, satisfying c * =argmin c d(x new ,μ c )’s category label.

6. The intelligent automatic identification method for a transfer machine according to claim 1, characterized in that: The enhanced environment model M C The stability and state analysis of the object includes the following steps: Split the voxels into i For multiple object areas Each area Represents object j, for each object Extract geometric, class and quality characteristics and construct a preliminary characteristic vector z j ={x j ,m j ,c j }, where x j For Objects The geometric center coordinates are denoted by x j =[x j1 ,x j2 ,x j3 ],m j is the mass of the object, and the volume V is calculated by j and volume density ρ j Get, that is, m j =ρ j ·V j , c j is the category label of the object; Introduce the object's center of gravity offset Δ COM,j and contact angle θ j Constructing stability factor S as a stability influencing factor j , and adjust its contribution according to the category weight, and define the dynamic inertia factor Γ according to the inertia of the object during transportation j To quantify the inertia factor Γ j The characteristics of the object are combined with the mass and class inertia coefficient to calculate, which is expressed as follows: C j =c cj ·m j ·r j Among them, S j Representing objects The stability factor is in the range of [0,1], with the largest value being the most stable; Indicates category c j The adjustment parameters of Δ COM,j and θ j The weight of COM,j Represents the object's center of gravity offset, defined as the geometric center x j With the center of mass x COM,j The distance, Δ COM,j =∥x COM,j -x j ∥;θ j Represents the inclination angle of the object, the angle between the object and the ground contact surface; ∈ represents a small positive value to prevent the denominator from being zero; Γ j Representing objects The dynamic inertia factor describes its inertia; Indicates category c j The inertia coefficient is adjusted according to the object type; m j Represents the mass of an object, represented by its volume V j and volume density ρ j Calculate; j Represents the object density coefficient, which is calculated based on voxel data and reflects the overall density of the object.

7. The intelligent automatic identification method for a transfer machine according to claim 1, characterized in that: The gripping force F j The calculation formula is as follows: Among them, F j Representing objects The gripping force, Represents the object category c j The grasping coefficient controls the grasping strength of different types of objects; m j Indicates the mass of an object; S j Represents the stability factor of the object; ∈ represents a small positive value that prevents the denominator from being zero; Ω j represents the enhanced regularization term, reflecting the morphological complexity of the contact surface of the object; η represents the regularization coefficient, adjusting the enhanced regularization term Ω j The impact of The transport speed v j and the maximum acceleration a j , calculated as follows: Among them, v j Representing objects The handling speed; a j Indicates the acceleration of an object, which determines the smoothness of transport, starting and braking; and They are the adjustment coefficients for speed and acceleration, respectively, which are determined by the object category; The final path The calculation is as follows: Among them, P j Represents object O j The final transport path is adapted to the dynamic characteristics; P j 0 represents the preliminary path planning; Indicates category c j The path adjustment coefficient reflects the path adjustment requirements of different objects; Δθ represents the change in the object's inclination angle, based on the object's stability S j and the tilt protection item generated by the current path angle.

8. The intelligent automatic identification method for a transfer machine according to claim 7, characterized in that: The state data of the object collected in real time by the sensor on the transfer machine includes the object The real-time center of gravity offset d COM,j 、The real-time tilt angle θ of the object j and the speed v of obstacles detected in the environment obs,j , then the feedback vector is f j ={d COM,j ,θ j ,v obs,j }; According to the center of gravity offset d COM,j and the real-time tilt angle θ j Adjust gripping force F j , which is expressed as follows: F′ j =F j +λ d ·d COM,j +λ θ ·sin(θ j ) Among them, F′ j Represents the adjusted grasping force, which is used to correct the grasper's strength in real time; d and λ θ are the adjustment coefficients of the center of gravity offset and the tilt angle, respectively, which are used to control the influence of the center of gravity offset and the tilt on the grasping force; sin(θ j ) represents the sine term of the tilt angle. When the object is tilted, sin(θ j ) will increase, thereby increasing the gripping force to prevent slipping; According to the obstacle speed v obs,j To adjust the transport speed v j At the same time, the transport path is fine-tuned through the real-time feedback of the inclination angle and path curvature, as shown below: v′ j =v j ·exp(-γ·v obs,j ) Among them, v′ j It represents the adjusted handling speed, ensuring that the object slows down when approaching an obstacle; γ represents the speed adjustment coefficient, which is used to adjust the degree of influence of the obstacle speed on the handling speed; According to the inclination angle θ of the object j Real-time fine-tuning of the transport path P j , so that the path avoids sudden sharp turns and unnecessary direction changes, is calculated as follows: P′ j =P j +k·i j Among them, P′ j represents the adjusted transport path; κ represents the path adjustment coefficient, which is used to control the influence of the inclination angle on the path smoothness.

9. The intelligent automatic identification method for a transfer machine according to claim 1, characterized in that: The data set for in For the The feedback data vector of the transport is expressed as Among them, each Including real-time adjusted grip Adjusted transport speed v′ j , actual acceleration a′ j , Actual transport path The stability factor S of an object j and the object's dynamic inertia factor Γ j .

10. An intelligent automatic identification system for a transfer machine, characterized in that: The system comprises: The multimodal data acquisition unit is used to install multiple sensors on the transfer machine to collect multimodal data, and generate an environmental model M in the form of 3D voxels based on the multimodal data, where M = {v i },v i Represents a single voxel, and optimizes and segmented the environment model M to obtain the segmented sub-regions, and then extracts regional features from each sub-region to construct the regional model M j ; Wherein, the multimodal data includes RGB image data, point cloud data of laser radar and temperature data of infrared sensor, the single voxel includes RGB color value, point cloud depth value and temperature value, and the features include shape feature, texture feature and temperature feature; A single voxel segmentation unit is used to segment a single voxel into multiple object regions to obtain a set of voxels of different objects, denoted as Each of these sets Represents a target object, and then the voxel set Each voxel v in i Perform weighted feature calculation to obtain the object The comprehensive feature vector x j , then the comprehensive feature vector x j Perform small sample learning to build a small sample learning model, and use the small sample learning model to calculate the central feature vector μ of each category c , when an unknown object is detected When , extract its feature vector And with each category center μ c Calculate the distance and place the unknown object Classify it into the category c with the smallest distance * , and finally each identified object category label c j Assign its eigenvector x j , and update the environment model to generate an enhanced environment model M containing the category label C , that is, M C = {v i ,c i }, where c i is the category label of the object corresponding to the voxel; The environment enhancement unit is used to enhance the environment model M C The stability and state analysis of the object is performed, and the object state characteristic vector is formed according to the results of the stability and state analysis, and then the object state characteristic vector is integrated into the enhanced environment model M C In the enhanced state model M CS ; Wherein, the results of the stability and state analysis include the stability factor S j , dynamic inertia factor Γ j , the enhanced state model M CS Indicated as M CS = {v i ,c i ,S i ,Γ i }; The preliminary handling strategy generation unit is used to generate a preliminary handling strategy according to the enhanced state model M CS Generate a preliminary handling strategy suitable for each object, wherein the preliminary handling strategy includes the grasping force F j , transport speed v j , maximum acceleration a j and the final path The real-time data acquisition unit is used to use the state data of the object collected in real time by the sensors on the transfer machine to generate a feedback vector. At the same time, the state changes and environmental factors of the object are monitored in real time through the sensors during the handling process. The handling parameters are adjusted and fine-tuned according to the feedback information, and updated in real time during the next handling process of the transfer machine. When the environment or the state of the object changes further, the transfer machine will repeat the above real-time feedback adjustment steps to form a closed-loop control mechanism. The handling strategy optimization unit is used to aggregate various parameters and adjustment data during the handling process to generate a data set Will transport data The category data and stability factor in the sample learning model are fed back to update the feature description of the category and update the enhanced state model M. CS Get the updated state model M opt , and finally according to the updated state model M opt Further update the object handling strategy.

Images