An intelligent automatic recognition method and system for a transfer machine

Abstract

The present invention provides an intelligent automatic recognition method and system for a transfer machine. The method includes: installing a variety of sensors on the transfer machine to collect multi-modal data; dividing a single voxel into multiple object regions to obtain voxel sets of different objects; analyzing the stability and state of the objects according to the enhanced environment model to form an object state characteristic vector, and integrating the object state characteristic vector into the enhanced environment model; generating a preliminary handling strategy suitable for each object according to the enhanced state model; using the state data of the objects collected in real time by the sensors on the transfer machine to generate a feedback vector, and adjusting and fine-tuning the handling parameters according to the feedback information; feeding back various parameters and adjustment data during the handling process, as well as the category data and stability factors in the handling data, back to the small-sample learning model to update the feature description of the category. The present invention overcomes the limitations of traditional transfer machines in a changing environment and can cope with the challenges of handling diverse objects.

Description

Technical Field

[0001] The present invention belongs to the field of intelligent automatic identification, and particularly relates to an intelligent automatic identification analysis method and system for a transfer vehicle. Background Art

[0002] In modern industrial and logistics automation, transfer vehicles (or automatic guided vehicles) have become core equipment for material handling and are widely used in warehouses, production workshops, and logistics centers. These automated devices typically rely on sensors, cameras, and algorithms to identify and transport objects in order to improve production and transportation efficiency. However, existing transfer vehicle systems still face many technical bottlenecks in practical applications, especially when dealing with complex scenarios and diverse objects, showing insufficient adaptability.

[0003] In current technologies, the object recognition function of most transfer vehicles relies on single or a few sensor data (such as cameras or lidar), and is only suitable for handling fixed or stable objects. In such a technical mode, the transfer vehicle system usually pre-sets recognition and handling parameters to generate fixed grasping and handling strategies for known objects. However, when the environment is complex and changeable, and the object types are numerous and diverse in shape, the transfer vehicle system often has difficulty quickly identifying new object characteristics, resulting in handling strategies that do not match the actual situation, thereby reducing the efficiency and stability of the system. In addition, traditional transfer vehicles are unable to flexibly adjust handling strategies in real time when facing a dynamically changing environment. For example, the handling object may shift due to vibration during transportation, or obstacles may suddenly appear around, which will lead to handling failure or even object damage. Even though some high-end transfer vehicles integrate multiple sensors, due to the lack of effective multi-modal data fusion and adaptive algorithms, the system still cannot fully utilize multi-dimensional data for object state analysis and real-time strategy optimization.

[0004] In addition to the flexibility issue of recognition and handling strategies, current transfer vehicle systems also have another significant defect, that is, they usually require a large amount of labeled data to train and identify new objects. When encountering unseen new objects, it is difficult for the system to generate reliable recognition and handling strategies in a short time, resulting in a significant reduction in work efficiency. In practical applications, especially when facing frequent changes and diverse handling requirements in a complex environment, traditional transfer vehicle systems lack the ability of self-learning and rapid adaptation to new objects. And obtaining sufficient labeled data is not only time-consuming but also not cost-effective. Therefore, how to develop an adaptive transfer vehicle system that can quickly identify new objects and flexibly adjust when the environment changes has become an important challenge in current intelligent handling technologies. Summary of the Invention

[0005] The object of the present invention is to provide an intelligent automatic recognition and analysis method and system for a transfer machine. Through a dynamic handling algorithm of adaptive scene perception, multi-object state evaluation, and few-shot learning, the limitations of traditional transfer machines in a changing environment are overcome, and the challenges of handling diverse objects are addressed.

[0006] To achieve the above object, in the first aspect of the present invention, an intelligent automatic recognition method for a transfer machine is provided. The method includes:

[0007] S1. Install a variety of sensors on the transfer machine to collect multi-modal data, generate an environmental model M in the form of 3D voxels according to the multi-modal data, M = {v i}, where v i represents a single voxel, and optimize and regionally segment the v in the environmental model M to obtain each sub-region after segmentation, and then extract regional features from each sub-region to construct a regional model M j ; wherein, the multi-modal data includes RGB image data, point cloud data of a lidar, and temperature data of an infrared sensor. The single voxel includes an RGB color value, a point cloud depth value, and a temperature value. The features include shape features, texture features, and temperature features;

[0008] S2. Segment a single voxel into multiple object regions to obtain a voxel set of different objects, denoted as where each set represents a target object. Then, calculate the weighted features of each voxel v in the voxel set i to obtain the comprehensive feature vector x of the object j . Then, perform few-shot learning on the comprehensive feature vector x j to construct a few-shot learning model, and use the few-shot learning model to calculate the central feature vector μ c of each category. When an unknown object is detected, extract its feature vector and calculate the distance from it to the central μ c of each category. Classify the unknown object into the category c * with the smallest distance from it. Finally, assign the category label c j of each recognized object to its feature vector x j , and update the environmental model to generate an enhanced environmental model M C containing category labels, that is, M C = {v i , c i}, where c i is the category label of the object corresponding to the voxel;

[0009] S3. According to the enhanced environment model M C perform stability and state analysis on the object, form an object state characteristic vector based on the results of the stability and state analysis, and then integrate the object state characteristic vector into the enhanced environment model M C to obtain an enhanced state model M CS ; wherein, the results of the stability and state analysis include a stability factor S j , a dynamic inertia factor Γ j , and the enhanced state model M CS is expressed as M CS ={v i , c i , S i , Γ i};

[0010] S4. Generate a preliminary handling strategy suitable for each object according to the enhanced state model M CS , wherein the preliminary handling strategy includes a grasping force F j , a handling speed v j , a maximum acceleration a j and a final path

[0011] S5. Use the state data of the object collected in real time by the sensors on the transfer machine to generate a feedback vector. At the same time, during the handling process, monitor the state changes of the object and environmental factors in real time through the sensors, adjust and fine-tune the handling parameters according to the feedback information, and update them in real time during the subsequent handling process of the transfer machine. When the environment or the object state changes further, the transfer machine will repeat the above real-time feedback adjustment steps to form a closed-loop control mechanism;

[0012] S6. Summarize the various parameters and adjustment data during the handling process to generate a data set Feed back the category data and stability factor in the handling data to the small sample learning model to update the feature description of the category, and at the same time update the enhanced state model M CS to obtain an updated state model M opt , and finally further update the object handling strategy according to the updated state model M opt .

[0013] Furthermore, in the S1, for the RGB image data I RGB , use image enhancement and segmentation methods to remove background impurities and improve the clarity of the edges of the target object; perform sparsification processing on the lidar point cloud data P Lidar to remove background noise points and retain the key points on the object surface; for the infrared temperature data T IR, perform normalization based on temperature thresholds to unify the temperature range and reduce the data distribution differences brought by different sensors; finally, fuse the preprocessed data into a multi-modal point set D f .

[0014] An intelligent automatic recognition method for a transfer machine, characterized in that the environment model M is optimized and regionally segmented to obtain each segmented sub-region, and then region feature extraction is performed on each sub-region to construct a region model M j , specifically including:

[0015] Adopt an adaptive voxel optimization algorithm to downsample the high-density area:

[0016] In each voxel v i , calculate its local density ρ i , defined as the number of surrounding voxels, and downsample the voxel group with a density greater than the set threshold ρ threshold to improve the calculation efficiency;

[0017] Use an adaptive segmentation method based on spatial connectivity and temperature distribution to divide the environment model M into independent regions to obtain each segmented sub-region M j ;

[0018] For each segmented sub-region M j , perform region feature extraction; among them, the shape feature is calculated based on the point cloud contour and surface curvature, the texture feature is analyzed based on color gradient and edge directionality, and the temperature feature is calculated based on the thermal distribution pattern of infrared data. Combining the above features, represent the updated model of each region as M j ={φ shape (M j ), φ texture (M j ), φ temp (M j )}, where the shape feature, texture feature, and temperature feature are φ shape (M j ), φ texture (M j ), and φ temp (M j ) respectively.

[0019] Furthermore, for each voxel v in the voxel set i , perform weighted feature calculation to obtain the comprehensive feature vector x of the object j , and the calculation is as follows:

[0020]

[0021] Among them, x j represents the comprehensive feature vector of the object , including the weighted mean of multi-modal features such as RGB color, depth, and temperature; represents the number of voxels in the object region , which is used for mean calculation; w RGB , w Depth and w Temp respectively represent the weight parameters of RGB, depth, and temperature modal features, which are used to adjust the influence of different modal features in the fused features; I RGB (v i ), P Lidar (v i ) and T IR (v i ) respectively represent the RGB color value, lidar depth value, and temperature of the voxel v i ;

[0022] Performing few-shot learning on the comprehensive feature vector x j to construct a few-shot learning model, and using the few-shot learning model to calculate the central feature vector μ c of each category, the calculation is as follows:

[0023]

[0024] where μ c represents the central feature vector of category c, which is used to represent the comprehensive features of the objects in this category; represents the sample set of category c, which contains all feature vectors belonging to category c; represents the set the number of samples in, which is used to calculate the mean; Ω c represents the regularization term, which is used to limit the deviation of the category center and reduce overfitting; λ represents the regularization coefficient, which controls the influence size of the regularization term Ω c ;

[0025] Furthermore, when an unknown object is detected, match the category of the new object, and the calculation is as follows:

[0026]

[0027] where d(x new , μ c ) represents the matching distance between the new object x new and the central feature vector μ c of category c, which is used to determine the category of the new object; x new,j and μ c,j respectively represent the new object feature x newand the class center μ c The component in the j-th feature dimension; σ c Represents the discrimination parameter of class c, obtained by calculating the average distance between different class center vectors, used to prevent class confusion and improve the discrimination between classes;

[0028] Classify the unknown object into the class c with the smallest distance from it * , that is, satisfying c * = argmin c d(x new , μ c ) for the class label.

[0029] Furthermore, the stability and state analysis of the object according to the enhanced environment model M C is specifically carried out according to the following steps:

[0030] Voxel segmentation v i is divided into multiple object regions Each region represents object j. For each object extract geometric, class, and mass characteristics, and construct a preliminary characteristic vector z j = {x j , m j , c j}, where x j is the geometric center coordinates of the object , expressed as x j = [x j1 , x j2 , x j3 , m j is the mass of the object, obtained by calculating the object volume V j and the bulk density ρ j , that is, m j = ρ j ·V j , c j is the class label of the object;

[0031] Introduce the center of gravity offset Δ COM,j and the contact angle θ j as stability influence factors to construct the stability factor S j , and adjust its contribution according to the class weight. At the same time, define the dynamic inertia factor Γ j according to the inertia of the object during the handling process to quantify the characteristics of the inertia factor Γ j and calculate it in combination with the object mass and the class inertia coefficient, as shown below:

[0032]

[0033] Among them, S j represents the stability factor of the object with a range of [0, 1], where the largest value indicates the highest stability; represents the adjustment parameter for category c j which controls the weights of Δ COM,j and θ j ; Δ COM,j represents the offset of the object's center of gravity, defined as the distance between the geometric center x j and the mass center x COM,j , that is, Δ COM,j = ∥x COM,j - x j ∥; θ j represents the tilt angle of the object, which is the angle with the ground contact surface; ∈ represents a small positive value to prevent the denominator from being zero; Γ j represents the dynamic inertia factor of the object describing its inertia magnitude; γ cj represents the inertia coefficient for category c j which adjusts the inertia sensitivity according to the object category; m j represents the mass of the object, calculated from the volume V j and the volume density ρ j ; ρ j represents the object density coefficient, calculated based on voxel data and reflecting the overall density of the object.

[0034] Furthermore, the calculation formula for the grasping force F j is as follows:

[0035]

[0036] Among them, F j represents the grasping force of the object , represents the grasping coefficient for object category c j which controls the grasping strength of objects of different categories; m j represents the mass of the object; S j represents the stability factor of the object; ∈ represents a small positive value to prevent the denominator from being zero; Ω j represents the enhanced regularization term, reflecting the morphological complexity of the object contact surface; η represents the regularization coefficient, adjusting the influence of the enhanced regularization term Ω j ;

[0037] The handling speed v j and the maximum acceleration a j are calculated as follows:

[0038]

[0039] Among them, v jIndicates the object 's handling speed; a j Indicates the acceleration of the object, which determines the smoothness of handling start and braking; and Are the adjustment coefficients of speed and acceleration respectively, determined by the object category;

[0040] The final path Is calculated as follows:

[0041]

[0042] Where, P j Indicates the final handling path of the object O j , adapting to dynamic characteristics; Indicates the preliminary path planning; Indicates the path adjustment coefficient of category c j , reflecting the path adjustment requirements of different objects; Δθ represents the change in the object's tilt angle, which is an inclination protection term generated based on the object's stability S j And the current path turning angle.

[0043] Furthermore, the state data of the object collected in real time by the sensors on the transfer machine includes the object 's real-time center of gravity offset d COM,j , the object's real-time tilt angle θ j And the speed v of the obstacle detected in the environment obs,j , then the feedback vector is f j ={d COM,j ,θ j ,v obs,j};

[0044] According to the center of gravity offset d COM,j And the real-time tilt angle θ j Adjust the grasping force F j , which is expressed as follows:

[0045] F′ j =F j +λ d ·d COM,j +λ θ ·sin(θ j )

[0046] Where, F′ j Indicates the adjusted grasping force, used to correct the force of the gripper in real time; λ d And λ θ Are the adjustment coefficients of the center of gravity offset and tilt angle respectively, used to control the influence of the center of gravity offset and tilt on the grasping force; sin(θ j)The sine term representing the tilt angle. When the object is tilted, sin(θ j )will increase, thereby increasing the grasping force to prevent slipping;

[0047] Adjust the handling speed v obs,j according to the obstacle speed v j , and at the same time, finely adjust the handling path through the tilt angle and path curvature feedback in real time, as follows:

[0048] v′ j =v j ·exp(-γ·v obs,j )

[0049] where v′ j represents the adjusted handling speed to ensure that the object decelerates when approaching the obstacle; γ represents the speed adjustment coefficient, which is used to adjust the influence degree of the obstacle speed on the handling speed;

[0050] Fine-tune the handling path P j in real time according to the tilt angle θ j of the object, so that the path avoids sudden sharp turns and unnecessary direction changes, and the calculation is as follows:

[0051] P′ j =P j +κ·θ j

[0052] where P′ j represents the adjusted handling path; κ represents the path adjustment coefficient, which is used to control the influence of the tilt angle on the path smoothness.

[0053] Furthermore, the data set is where is the feedback data vector for the th handling, expressed as where each includes the grasping force adjusted in real time s the adjusted handling speed v′ j the actual acceleration a′ j the actual handling path the stability factor S of the object j and the dynamic inertia factor Γ of the object j .

[0054] In another aspect of the present invention, an intelligent automatic recognition system for a transfer machine is provided. The system includes:

[0055] Multi-modal data acquisition unit, which is used to install multiple sensors on a transfer machine to collect multi-modal data, generate an environmental model M in the form of 3D voxels according to the multi-modal data, M = {v i}, where v i represents a single voxel, and optimizes and regionally segments the environmental model M to obtain each sub-region after segmentation, and then extracts regional features from each sub-region to construct a regional model M j ; among them, the multi-modal data includes RGB image data, point cloud data of lidar, and temperature data of infrared sensors, the single voxel contains RGB color values, point cloud depth values, and temperature values, and the features include shape features, texture features, and temperature features;

[0056] Single voxel segmentation unit, which is used to segment a single voxel into multiple object regions to obtain a voxel set of different objects, denoted as where each set represents a target object, and then calculates the weighted features of each voxel v in the voxel set i to obtain the comprehensive feature vector x of the object j , and then conducts few-shot learning on the comprehensive feature vector x j to construct a few-shot learning model, uses the few-shot learning model to calculate the central feature vector μ c of each category. When an unknown object is detected, extracts its feature vector and calculates the distance from the central μ c of each category. Classifies the unknown object into the category c * with the smallest distance from it. Finally, assigns the category label c j of each recognized object to its feature vector x j , and updates the environmental model to generate an enhanced environmental model M C , that is, M C = {v i , c i}, where c i is the category label of the object corresponding to the voxel;

[0057] Environmental enhancement unit, which is used to analyze the stability and state of the object according to the enhanced environmental model M C , form an object state characteristic vector according to the results of the stability and state analysis, and then integrate the object state characteristic vector into the enhanced environmental model M C to obtain an enhanced state model M CS ; among them, the results of the stability and state analysis include a stability factor S j , a dynamic inertia factor Γj , the enhanced state model M CS is represented as M CS = {v i , c i , S i , Γ i};

[0058] The preliminary handling strategy generation unit is used to generate a preliminary handling strategy suitable for each object according to the enhanced state model M CS , where the preliminary handling strategy includes the grasping force F j , the handling speed v j , the maximum acceleration a j and the final path

[0059] The real-time data acquisition unit is used to generate a feedback vector by using the state data of the object collected in real time by the sensors on the transfer vehicle. At the same time, during the handling process, the state changes of the object and environmental factors are monitored in real time by the sensors. According to the feedback information, the handling parameters are adjusted and fine-tuned, and are updated in real time during the next handling process of the transfer vehicle. When the environment or the state of the object changes further, the transfer vehicle will repeat the above real-time feedback adjustment steps to form a closed-loop control mechanism;

[0060] The handling strategy optimization unit is used to summarize and generate a data set for the various parameters and adjustment data during the handling process The category data and stability factor in the handling data are fed back to the small-sample learning model to update the feature description of the category, and at the same time, the enhanced state model M CS is updated to obtain the updated state model M opt , and finally, according to the updated state model M opt the object handling strategy is further updated.

[0061] The beneficial technical effects of the present invention are at least as follows:

[0062] (1) The present invention innovatively integrates multi-modal sensing data, including the inputs of cameras, lidar, and force sensors, and analyzes the changes of objects and the surrounding environment in real time through an adaptive algorithm. Through the scene perception module, the transfer vehicle can construct an accurate three-dimensional environment model and identify the spatial distribution and dynamic state of the objects. Even if new obstacles are added or the positions of the objects change in a complex environment, the system can quickly update the recognition results and handling strategies. This adaptive scene perception solves the defects of single sensors in traditional systems being limited and insensitive to dynamic changes, enabling the transfer vehicle to more flexibly respond to changes in the handling environment.

[0063] (2) The present invention introduces a multi-object state evaluation technology to comprehensively analyze the geometric shape, placement position, center of gravity, and stability of an object. For different types of objects (such as fragile, sliding, or irregular objects), the system automatically generates corresponding clamping forces and handling paths. For unstable objects, the system adopts protective clamping and avoids sharp turns and paths with excessive vibrations in path planning. Through dynamic strategy generation, the system can automatically adjust the clamping force and handling route according to real-time feedback during handling, thus preventing the object from falling or being damaged. This innovation effectively solves the defect that traditional transfer machines are difficult to stably handle irregular objects in a changing environment.

[0064] (3) The present invention innovatively introduces a small-sample learning module. Through a small number of training samples, the system can quickly learn and classify new objects and generate adaptive handling strategies, enabling efficient operation even when facing new object categories. Combining with fast re-identification technology, the system can re-identify and adapt to the state changes of objects in a dynamic environment. For example, if an object undergoes displacement or pose change during handling, the system can update the strategy in real time to ensure the stability and safety of handling. Description of the Drawings

[0065] The present invention will be further described with reference to the drawings. However, the embodiments in the drawings do not constitute any limitation to the present invention. For those of ordinary skill in the art, other drawings can be obtained according to the following drawings without creative efforts.

[0066] Figure 1 It is a flowchart of an intelligent automatic recognition method for a transfer machine according to an embodiment of the present invention.

[0067] Figure 2 It is a framework diagram of an intelligent automatic recognition system for a transfer machine according to an embodiment of the present invention. Detailed Embodiments

[0068] The embodiments of the present invention will be described in detail below. The examples of the embodiments are shown in the drawings, where the same or similar reference numerals represent the same or similar elements or elements with the same or similar functions throughout. The embodiments described below with reference to the drawings are exemplary and are only used to explain the present invention and should not be construed as a limitation to the present invention.

[0069] As Figure 1 shown, an intelligent automatic recognition method for a transfer machine provided by an embodiment of the present invention includes the following steps S1 - S6:

[0070] S1. Install a variety of sensors on the transfer machine to collect multi-modal data, and generate an environmental model M in the form of 3D voxels according to the multi-modal data, M = {v i}, v iRepresents a single voxel, optimizes and performs region segmentation on the environmental model M to obtain each sub-region after segmentation, and then extracts region features from each sub-region to construct a region model M j ; wherein, the multi-modal data includes RGB image data, point cloud data of a lidar, and temperature data of an infrared sensor, the single voxel includes an RGB color value, a point cloud depth value, and a temperature value, and the features include shape features, texture features, and temperature features.

[0071] Specifically, first, use a variety of sensors (such as an RGB camera, a lidar, and an infrared sensor) installed on a transfer machine to scan the current working area and collect multi-modal data. Let the data collected by the sensors be D, where D = {I RGB , P Lidar , T IR}, representing RGB image data, point cloud data of a lidar, and temperature data of an infrared sensor respectively.

[0072] For the RGB image data I RGB , use image enhancement and segmentation methods to remove background impurities and improve the clarity of the edges of the target object for subsequent accurate recognition. Sparsify the lidar point cloud data P Lidar , remove background noise points, and retain key points on the object surface to ensure data quality. For the infrared temperature data T IR , perform normalization processing based on a temperature threshold to unify the temperature range and reduce the data distribution difference brought by different sensors. Finally, fuse the pre-processed data into a multi-modal point set D f for the next step.

[0073] Furthermore, the generated multi-modal data D f is used to construct the environmental model M. Project all sensor data into a unified coordinate system and generate the environmental model M in the form of 3D voxels (Voxel), where M = {v i}, and v i represents a single voxel. Assign the following feature attributes to each voxel v i : RGB color value, point cloud depth value, and temperature value.

[0074] To improve the modeling efficiency and reduce redundant information, perform weighted calculation on each voxel by defining a feature function g(v i ):

[0075] g(v i ) = w RGB ·I RGB (v i ) + w Depth ·P Lidar (v i) + w Temp ·T IR (v i )

[0076] where w RGB , w Depth and w Temp are the weight coefficients of each modality, respectively, used to adjust the importance of different modalities. The output result of this function serves as the overall eigenvalue of this voxel in the environmental model M.

[0077] Furthermore, to improve the accuracy of subsequent object recognition, the feature data in the model M is optimized and regionally segmented. An adaptive voxel optimization algorithm is used to downsample the high-density regions, thereby reducing the computational amount. For each voxel v i , its local density ρ i is calculated, defined as the number of surrounding voxels. The voxel group with a density greater than the set threshold ρ threshold is downsampled to improve the computational efficiency.

[0078] According to the optimized environmental model, an adaptive segmentation method based on spatial connectivity and temperature distribution is used to divide the environmental model M into independent regions. In this way, high-temperature objects or regions with special shapes can be segmented into independent subsets M j , and each subset contains information about various target objects, providing accurate data for subsequent object recognition.

[0079] Furthermore, for each sub-region M j after segmentation, region feature extraction is performed. Three main types of features are extracted for each region, including shape features, texture features, and temperature features, to form a more refined regional model. The shape features can be calculated based on the point cloud contour and surface curvature, the texture features are based on color gradient and edge directionality analysis, and the temperature features are calculated based on the thermal distribution pattern of infrared data. Let the shape features, texture features, and temperature features be φ shape (M j ), φ texture (M j ), and φ temp No. (M j ), respectively.

[0080] Combining the above features, the updated model of each region is represented as M j = {φ shape (M j ), φ texture (M j ), φ temp (M j )}, which contains three types of feature data. This updated model provides accurate multi-modal region features for the next step of object recognition.

[0081] The output finally generated by the above steps is the multi-modal fusion environment model M, which will serve as the basic input for object recognition and status analysis in subsequent steps, providing accurate and efficient basic data for the overall process of this patent.

[0082] S2. Divide each voxel into multiple object regions to obtain voxel sets of different objects, denoted as where each set represents a target object, and then perform weighted feature calculation on each voxel v in the voxel set i to obtain the comprehensive feature vector x of the object. Then, perform few-shot learning on the comprehensive feature vector x j to construct a few-shot learning model, and use the few-shot learning model to calculate the central feature vector μ j of each category. When an unknown object c is detected, extract its feature vector and calculate the distance from it to the central μ of each category. Classify the unknown object c into the category c with the smallest distance from it. Finally, assign the category label c * of each recognized object to its feature vector x j , and update the environment model to generate an enhanced environment model M j containing the category label, that is, M C ={v C ,c i}, where c i is the category label of the object corresponding to the voxel. i

[0083] Specifically, the input is the multi-modal environment model M={v i} output by step one, where v i is a single voxel, including the geometric position, RGB color, point cloud depth, and temperature characteristics of the object. First, divide each voxel into multiple object regions to obtain voxel sets of different objects, denoted as where each set represents a target object.

[0084] To extract the comprehensive features of each object, perform weighted feature calculation on each voxel v in i . Define the regional feature extraction function as:

[0085]

[0086] where, xj Represents the comprehensive feature vector of an object, including the weighted mean of multi-modal features such as RGB color, depth, and temperature. The weighted mean of multi-modal features such as RGB color, depth, and temperature. Represents the number of voxels in the object region, used for mean calculation. w The number of voxels in the object region, used for mean calculation. w RGB w Depth and w Temp and w RGB (v i )、P Lidar (v i ) and T IR (v i ) represent the RGB color value, lidar depth value, and temperature value of the voxel v i The RGB color value, lidar depth value, and temperature value of the voxel v

[0087] Through the feature extraction function Obtain the comprehensive feature vector x of the object j , used for subsequent class recalculation and matching.

[0088] Furthermore, in this step, based on few-shot learning, the central feature of each class of objects is calculated. Through a small number of labeled samples {x i , y i} (where y i is the class label), calculate the central feature vector μ c of each class. The calculation formula for the class center vector is:

[0089]

[0090] where μ c represents the central feature vector of class c, used to represent the comprehensive feature of objects in this class. Represents the sample set of class c, containing all feature vectors belonging to class c. Represents the set The number of samples in the set, used for calculating the mean. Ω c Represents the regularization term, used to limit the offset of the class center and reduce overfitting. Ω c Can be set to the variance of the features of each class to smooth the class center and increase the generalization ability. λ represents the regularization coefficient, controlling the influence size of the regularization term Ω c . The calculated class center μ c is used to describe the average feature of each object class, facilitating similarity matching in subsequent new object recognition.

[0091] Furthermore, for the newly detected unknown object in the environmental model M Extract its feature vector and calculate the distance from the center μ of each category c . Define an improved similarity metric function d(x new , μ c ) for matching the category of the new object:

[0092]

[0093] where d(x new , μ c ) represents the matching distance between the new object x new and the central feature vector μ of category c c , used to determine the category of the new object. x new,j and μ c,j represent the components of the new object feature x new and the category center μ c on the j-th feature dimension respectively. σ c represents the discrimination parameter of category c, obtained by calculating the average distance between different category center vectors, to prevent category confusion and improve the discrimination between categories.

[0094] The system classifies the new object into the category c with the smallest distance, that is, satisfying c * = argmin * d(x c , μ new ) c of the category label.

[0095] Finally, assign the category label c j of each recognized object to its feature vector x j , and update the environmental model to generate an enhanced environmental model M[[ID=SS]] C , that is, M C = {v i , c i}, where c i is the category label of the object corresponding to the voxel. The environmental model M C will be used as the basic input for the next object state evaluation.

[0096] In this solution, by introducing a regularization term and a category discrimination parameter, the system achieves efficient and accurate object recognition under small sample conditions, enabling the updated environmental model M C to accurately represent different objects in the environment and provide reliable support for the state analysis and handling strategy in subsequent steps.

[0097] S3. According to the enhanced environmental model M CAnalyze the stability and state of an object, form an object state characteristic vector based on the results of the stability and state analysis, and then integrate the object state characteristic vector into the enhanced environment model M C to obtain the enhanced state model M CS ; among them, the results of the stability and state analysis include a stability factor S j and a dynamic inertia factor Γ j , and the enhanced state model M CS is expressed as M CS ={v i ,c i ,S i ,Γ i}.

[0098] Specifically, use the output enhanced environment model M C ={v i ,c i} as the input, where v i represents a single voxel, and c i is the class label of the object to which the voxel belongs. Divide the voxel into multiple object regions Each region represents object j. For each object extract geometric, class, and mass characteristics, and construct a preliminary characteristic vector z j ={x j ,m j ,c j [[ID=DB]] j}, where x j is the geometric center coordinate of the object , expressed as x j =[x j1 ,x j2 ,x j3 ; m j is the mass of the object, obtained by calculating the object volume V j and the bulk density ρ j , that is, m j =ρ j ·V j ; c j is the class label of the object, and the subsequent calculation parameters are adjusted according to the object class.

[0099] Furthermore, the stability factor S j is used to quantify the balance of the object during handling. Introduce the center of gravity offset Δ COM,j and the contact angle θ j as stability influence factors, and adjust their contributions according to the class weight. The calculation of the stability factor S j is as follows:

[0100]

[0101] Among them, S j represents the stability factor of the object , with a range of [0, 1], and the larger the value, the more stable; represents the adjustment parameter of category c j , which controls the weights of Δ COM,j and θ j ; Δ COM,j represents the offset of the center of gravity of the object, defined as the distance between the geometric center x j and the mass center x COM,j , that is, Δ COM,j = ∥x COM,j - x j ∥; θ j represents the tilt angle of the object, the angle with the ground contact surface; ∈ represents a small positive value to prevent the denominator from being zero.

[0102] Furthermore, during the handling process, the inertial response of the object will significantly affect its dynamic stability. Define the dynamic inertia factor Γ j to quantify this characteristic and calculate it in combination with the object mass and the category inertia coefficient. The formula is as follows:

[0103]

[0104] Among them, Γ j represents the dynamic inertia factor of the object , describing its inertia magnitude; represents the inertia coefficient of category c j , which adjusts the inertia sensitivity according to the object category; m j represents the object mass, which is calculated from the volume V j and the bulk density ρ j ; ρ j represents the object density coefficient, which is calculated based on voxel data and reflects the overall density of the object.

[0105] Furthermore, according to the calculated S j and Γ j , combine them into the object state characteristic vector s j = {S j , Γ j , m j , c j}. Integrate the state characteristics into the voxel set of each object to obtain the updated environment model M CS = {v i , c i , S i , Γ i}, where S i and Γ iThe state characteristics of the object corresponding to each voxel. The model M CS Provides input for the generation of the next handling strategy.

[0106] Furthermore, the output environmental model M CS Contains the category, stability, and dynamic inertia information of each object for use in generating the next handling path and strategy. Through S j and Γ j Joint evaluation, the system can generate personalized handling plans according to the object category and dynamic characteristics to ensure stability and safety during handling.

[0107] This step passes through the stability factor S j and the dynamic inertia factor Γ j Comprehensive analysis to ensure that the transfer machine can respond to different characteristics of the object, and the generated model M CS Provides reliable data support for the generation of handling strategies.

[0108] S4. According to the enhanced state model M CS Generate a preliminary handling strategy suitable for each object, where the preliminary handling strategy includes the grasping force F j , handling speed v j , maximum acceleration a j and the final path

[0109] Specifically, the input of this step is the enhanced state model M CS ={v i ,c i ,S i ,Γ i}, where v i is the voxel, c i is the category label, S i is the stability factor of the voxel, and Γ i is the dynamic inertia factor of the voxel. The model is decomposed into multiple object regions Each object region Represents object j. The system extracts the overall characteristics of object j, including stability S j , inertia factor Γ j , mass m j , category c j , for subsequent strategy generation.

[0110] Furthermore, the grasping force F j Needs to be accurately calculated according to the category, stability factor, and mass of the object to ensure that the object will not slip or be damaged due to improper grasping force during handling. Define the calculation formula of the grasping force F j as follows:

[0111]

[0112] Among them, F j represents the grasping force of the object . represents the grasping coefficient of the object category c j , which controls the grasping force of different categories of objects. For example, the grasping coefficient of fragile objects is small, while that of heavy objects is large. m j represents the mass of the object. S j represents the stability factor of the object. The smaller the value, the less stable the object is. Therefore, the grasping force F j needs to be appropriately increased. ∈ represents a tiny positive value to prevent the denominator from being zero. Ω j represents the enhanced regularization term, which reflects the morphological complexity of the object contact surface. If the object surface is smooth and the grasping points are limited, Ω j will have a larger value to increase the accuracy of the grasping force. η represents the regularization coefficient, which adjusts the influence of the enhanced regularization term Ω j .

[0113] The enhanced regularization term Ω in the grasping force calculation j is adjusted based on the smoothness and morphological complexity of the object surface to ensure that even objects with irregular surfaces or insufficient contact can be firmly grasped.

[0114] Furthermore, according to the dynamic inertia factor Γ of the object j to determine the handling speed and acceleration. Objects with stronger inertia require lower acceleration during handling to avoid unnecessary shaking. Define the handling speed v of the object j and the maximum acceleration a j , where:

[0115]

[0116] Among them, v j represents the handling speed of the object , which needs to be kept at a lower value when the inertia is large. a j represents the acceleration of the object, which determines the smoothness of the handling start and stop. and are the adjustment coefficients of the speed and acceleration respectively, which are determined by the object category. The coefficients of fragile objects are small to ensure smooth handling. Γ j represents the dynamic inertia factor, which represents the inertial characteristics of the object. When the inertia factor is larger, the speed v j and the acceleration a j need to be appropriately reduced to prevent the object from being unstable during acceleration and turning.

[0117] Among them, the generated speed v j and the acceleration aj Will be used as the basic parameters for path planning to ensure a smooth handling process. Especially when handling fragile or high-inertia objects, it helps to avoid unnecessary shaking or damage.

[0118] Furthermore, path planning is to reduce the shaking of objects caused by turning and obstacles during the handling process. Especially for objects with high inertia or low stability, it is more necessary to optimize the path to maintain stability. Let the preliminary path be Define the final path of the object As follows:

[0119]

[0120] Where, Represents the final handling path of the object, adapting to dynamic characteristics. Represents the preliminary path planning (such as a straight line or the shortest path). Represents the path adjustment coefficient of category c Represents category c j Of the path adjustment coefficient, reflecting the path adjustment requirements of different objects. The path adjustment coefficient of fragile or heavy object categories is larger. Δθ represents the change in the tilt angle of the object, and is an inclination protection term generated based on the object stability S j And the current path turning angle. When the inclination is large, the path will avoid sharp turns and vibrations.

[0121] The system dynamically calculates the path during the handling process, automatically adapting to the turning angle and obstacle distribution to ensure that the object will not tilt or shake due to unstable path during the handling process.

[0122] Furthermore, the force F j , handling speed v j , maximum acceleration a j And the final path These parameters are adaptively adjusted according to the category and state of the object to ensure stability and safety during the handling process. The output handling strategy serves as the input basis for the next real-time feedback adjustment, providing a foundation for further adjustment according to environmental changes during the handling process.

[0123] In this step, by introducing the gripping force, speed, and path planning strategies based on the state model, the system can generate optimal handling parameters according to the dynamic characteristics of different objects, ensuring the handling efficiency while greatly improving the stability and safety of the object. The generated model M CS And the handling strategy are used as the input for the next real-time feedback adjustment.

[0124] S5. Use the status data of the object collected in real time by the sensors on the transfer machine to generate a feedback vector. At the same time, during the handling process, monitor the status changes of the object and environmental factors in real time through the sensors, adjust and fine-tune the handling parameters according to the feedback information, and update them in real time during the next handling process of the transfer machine. When the environment or the object status changes further, the transfer machine will repeat the above real-time feedback adjustment steps to form a closed-loop control mechanism.

[0125] Specifically, use the sensors installed on the transfer machine (such as accelerometers, gyroscopes, and force sensors) to collect the status data of the object in real time and form a feedback vector f j ={d COM,j , θ j , v obs,j}, where d COM,j represents the real-time centroid offset of the object, indicating the real-time offset degree between the mass center and the geometric center of the object. θ represents the real-time tilt angle of the object, indicating the tilt situation of the object during the handling process. v j represents the speed of the obstacle detected in the environment, which is used to adjust the handling speed to avoid collision with moving obstacles. obs,j The feedback vector f

[0126] will be used as the input of the fine-tuning algorithm and combined with the initial handling strategy parameters j to generate the parameters adjusted in real time.

[0127] Furthermore, adjust the grasping force F COM,j according to the centroid offset d j and the real-time tilt angle θ j to improve the stability of the object during the handling process. Define the dynamic adjustment formula of the grasping force as follows:

[0128] F' j = F j + λ d · d COM,j + λ θ · sin(θ j )

[0129] where F' j represents the adjusted grasping force, which is used to correct the force of the gripper in real time. F j represents the initial grasping force. λ d and λ θ are the adjustment coefficients of the centroid offset and the tilt angle respectively, which are used to control the influence of the centroid offset and the tilt on the grasping force. sin(θ j ) represents the sine term of the tilt angle. When the object is tilted, sin(θ j) will increase, thus increasing the grasping force to prevent slipping.

[0130] By adjusting the grasping force F′ in real time j , the system can better handle the instability caused by the center-of-gravity shift or tilt during the handling process, and avoid the object from slipping out of the gripper or accidentally tipping over.

[0131] Furthermore, if a dynamic obstacle (such as a forklift or other mobile device) is detected in the environment during the handling process, the system will adjust the handling speed v obs,j according to the speed v of the obstacle j . At the same time, the handling path is finely adjusted through the real-time feedback of the tilt angle and path curvature to ensure the smooth handling of the object. The adjustment formula for the handling speed is defined as follows:

[0132] v′ j = v j · exp(-γ · v obs,j )

[0133] where v′ j represents the adjusted handling speed to ensure that the object decelerates when approaching the obstacle. v j represents the initial handling speed. γ represents the speed adjustment coefficient, which is used to adjust the influence degree of the obstacle speed on the handling speed. v obs,j represents the speed of the obstacle. When the obstacle approaches, the handling speed will gradually decrease.

[0134] In addition, according to the tilt angle θ of the object j the handling path is finely adjusted in real time to avoid sudden sharp turns and unnecessary direction changes in the path. The final path fine-tuning formula is defined as:

[0135]

[0136] where, represents the adjusted handling path. represents the initial path. κ represents the path adjustment coefficient, which is used to control the influence of the tilt angle on the path smoothness.

[0137] By adjusting the handling speed and path, the system can effectively respond to environmental changes and reduce the vibration and instability of the object during the handling process.

[0138] Furthermore, the handling parameters after dynamic fine-tuning It will be fed back to the handling control module and updated in real time during the subsequent handling process. When further changes occur in the environment or the object state, the system will repeat the above real-time feedback adjustment steps to form a closed-loop control mechanism, ensuring that the entire handling process remains in a stable state at all times. Finally, a fine-tuned handling strategy will be output to ensure the efficient completion of the handling task.

[0139] In this step, through the real-time feedback mechanism, the system can dynamically fine-tune the grasping force, handling speed, and path to cope with changes in the object state and the environment, ensuring the continuous smoothness and efficiency of the handling process. The output fine-tuning strategy provides effective support for the closed-loop control of the entire handling process.

[0140] S6. Aggregate the parameters and adjustment data during the handling process to generate a data set Feed the handling data The category data and stability factor in it back to the few-shot learning model to update the feature description of the category and simultaneously update the enhanced state model M CS Obtain the updated state model M opt , and finally, based on the updated state model M opt Further update the object handling strategy.

[0141] Specifically, collect the parameters and adjustment data during the handling process and aggregate them to generate a data set where d k is the feedback data vector for the k-th handling, expressed as Each d k contains the following: F′ j represents the grasping force after real-time adjustment, indicating the final actual force of the gripper. v′ j represents the adjusted handling speed, reflecting the speed adjustment in the actual environment. a′ j represents the actual acceleration, indicating the actual feedback during acceleration and deceleration. represents the actual handling path, combined with path fine-tuning to avoid obstacles. S j represents the stability factor of the object, used to reflect the object state. Γ j represents the dynamic inertia factor of the object, indicating the sensitivity of the object to dynamic response.

[0142] These feedback data will provide a statistical basis for future tasks and offer data support for optimizing model parameters and generating more accurate strategies.

[0143] Furthermore, for the classification and recognition module of the object category, feed the category data and stability factor in the handling data back to the few-shot learning model. By calculating the new category center vector μ′c , the model can gradually update the feature descriptions of various categories. The formula for updating the category center vector is defined as follows:

[0144]

[0145] where, μ′ c represents the updated category center vector, which is used for category determination in subsequent tasks. α represents the update coefficient, which controls the influence of new data on the category center. Usually, a small value is taken (for example, α = 0.1) so that the model can gradually adapt to new samples. represents the set of feedback samples for category c, which is composed of the object features collected in the handling task. This update formula enables the category center to gradually adapt to new data. Especially when the environment or object characteristics change, it can automatically adjust the generalization ability of the category classification model.

[0146] Furthermore, by analyzing the relationship between the stability factor S j and the dynamic inertia factor Γ j , historical data is used to update the weight parameters and (parameters related to stability and inertia respectively). The parameter update formula is defined as:

[0147]

[0148] where, and represent the updated parameters related to stability and inertia. η represents the learning rate, which controls the speed of parameter update. Usually, a small value is taken (for example, η = 0.01). S k and Γ k represent the stability and inertia feedback values in the k-th task. n represents the cumulative number of tasks.

[0149] Through these updates, the system can automatically adjust the parameters of the state model, making it more accurate to evaluate the object state in future handling tasks, thereby improving the stability and reliability of the entire system.

[0150] Furthermore, after each handling task is completed, the feedback path adjustment parameter Δθ k and data such as speed and acceleration are accumulated as experience and applied to the optimization of the handling strategy. The optimized fine-tuning strategy for the handling path is defined as follows:

[0151]

[0152] where, represents the optimized handling path, which automatically tends to a smooth path in future tasks. Δθk Represents the path tilt adjustment amount in the historical task, and obtains the adaptive changes in the handling path through accumulation.

[0153] By superimposing the mean value of all path adjustment data in the historical data on the initial path The system can gradually optimize the adaptability of the path and reduce the need for additional path adjustments during handling.

[0154] Furthermore, the updated model M opt Will replace the original state model M CS , including optimized class centers, parameter updates, and policy path optimizations. This model is used as input for object recognition, state evaluation, and handling strategy generation in the next task, achieving the ability of dynamic learning and self-optimization.

[0155] By adjusting various parameters and models of the system after the task is completed, this step provides a more reliable and intelligent decision-making basis for future handling tasks. The self-optimization mechanism of the model can improve the adaptability of the transfer machine in a dynamic environment and gradually achieve a double improvement in handling efficiency and stability.

[0156] Such as Figure 2 As shown, in another embodiment of the present invention, an intelligent automatic recognition system for a transfer machine is provided, and the system includes:

[0157] A multi-modal data acquisition unit 1011, which is used to install various sensors on the transfer machine to collect multi-modal data, generate an environmental model M in the form of 3D voxels according to the multi-modal data, M = {v i}, v i Represents a single voxel, and optimizes and regionally segments the environmental model M to obtain each sub-region after segmentation, and then extracts regional features from each sub-region to construct a regional model M j ; wherein, the multi-modal data includes RGB image data, point cloud data of a lidar, and temperature data of an infrared sensor, the single voxel includes RGB color values, point cloud depth values, and temperature values, and the features include shape features, texture features, and temperature features;

[0158] A single voxel segmentation unit 1012, which is used to segment a single voxel into multiple object regions to obtain a voxel set of different objects, denoted as Where each set Represents a target object, and then calculates the weighted features of each voxel v In to obtain the comprehensive feature vector x i Of the object , and then for the comprehensive feature vector x j , and then for the comprehensive feature vector x jPerform few-shot learning to construct a few-shot learning model, and use the few-shot learning model to calculate the central feature vector μ of each category c , when an unknown object is detected , extract its feature vector and calculate the distance from the central μ of each category c , classify the unknown object into the category c with the smallest distance * , and finally assign the category label c of each recognized object j to its feature vector x j , and update the environment model to generate an enhanced environment model M containing category labels C , that is, M C ={v i ,c i}, where c i is the category label of the object corresponding to the voxel;

[0159] The environment enhancement unit 1013 is used to analyze the stability and state of the object according to the enhanced environment model M C , form an object state characteristic vector according to the results of the stability and state analysis, and then integrate the object state characteristic vector into the enhanced environment model M C to obtain an enhanced state model M CS ; where, the results of the stability and state analysis include a stability factor S j , a dynamic inertia factor Γ j , and the enhanced state model M CS is expressed as M CS ={v i ,c i ,S i ,Γ i};

[0160] The preliminary handling strategy generation unit 1014 is used to generate a preliminary handling strategy suitable for each object according to the enhanced state model M CS , where the preliminary handling strategy includes a grasping force F j , a handling speed v j , a maximum acceleration a j and a final path

[0161] The real-time data acquisition unit 1015 is used to generate a feedback vector using the status data of the object collected in real time by the sensors on the transfer machine. At the same time, during the handling process, it monitors the status changes of the object and environmental factors in real time through the sensors, adjusts and fine-tunes the handling parameters according to the feedback information, and updates them in real time during the next handling process of the transfer machine. When the environment or the object status changes further, the transfer machine will repeat the above real-time feedback adjustment steps to form a closed-loop control mechanism;

[0162] The handling strategy optimization unit 1016 is used to summarize and generate a data set from the various parameters and adjustment data during the handling process Feed the category data and stability factor in the handling data back to the small sample learning model to update the feature description of the category, and at the same time update the enhanced state model M CS Obtain the updated state model M opt Finally, according to the updated state model M opt further update the object handling strategy.

[0163] It should be noted that the above-described work process is only illustrative and does not limit the protection scope of the present invention. In actual applications, those skilled in the art can select some or all of them according to actual needs to achieve the purpose of the solution of this embodiment, and no limitation is made here.

[0164] In addition, for the technical details not described in detail in this embodiment, reference can be made to the parameter operation method provided in any embodiment of the present invention, which will not be elaborated here.

[0165] It should be noted that in this article, the terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements not only includes those elements, but also includes other elements not expressly listed, or also includes elements inherent to such process, method, article or system. Without further limitation, an element defined by the statement "including one..." does not exclude the existence of another identical element in the process, method, article or system including that element.

[0166] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0167] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-described example methods can be implemented by means of software plus a necessary general hardware platform. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as a read-only memory / random access memory, magnetic disk, optical disk), and includes several instructions for causing a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0168] The above are only the preferred embodiments of the present invention, and do not limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made by using the content of the specification and drawings of the present invention, or directly or indirectly applied in other related technical fields, shall be equally included in the patent protection scope of the present invention.

Claims

1. An intelligent automatic recognition method for a transfer machine, characterized in that, The method includes: S1. Install multiple sensors on the transfer machine to collect multimodal data, generate an environmental model M in the form of 3D voxels according to the multimodal data, M = {v i}, where v i represents a single voxel, optimize and regionally segment the environmental model M, obtain each sub-region after segmentation, and then extract regional features from each sub-region to construct a regional model M j ; among them, the multimodal data includes RGB image data, point cloud data of lidar, and temperature data of infrared sensors, the single voxel contains RGB color values, point cloud depth values, and temperature values, and the features include shape features, texture features, and temperature features; S2. Divide a single voxel into multiple object regions to obtain voxel sets of different objects, denoted as where each set represents a target object. Then, for each voxel v in the voxel set i , calculate the weighted feature to obtain the comprehensive feature vector x of the object. Then, perform few-shot learning on the comprehensive feature vector x j to construct a few-shot learning model, and use the few-shot learning model to calculate the central feature vector μ j of each category. When an unknown object c is detected, extract its feature vector and calculate the distance from it to the central μ of each category. Classify the unknown object c into the category c with the smallest distance to it. Finally, assign the category label c * of each recognized object to its feature vector x j , and update the environment model to generate an enhanced environment model M j containing the category label, that is, M C = {v C , c i , c i}, where c i is the category label of the object corresponding to the voxel; S3. According to the enhanced environment model M C perform stability and state analysis on the object, form an object state characteristic vector based on the results of the stability and state analysis, and then integrate the object state characteristic vector into the enhanced environment model M C to obtain an enhanced state model M CS ; wherein, the results of the stability and state analysis include a stability factor S j , a dynamic inertia factor Γ j , and the enhanced state model M CS is expressed as M CS ={v i ,c i ,S i ,Γ i}; S4. Generate a preliminary handling strategy suitable for each object according to the enhanced state model M CS wherein the preliminary handling strategy includes a grasping force F j , a handling speed v j , a maximum acceleration a j and a final path S5. Using the state data of the object collected in real time by the sensors on the transfer machine to generate a feedback vector. At the same time, during the handling process, the state changes of the object and environmental factors are monitored in real time through the sensors. The handling parameters are adjusted and fine-tuned according to the feedback information and are updated in real time during the subsequent handling process of the transfer machine. When the environment or the state of the object changes further, the transfer machine repeats the above real-time feedback adjustment steps to form a closed-loop control mechanism; S6. Aggregate various parameters and adjustment data during the handling process to generate a data set The handling data The category data and stability factors in are fed back into the small-sample learning model to update the feature description of the category and simultaneously update the enhanced state model M CS Obtain the updated state model M opt , and finally, based on the updated state model M opt Further update the object handling strategy.

2. The intelligent automatic recognition method for a transfer machine according to claim 1, characterized in that, In the step S1, for the RGB image data I RGB , an image enhancement and segmentation method is used to remove background impurities and improve the clarity of the edges of the target object; for the lidar point cloud data P Lidar , a sparsification process is performed to remove background noise points and retain the key points on the object surface; for the infrared temperature data T IR , a normalization process based on a temperature threshold is carried out to unify the temperature range and reduce the data distribution differences brought by different sensors; finally, the preprocessed data is fused into a multi-modal point set D f .

3. The intelligent automatic recognition method for a transfer machine according to claim 1, characterized in that, Optimize and perform region segmentation on the environmental model M to obtain each sub-region after segmentation, and then extract region features from each sub-region to construct a region model M j , which specifically includes: Adopting an adaptive voxel optimization algorithm to downsample the high-density area: At each voxel v i , calculate its local density ρ i , defined as the number of surrounding voxels, and downsample the voxel group with a density greater than the set threshold ρ threshold to improve the calculation efficiency; Using an adaptive segmentation method based on spatial connectivity and temperature distribution, the environmental model M is divided into independent regions to obtain each sub-region M after segmentation j ; For each of the segmented sub-regions M j , region feature extraction is performed; among them, the shape feature is calculated based on the point cloud contour and surface curvature, the texture feature is analyzed based on color gradient and edge directionality, and the temperature feature is calculated based on the thermal distribution pattern of infrared data. Combining the above features, the updated model of each region is represented as M j = {φ shape (M j ), φ texture (M j ), φ temp (M j )}, where the shape feature, texture feature, and temperature feature are φ shape (M j ), φ texture (M j ), and φ temp (M j ), respectively.

4. The intelligent automatic recognition method for a transfer machine according to claim 1, characterized in that For each voxel v in the voxel set i weighted feature calculation is performed to obtain the comprehensive feature vector x of the object j , and the calculation is as follows: Among them, x j represents the comprehensive feature vector of the object , which is the weighted mean of multi-modal features including RGB color, depth, temperature, etc.; represents the number of voxels in the object region , which is used for mean calculation; w RGB , w Depth and w Temp represent the weight parameters of RGB, depth, and temperature modal features respectively, which are used to adjust the influence of different modal features in the fused features; I RGB (v i ), P Lidar (v i ) and T IR (v i ) represent the RGB color value, lidar depth value, and temperature of the voxel v i respectively; The comprehensive feature vector x j is used for few-shot learning to construct a few-shot learning model, and the central feature vector μ c of each category is calculated using the few-shot learning model. The calculation is as follows: Among them, μ c represents the central feature vector of class c, which is used to represent the comprehensive features of the objects in this class; represents the sample set of class c, which contains all the feature vectors belonging to class c; represents the set the number of samples in, which is used to calculate the mean; Ω c represents the regularization term, which is used to limit the deviation of the class center and reduce overfitting; λ represents the regularization coefficient, which controls the influence size of the regularization term Ω c of.

5. The intelligent automatic recognition method for a transfer machine according to claim 4, wherein, When an unknown object is detected match the category of the new object, and the calculation is as follows: Among them, d(x new , μ c ) represents the matching distance between the new object x new and the central feature vector μ of category c c , which is used to judge the category of the new object; x new,j and μ c,j respectively represent the components of the new object feature x new and the category center μ c on the j-th feature dimension; σ c represents the discrimination parameter of category c, which is obtained by calculating the average distance between different category center vectors, and is used to prevent category confusion and improve the discrimination between categories; Classify the unknown object into the category c with the smallest distance from it * , that is, satisfying c * = argmin c d(x new , μ c ), where μ is the category label 6. The intelligent automatic recognition method for a transfer machine according to claim 1, characterized in that, The stability and state of the object are analyzed according to the enhanced environment model M C Specifically, the following steps are included: Segment the voxel v i into multiple object regions For each region represent object j. For each object Extract geometric, category, and mass characteristics to construct a preliminary characteristic vector z j ={x j , m j , c j}, where x j is the geometric center coordinates of the object denoted as x j =[x j1 , x j2 , x j3 , m j is the mass of the object, obtained by calculating the object volume V j and the bulk density ρ j , i.e., m j =ρ j ·V j , c j is the category label of the object; Introduce the center-of-gravity offset Δ of the object COM,j and the contact angle θ j Construct the stability factor S as a stability influence factor j , and adjust its contribution according to the category weight. At the same time, define the dynamic inertia factor Γ according to the inertia of the object during handling j to quantify the characteristics of the inertia factor Γ j and calculate it in combination with the object mass and the category inertia coefficient, as shown below: Γ j = γ cj · m j · ρ j Among them, S j represents the stability factor of the object , ranging from [0, 1], and the largest value indicates the most stable state; represents the adjustment parameter for category c j , which controls the weights of Δ COM,j and θ j ; Δ COM,j represents the offset of the object's center of gravity, defined as the distance between the geometric center x j and the mass center x COM,j , that is, Δ COM,j =∥x COM,j -x j ∥; θ j represents the tilt angle of the object, which is the angle with the ground contact surface; ∈ represents a tiny positive value to prevent the denominator from being zero; Γ j represents the dynamic inertia factor of the object , which describes its inertia magnitude; represents the inertia coefficient for category c j , which adjusts the inertia sensitivity according to the object category; m j represents the mass of the object, which is calculated from the volume V j and the bulk density ρ j ; ρ j represents the object density coefficient, which is calculated based on voxel data and reflects the overall density of the object.

7. An intelligent automatic recognition method for a transfer machine according to claim 1, characterized in that, The grasping force F j has the following calculation formula: Among them, F j represents the grasping force of the object , represents the grasping coefficient of the object category c j , controlling the grasping force of different categories of objects; m j represents the mass of the object; S j represents the stability factor of the object; ∈ represents a tiny positive value to prevent the denominator from being zero; Ω j represents the enhanced regularization term, reflecting the morphological complexity of the object contact surface; η represents the regularization coefficient, adjusting the influence of the enhanced regularization term Ω j . The handling speed v j and the maximum acceleration a j are calculated as follows: Among them, v j represents the handling speed of the object ; a j represents the acceleration of the object, which determines the smoothness of the handling start and braking; and are the adjustment coefficients of the speed and acceleration respectively, which are determined by the object category; The final path is calculated as follows: Among them, P j represents the final handling path of the object O j to adapt to dynamic characteristics; P j 0 represents the preliminary path planning; represents the path adjustment coefficient of the category c j to reflect the path adjustment requirements of different objects; Δθ represents the change in the tilt angle of the object, which is the tilt protection term generated based on the object stability S j and the current path turning angle.

8. An intelligent automatic recognition method for a transfer machine according to claim 7, characterized in that, The state data of the object collected in real time by the sensors on the transfer machine, including the object 's real-time center of gravity offset d COM,j , the real-time tilt angle θ of the object j and the speed v of the obstacle detected in the environment obs,j , then the feedback vector is f j ={d COM,j ,θ j ,v obs,j}; According to the center-of-gravity offset d COM,j and the real-time tilt angle θ j adjust the grasping force F j , which is expressed as follows: F′ j = F j + λ d ·d COM,j + λ θ ·sin(θ j ) Among them, F′ j represents the adjusted grasping force, which is used to correct the force of the gripper in real time; λ d and λ θ are the adjustment coefficients of the center-of-gravity offset and the tilt angle respectively, which are used to control the influence of the center-of-gravity offset and the tilt on the grasping force; sin(θ j ) represents the sine term of the tilt angle. When the object is tilted, sin(θ j ) will increase, thereby increasing the grasping force to prevent slipping; Adjust the handling speed v according to the obstacle speed v obs,j to adjust the handling speed v j , and at the same time, fine-tune the handling path based on the tilt angle and path curvature obtained through real-time feedback, as shown below: v′ j = v j · exp(-γ· v obs,j ) Among them, v′ j represents the adjusted handling speed to ensure that the object decelerates when approaching an obstacle; γ represents the speed adjustment coefficient, which is used to adjust the influence degree of the obstacle speed on the handling speed; According to the inclination angle θ of the object j Fine-tune the handling path P in real time j , so that the path avoids sudden sharp turns and unnecessary direction changes, and the calculation is as follows: P′ j = P j + κ·θ j Among them, P′ j represents the adjusted handling path; κ represents the path adjustment coefficient, which is used to control the influence of the inclination angle on the path smoothness.

9. The intelligent automatic recognition method for a transfer machine according to claim 1, characterized in that, The said data set is wherein is the feedback data vector for the th handling, expressed as wherein each includes the grasping force after real-time adjustment the adjusted handling speed v′ j , the actual acceleration a′ j , the actual handling path the stability factor S of the object j and the dynamic inertia factor Γ of the object j .

10. An intelligent automatic identification system for a transfer machine, characterized in that, The system includes: Multi-modal data acquisition unit, which is used to install multiple sensors on the transfer machine to collect multi-modal data, generate an environmental model M in the form of 3D voxels according to the multi-modal data, M = {v i}, where v i represents a single voxel, optimize and regionally segment the environmental model M, obtain each segmented sub-region, and then extract region features from each sub-region to construct a region model M j ; among them, the multi-modal data includes RGB image data, point cloud data of lidar, and temperature data of infrared sensors, the single voxel contains RGB color values, point cloud depth values, and temperature values, and the features include shape features, texture features, and temperature features; A single voxel segmentation unit is used to divide a single voxel into multiple object regions, obtaining voxel sets of different objects, denoted as where each set represents a target object. Then, for each voxel v in the voxel set i weighted feature calculation is performed to obtain the comprehensive feature vector x of the object j . Then, few-shot learning is carried out on the comprehensive feature vector x j to construct a few-shot learning model, and the central feature vector μ c of each category is calculated using the few-shot learning model. When an unknown object is detected, its feature vector is extracted and the distance is calculated with the central μ c of each category. The unknown object is classified into the category c * with the smallest distance to it. Finally, the category label c j of each recognized object is assigned to its feature vector x j , and the environmental model is updated to generate an enhanced environmental model M C containing the category label, that is, M C = {v i , c i}, where c i is the category label of the object corresponding to the voxel; An environment enhancement unit for performing stability and state analysis on an object according to an enhanced environment model M C Performing stability and state analysis on the object, forming an object state characteristic vector according to the results of the stability and state analysis, and then integrating the object state characteristic vector into the enhanced environment model M C to obtain an enhanced state model M CS ; wherein, the results of the stability and state analysis include a stability factor S j , a dynamic inertia factor Γ j , and the enhanced state model M CS is expressed as M CS = {v i , c i , S i , Γ i}; A preliminary handling strategy generation unit, which is used to generate a preliminary handling strategy suitable for each object according to the enhanced state model M CS The preliminary handling strategy includes a grasping force F j , a handling speed v j , a maximum acceleration a j and a final path A real-time data acquisition unit, which is used to use the state data of the object collected in real time by the sensors on the transfer machine to generate a feedback vector. At the same time, during the handling process, the state changes of the object and environmental factors are monitored in real time through the sensors. The handling parameters are adjusted and fine-tuned according to the feedback information and are updated in real time during the subsequent handling process of the transfer machine. When the environment or the state of the object changes further, the transfer machine repeats the above real-time feedback adjustment steps to form a closed-loop control mechanism; The handling strategy optimization unit is used to summarize various parameters and adjustment data during the handling process to generate a data set Feed the category data and stability factor in the handling data back to the small-sample learning model to update the feature description of the category and simultaneously update the enhanced state model M CS Obtain the updated state model M opt Finally, based on the updated state model M opt further update the object handling strategy.

Images