Intelligent self-adaptive multi-layer flexible material wobble plate system

Abstract

The invention discloses an intelligent self-adaptive multi-layer flexible material wobble plate system. The intelligent self-adaptive multi-layer flexible material wobble plate system comprises a plurality of carrier lifting assemblies, a plurality of tray conveying assemblies, a plurality of push plate mechanism assemblies and a plurality of machine head assemblies. The carrier lifting assembly is used for bearing a basket assembly; the tray conveying assemblies are arranged on the first sides of the corresponding basket assemblies; the push plate mechanism assemblies are arranged on the second sides of the corresponding basket assemblies and used for pushing the target trays in the basket assemblies to the corresponding tray conveying assemblies. The tray conveying assembly is configured to adjust the width of a track in the tray conveying assembly according to the size of a target tray so as to adapt to the target tray; the machine head assemblies are used for adsorbing materials from the corresponding flexible vibration discs and placing the materials in the target trays. And after the materials are placed in the target tray, a push plate unit in the tray conveying assembly pushes the target tray into the basket assembly. In this way, the tray placing function requirements of trays of various sizes are met.

Description

Technical Field

[0001] The present application relates to the technical field of material tray placement systems, and in particular to an intelligent adaptive multi-layer flexible material tray placement system. Background Art

[0002] The existing material tray placement system automatically places the trays through equipment, and the trays are taken away manually after the baskets are full. The disadvantage is that the material tray placement system has a single function and cannot meet the requirements of tray placement for various sizes. Summary of the invention

[0003] The intelligent adaptive multi-layer flexible material display system provided in this application can meet the display function requirements of pallets of various sizes.

[0004] In a first aspect, the present application provides an intelligent adaptive multi-layer flexible material tray system, which includes: a plurality of carrier lifting assemblies, each carrier lifting assembly is used to carry a basket assembly, wherein the basket assembly is used to accommodate a pallet; a plurality of pallet conveying assemblies, each pallet conveying assembly is arranged on the first side of its corresponding basket assembly; a plurality of push plate mechanism assemblies, each push plate mechanism assembly is arranged on the second side of its corresponding basket assembly, and is used to push the target pallet in the basket assembly to the corresponding pallet conveying assembly; wherein the pallet conveying assembly is used to convey the target pallet to the material placement position, and the pallet conveying assembly is configured to adjust the track width in the pallet conveying assembly according to the size of the target pallet to adapt to the target pallet; a plurality of head assemblies, each head assembly is used to absorb materials from the corresponding flexible vibration plate, and move to the material placement position to place the materials in the target pallet; wherein, after the materials are placed in the target pallet, the push plate unit in the pallet conveying assembly pushes the target pallet into the basket assembly.

[0005] Among them, the intelligent adaptive multi-layer flexible material palletizing system also includes: a first multispectral image acquisition component, used to collect multispectral images of the target pallet; wherein the multispectral image includes visible light images, infrared images and ultraviolet images; a processing component, coupled to the first multispectral image acquisition component, used to fuse the multispectral images, and identify the target pallet from the fused multispectral images, and control the pallet conveying component to adjust the track width in the pallet conveying component according to the identification result.

[0006] Among them, the processing component is also used to extract image features of visible light images, infrared images and ultraviolet images respectively, and fuse the image features to obtain multi-spectral fusion features, and use a low-rank metric model based on orthogonal learning to reduce the dimension and denoise the multi-spectral fusion features to obtain a low-rank feature representation, and identify the target pallet based on the low-rank feature representation.

[0007] The processing component is also used to compensate for the system error corresponding to the first multispectral image acquisition component according to a preset period.

[0008] Among them, the processing component is also used to perform multi-axis coordinated control of the motors in the carrier lifting component, the pallet conveying component, the push plate mechanism component and the head component.

[0009] Among them, the material strength of the carrier lifting component is greater than a threshold, and the material weight is less than a threshold, wherein the carrier lifting component is used to dynamically adjust the operating speed according to the weight of the basket component; and each carrier lifting component is configured with an independent control unit.

[0010] Among them, each head component is provided with a flexible material suction nozzle, which has a vacuum cavity structure, and each vacuum cavity is configured to adjust the vacuum degree according to the shape and size of the material, and each head component is used to collect a multispectral image corresponding to the flexible vibration disk; wherein the multispectral image includes a visible light image, an infrared image and an ultraviolet image; the multispectral image is fused, and the material posture, surface texture, material, predicted contact position, adsorption force, and temperature in the flexible vibration disk are identified from the fused multispectral image, and the material is adsorbed from the flexible vibration disk according to the material posture, surface texture, material, predicted contact position, adsorption force, and temperature.

[0011] Among them, multiple carrier lifting components, multiple pallet conveying components, multiple push plate mechanism components, use value-based reinforcement learning and strategy-based reinforcement learning to optimize the moving path, and multiple head components use spatiotemporal graph neural networks to integrate time information, spatial information and material information to dynamically predict the optimal placement path of materials on the target pallet; among them, the time information includes the arrival time of the material, and the spatial information includes the loading status of the pallet.

[0012] Among them, multiple head components are also used to classify defects of materials in the flexible vibration disk; among them, material information can be queried in the blockchain traceability system.

[0013] Among them, the intelligent adaptive multi-layer flexible material display system also includes an adaptive wireless energy transmission module, which is used to dynamically adjust the wireless energy transmission power and frequency according to the equipment positions and load conditions of multiple carrier lifting components, multiple pallet conveying components, multiple push plate mechanism components, and multiple head components; wherein time-sensitive network communication is adopted between multiple carrier lifting components, multiple pallet conveying components, multiple push plate mechanism components, and multiple head components, and multiple carrier lifting components, multiple pallet conveying components, multiple push plate mechanism components, and multiple head components are self-assembly mechanisms.

[0014] The beneficial effects of the present application are as follows: Different from the prior art, the present application provides an intelligent adaptive multi-layer flexible material tray system, which includes: a plurality of carrier lifting assemblies, each of which is used to carry a basket assembly, wherein the basket assembly is used to accommodate a pallet; a plurality of pallet conveying assemblies, each of which is arranged on the first side of its corresponding basket assembly; a plurality of push plate mechanism assemblies, each of which is arranged on the second side of its corresponding basket assembly, and is used to push the target pallet in the basket assembly to the corresponding pallet conveying assembly; wherein the pallet conveying assembly is used to convey the target pallet to the material placement position, and the pallet conveying assembly is configured to adjust the track width in the pallet conveying assembly according to the size of the target pallet to adapt to the target pallet; a plurality of head assemblies, each of which is used to absorb materials from the corresponding flexible vibration plate, and move to the material placement position to place the materials in the target pallet; wherein, after the materials are placed in the target pallet, the push plate unit in the pallet conveying assembly pushes the target pallet into the basket assembly. Through the above method, each carrier lifting component, each pallet conveying component, each push plate mechanism component and each head component can constitute an independent material plating device, that is, the intelligent adaptive multi-layer flexible material plating system can set up multiple plating stations without affecting each other, thereby improving the plating efficiency of the equipment. At the same time, the pallet conveying component is configured to adjust the track width in the pallet conveying component according to the size of the target pallet to adapt to the target pallet, thereby meeting the plating function requirements of pallets of various sizes. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. Among them:

[0016] Figure 1 It is a structural schematic diagram of an embodiment of an intelligent adaptive multi-layer flexible material tray system provided by the present application;

[0017] Figure 2 A schematic structural diagram of another embodiment of the intelligent adaptive multi-layer flexible material tray system provided by the present application;

[0018] Figure 3 It is a structural schematic diagram of another embodiment of the intelligent adaptive multi-layer flexible material tray system provided by the present application;

[0019] Figure 4 It is a structural schematic diagram of an embodiment of a head assembly provided by the present application;

[0020] Figure 5 yes Figure 4 The corresponding front view;

[0021] Figure 6 It is a structural schematic diagram of an embodiment of a carrier lifting assembly provided by the present application;

[0022] Figure 7 yes Figure 6 The corresponding front view;

[0023] Figure 8 yes Figure 6 The corresponding side view;

[0024] Fig. 9 It is a structural schematic diagram of an embodiment of a pallet conveying assembly provided by the present application;

[0025] Fig.10 yes Fig. 9 The corresponding left view;

[0026] Fig.11 yes Fig. 9 The corresponding front view;

[0027] Fig.12 yes Fig. 9 Corresponding rear view. DETAILED DESCRIPTION

[0028] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It will be understood that the specific embodiments described herein are only used to explain the present application, rather than to limit the present application. It should also be noted that, for ease of description, only some but not all structures related to the present application are shown in the drawings. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0029] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0030] See also Figure 1 , Figure 2 and Figure 3The intelligent self-adaptive multi-layer flexible material tray system 100 includes: a plurality of carrier lifting components 10, a plurality of tray conveying components 20, a plurality of push plate mechanism components 30 and a plurality of head components 40. Further, the intelligent self-adaptive multi-layer flexible material tray system 100 also includes: a housing 70 and an operating device 80.

[0031] Each carrier lifting assembly 10 is used to carry a basket assembly 50 , wherein the basket assembly 50 is used to accommodate a tray.

[0032] Each tray conveying assembly 20 is disposed on a first side of its corresponding basket assembly 50 .

[0033] Each push plate mechanism assembly 30 is arranged on the second side of its corresponding basket assembly 50, and is used to push the target pallet in the basket assembly 50 to the corresponding pallet conveying assembly 20; wherein, the pallet conveying assembly 20 is used to convey the target pallet to the material placement position, and the pallet conveying assembly 20 is configured to adjust the track width in the pallet conveying assembly according to the size of the target pallet to adapt to the target pallet.

[0034] Each head assembly 40 is used to absorb materials from the corresponding flexible vibration plate 60, and move to the material placement position to place the materials in the target pallet; wherein, after the materials are placed in the target pallet, the push plate unit in the pallet conveying assembly 20 pushes the target pallet into the basket assembly 50.

[0035] See also Figure 4 and Figure 5 The head assembly 40 is composed of a motor 45, a motor 43, a slide rail 47, a synchronous wheel 42, a synchronous belt 44, a visual component 41, and a suction nozzle 46. The motor 43 at the top controls the Z-axis lifting and lowering action of the head, and the R-axis motor 45 at the bottom controls the rotation action of the suction nozzle 46. The flexible vibration plate 60 vibrates the material, the visual component 41 (camera) detects the front of the material, and the head suction nozzle 46 sucks the material and puts it into the tray.

[0036] See also Figure 6 , Figure 7 and Figure 8 The carrier lifting assembly 10 is composed of a motor 11, a lead screw 14, a guide rail, a synchronous wheel 12, and a synchronous belt 13. When the basket assembly 50 is manually loaded, it is placed on the carrier lifting Z-axis station. At this time, the carrier lifting assembly 10 pushes the plate push mechanism assembly 30 to push the pallet into the pallet conveying assembly 20 and flows into the next station.

[0037] See also Fig. 9 , Fig.10 , Fig.11 and Fig.12The pallet conveying assembly 20 is composed of a stepper motor 21, a stepper motor 27, a stepper motor 26, a synchronous wheel, a synchronous belt 25, a screw rod 24, a guide rail, a push plate unit 22, and a Z-axis lift 23. The pallet conveying assembly 20 can be adjusted according to the width of the pallet. When the pallet flows into the conveying track formed by the synchronous wheel and the synchronous belt 25, the Z-axis lift 23 lifts the pallet, and the head assembly 40 takes the material and arranges the material on the plate. When the pallet is full, the Z-axis lift 23 lowers the pallet into the conveying track. The push plate unit 22 pushes the pallet into the basket assembly 50.

[0038] Exemplarily, the production process is as follows:

[0039] 1. Manually place the basket assembly 50 filled with empty pallets into the corresponding carrier lifting assembly 10.

[0040] 2. The carrier lifting assembly 10 docks with the plate pushing mechanism assembly 30 to push the pallet into the pallet conveying assembly 20 .

[0041] 3. The pallet in the pallet conveying assembly 20 is lifted by the Z-axis lift 23 at the end of the belt track.

[0042] 4. The flexible vibration plate is loaded with materials. The camera detects the front and back of the materials, and the head assembly 40 sucks eight materials in turn and puts them on the tray.

[0043] 5. After the materials in the pallet are filled in sequence, the pallet conveying assembly 20 conveys the pallet to the basket assembly 50 on the carrier lifting assembly 10 .

[0044] 6. The push plate unit 22 of the tray conveying assembly 20 pushes the trays filled with materials into the basket assembly 50 in sequence. After the basket assembly 50 is full, it is taken away manually.

[0045] In some embodiments, the above-mentioned intelligent adaptive multi-layer flexible material tray system 100 further includes a first multispectral image acquisition component and a processing component. The first multispectral image acquisition component is used to acquire a multispectral image of a target pallet; wherein the multispectral image includes a visible light image, an infrared image, and an ultraviolet image. The processing component is coupled to the first multispectral image acquisition component, and is used to fuse the multispectral images, and identify the target pallet from the fused multispectral images, and control the pallet conveying component to adjust the track width in the pallet conveying component according to the identification result.

[0046] Among them, the processing component is also used to extract image features of visible light images, infrared images and ultraviolet images respectively, and fuse the image features to obtain multi-spectral fusion features, and use a low-rank metric model based on orthogonal learning to reduce the dimension and denoise the multi-spectral fusion features to obtain a low-rank feature representation, and identify the target pallet based on the low-rank feature representation.

[0047] The processing component is also used to compensate for the system error corresponding to the first multispectral image acquisition component according to a preset period.

[0048] Among them, the processing component is also used to perform multi-axis coordinated control of the motors in the carrier lifting component, the pallet conveying component, the push plate mechanism component and the head component.

[0049] That is, the intelligent adaptive multi-layer flexible material tray arrangement system 100 of the present application can realize adaptive tray processing, which is specifically realized by using a high-precision visual recognition system and an adaptive adjustment mechanism driven by a servo motor.

[0050] For example, multi-spectral vision fusion technology is used. In addition to visible light, infrared and ultraviolet spectra are added in this application to enhance the ability to recognize pallets of different materials and colors, reduce ambient light interference, and improve recognition robustness, especially for reflective pallets and transparent pallets.

[0051] The detailed scheme of multispectral vision fusion technology can adopt sensor fusion, feature fusion, adaptive multi-scale feature fusion, low-rank metric based on orthogonal learning, mixed loss function, and data enhancement technology.

[0052] Among them, sensor fusion is mainly reflected in the use of a multispectral camera (the first multispectral image acquisition component) to simultaneously collect visible light, infrared and ultraviolet images, and align these images to form a multispectral data set.

[0053] Among them, feature fusion is mainly reflected in the feature extraction stage, which extracts the features of visible light, infrared and ultraviolet images respectively, and then fuses these features. The fusion method can adopt channel splicing, weighted fusion or attention mechanism.

[0054] Among them, channel stitching can be reflected in directly stitching the feature maps of different spectra in the channel dimension as multi-spectral fusion features.

[0055] Among them, weighted fusion can be reflected in weighted fusion of feature maps according to the contribution of different spectra. For example, the infrared spectrum may have a higher weight at night or in low light conditions.

[0056] Among them, the attention mechanism can be reflected in using the attention mechanism to dynamically adjust the weights of different spectral features, thereby highlighting the features that are beneficial to recognition.

[0057] Among them, adaptive multi-scale feature fusion is mainly reflected in the fused multi-spectral features. The adaptive multi-scale feature fusion module is used to extract shallow features of different scales. This module can solve the recognition problem of pallets of different sizes and types, especially pallets with complex shapes or structures.

[0058] Among them, the low-rank metric based on orthogonal learning is mainly reflected in the use of a low-rank metric model based on orthogonal learning to reduce the dimension and denoise the fused multi-spectral features to obtain a low-rank feature representation. This step can improve the robustness of the model and reduce the impact of noise on the recognition results.

[0059] The hybrid loss function mainly uses the adaptive hybrid loss module to optimize the classification results and solve the problem of uneven distribution of pallet categories. It can also balance the learning capabilities of different categories and improve the classification accuracy of small categories through adaptive weights and multi-classification heads.

[0060] Among them, data enhancement is mainly reflected in the fact that since there are relatively few infrared and ultraviolet image data, data enhancement techniques such as rotation, scaling, translation, noise addition, etc. can be used to expand the training data set.

[0061] Data augmentation methods should be customized according to the characteristics of different spectra.

[0062] The algorithm flow is as follows:

[0063] Visible light images, infrared images and ultraviolet images are collected. Then multispectral image registration is performed to form a multispectral dataset. And the multispectral fusion features are extracted using an adaptive multiscale feature fusion module. And the features are reduced in dimension and denoised using a low-rank metric model of orthogonal learning. And the classification model is optimized using a mixed loss function. Finally, the pallet recognition result is output.

[0064] The technical effects of the above method are: improving recognition robustness, enabling more accurate pallet recognition, and having stronger anti-interference ability.

[0065] Improving recognition robustness is mainly reflected in the fact that multi-spectral visual fusion can effectively reduce the interference of ambient light and improve the robustness of the recognition system under different lighting conditions. The addition of infrared and ultraviolet spectra can enhance the system's recognition ability for different materials and color pallets.

[0066] More accurate pallet identification is mainly reflected in the fact that infrared spectroscopy can distinguish pallets with different thermal characteristics, while ultraviolet spectroscopy can identify pallets with special surface coatings, thereby improving the recognition accuracy of complex pallets. In particular, infrared and ultraviolet spectroscopy can help solve the problem of identifying reflective and transparent pallets.

[0067] Stronger anti-interference ability is mainly reflected in the low-rank metric and mixed loss function through data enhancement, orthogonal learning, which can further improve the system's anti-noise and interference ability.

[0068] Furthermore, the processing components can use self-calibration and compensation algorithms. For example, a self-calibration algorithm based on deep learning can be developed to regularly calibrate the visual system and automatically compensate for system errors caused by temperature changes, vibrations, etc., to ensure long-term stability and high accuracy.

[0069] Furthermore, the processing component can also upgrade the servo motor-driven linear adjustment to multi-axis coordinated adjustment, achieving fine adjustment of the pallet in the horizontal and vertical directions, enhancing adaptability to complex pallet shapes, and making the adjustment process smoother and more stable.

[0070] In some embodiments, the material strength of the carrier lifting assembly 10 is greater than a threshold value, and the material weight is less than a threshold value, wherein the carrier lifting assembly 10 is used to dynamically adjust the operating speed according to the weight of the basket assembly; and each carrier lifting assembly 10 is configured with an independent control unit.

[0071] In some embodiments, the intelligent adaptive multi-layer flexible material placement system 100 of the present application can achieve multi-layer efficient placement, which is specifically embodied in the use of a multi-Z-axis lifting mechanism (carrier lifting assembly 10), a multi-head assembly 40, and a synchronous control algorithm.

[0072] For example, a lightweight modular Z-axis mechanism is used. This is mainly reflected in the use of high-strength lightweight materials and the optimization of the Z-axis structural design to reduce inertia, improve movement speed and response, and reduce energy consumption.

[0073] The use of a distributed multi-core control system is mainly reflected in the upgrading of the synchronization control algorithm to a distributed multi-core control system. Each Z-axis mechanism has an independent control unit, which can be processed in parallel, reducing the computing pressure of the main control unit, achieving more sophisticated collaborative control, and improving the real-time and reliability of the overall operation.

[0074] The use of dynamic load balancing algorithm is mainly reflected in the development of dynamic load balancing algorithm, which dynamically adjusts the running speed and acceleration of the Z axis according to the weight and quantity of materials on each layer of pallet, reduces equipment vibration, improves overall stability, and reduces equipment loss.

[0075] In some embodiments, each head assembly 40 is provided with a flexible material suction nozzle, which is in a vacuum cavity structure, and each vacuum cavity is configured to adjust the vacuum degree according to the shape and size of the material, and each head assembly is used to collect a multispectral image corresponding to the flexible vibration disk 60; wherein the multispectral image includes a visible light image, an infrared image and an ultraviolet image; the multispectral image is fused, and the material posture, surface texture, material, predicted contact position, adsorption force, and temperature in the flexible vibration disk 60 are identified from the fused multispectral image, and the material is adsorbed from the flexible vibration disk according to the material posture, surface texture, material, predicted contact position, adsorption force, and temperature.

[0076] In some embodiments, the intelligent adaptive multi-layer flexible material tray system 100 of the present application can realize the grasping of flexible materials, which is specifically realized by using a flexible material suction nozzle, pressure feedback control, material posture recognition based on deep learning, and a force feedback sensor.

[0077] For example, the use of adaptive multi-cavity vacuum nozzle is mainly reflected in the design of flexible material nozzle into multiple independently controlled vacuum chambers. Each vacuum chamber can independently adjust the vacuum degree according to the shape and size of the material, thereby improving the flexibility and stability of grasping, especially for irregular and fragile materials.

[0078] The use of multimodal tactile feedback is mainly reflected in the addition of multiple tactile sensors such as temperature, vibration and capacitance in addition to force feedback, to build a more comprehensive tactile feedback system, to achieve the perception of material, temperature, surface texture and other characteristics of the material, to further improve the success rate of grasping, and to achieve the identification and safe grasping of special materials.

[0079] The detailed scheme of multimodal tactile feedback can adopt technologies such as multispectral vision, multimodal tactile sensing, deep learning models, data processing and fusion, multimodal data fusion and system integration.

[0080] Among them, multispectral vision is mainly reflected in the use of multispectral cameras to capture visible light, infrared and ultraviolet images to identify the material, color and shape of materials.

[0081] Among them, multimodal tactile sensing mainly includes force sensors, temperature sensors, vibration sensors, capacitive sensors, etc.

[0082] Among them, the force sensor is mainly reflected in the use of microstructure-enhanced flexible force sensors, which are made of transparent PDMS and black graphite / PDMS composite materials to improve sensitivity and spatial resolution.

[0083] Among them, the temperature sensor is mainly embodied in an integrated flexible thermistor or thermocouple, which is used to measure the surface temperature of an object.

[0084] Among them, vibration sensors are mainly embodied in integrated piezoelectric materials or micro-electromechanical systems (MEMS) accelerometers, which are used to sense the surface texture and vibration characteristics of objects.

[0085] Among them, capacitive sensors are mainly integrated capacitive sensors, which are used to measure the capacitance characteristics of objects to distinguish different materials.

[0086] In some embodiments, force, temperature, vibration, and capacitance sensors can be integrated onto the same flexible substrate to form a compact tactile sensing array.

[0087] Among them, signal processing is mainly reflected in the development of corresponding circuits and algorithms to process, filter and calibrate multimodal tactile signals.

[0088] Among them, deep learning models are mainly reflected in technologies such as multimodal feature extraction, feature fusion, multi-task learning, model training, data processing and fusion, and system integration.

[0089] Among them, multimodal feature extraction is mainly reflected in the use of independent CNN models to extract features from multispectral images and data from various tactile sensors.

[0090] Among them, feature fusion is mainly reflected in the use of attention mechanism or gating network to adaptively fuse features from different modalities to obtain a more comprehensive feature representation.

[0091] Among them, multi-task learning is mainly reflected in the use of multi-task learning methods to simultaneously predict the posture, position, force, temperature, surface texture and material category of an object.

[0092] Among them, model training mainly involves using a large amount of experimental data to train deep learning models so that they can accurately predict the properties and states of objects.

[0093] Among them, data processing and fusion are mainly reflected in data registration, multimodal data fusion, and data enhancement.

[0094] Among them, data registration is mainly reflected in the registration of multispectral images and multimodal tactile sensor data to achieve synchronization in time and space.

[0095] Among them, multimodal data fusion is mainly reflected in early fusion, late fusion and dynamic fusion.

[0096] Among them, early fusion is mainly reflected in the fusion of multimodal data before feature extraction, such as channel splicing or weighted summation.

[0097] Among them, late fusion is mainly reflected in the extraction of multimodal features separately, and then the fusion is carried out using attention mechanism, gating network or decision fusion method.

[0098] Among them, dynamic fusion is mainly reflected in the dynamic adjustment of the weights of different modal information according to different tasks and scenarios.

[0099] Among them, data enhancement is mainly reflected in the use of data enhancement methods to increase the diversity of training data and improve the generalization ability of the model.

[0100] Among them, system integration is mainly reflected in hardware integration and software integration.

[0101] Hardware integration mainly consists of integrating a multispectral camera and a multimodal tactile sensing array into a compact sensing unit.

[0102] Software integration is mainly reflected in the development of a unified software framework to achieve the collection, processing, analysis, fusion and visualization of multimodal data.

[0103] The technical effects of the above method mainly include enhanced object recognition capabilities, safer grasping operations, higher grasping success rates, wider application scenarios, and the ability to recognize and safely grasp special materials.

[0104] The enhanced object recognition capability is mainly reflected in the improvement of the recognition accuracy of the object material, temperature and surface texture by fusing multimodal information. For example, it can distinguish between metal and plastic, high-temperature objects and normal-temperature objects, and surfaces with different textures.

[0105] Safer grasping operations mainly rely on the use of multimodal tactile feedback to achieve safe grasping of fragile objects, high-temperature objects, or objects with special surface textures.

[0106] The higher grasping success rate is mainly reflected in the fact that more comprehensive perception and control capabilities can improve the grasping success rate in complex scenarios.

[0107] The wider application scenarios are mainly reflected in the fact that the system can be applied to a wider range of fields, such as intelligent manufacturing, medical robots, home service robots and wearable electronic devices.

[0108] The recognition and safe grasping of special materials is mainly reflected in the fact that by combining multimodal tactile feedback, special materials can be recognized, such as high-temperature objects, fragile objects or objects with special surface textures, to ensure safe grasping.

[0109] The specific algorithm process includes: data acquisition, preprocessing, feature extraction, data fusion, multi-task prediction and output.

[0110] Data collection mainly involves using multispectral cameras to collect visible light, infrared and ultraviolet images of materials, and using multimodal tactile sensor arrays to collect contact force, temperature, vibration and capacitance data.

[0111] Preprocessing mainly involves preprocessing of multispectral images and multimodal tactile data, such as denoising, enhancement, filtering, and calibration.

[0112] Feature extraction mainly involves using independent CNN models to extract features from multispectral images and data from various tactile sensors.

[0113] Data fusion is mainly reflected in the use of attention mechanisms or gating networks to adaptively fuse features from different modalities.

[0114] Multi-task prediction mainly involves using deep learning models to identify material postures and simultaneously predict contact positions, force magnitudes, temperatures, surface textures, and material categories.

[0115] The output is mainly reflected in the output material's posture information, force feedback information, temperature information, surface texture information and material category.

[0116] In some embodiments, gesture recognition based on 3D point clouds may be used, which may specifically upgrade gesture recognition based on 2D images to gesture recognition based on 3D point clouds, so as to more accurately obtain the three-dimensional information of the material, reduce the recognition errors caused by factors such as lighting changes and occlusion, and make the grasping gesture more accurate.

[0117] 3D point cloud posture recognition mainly includes 3D Gaussian sputtering initialization, adaptive density control, posture estimation, etc.

[0118] Among them, the 3D Gaussian sputtering initialization is mainly reflected in the use of a 3D Gaussian sputtering framework based on point graph prior and K-nearest neighbor algorithm adaptive density control, and the use of a view similarity-based grouping strategy and a powerful point graph prior to initialize the 3D Gaussian ellipsoid to obtain more accurate point clouds and camera poses.

[0119] Among them, adaptive density control is mainly reflected in the use of K-nearest neighbor algorithm to adaptively segment Gaussian basis elements according to the shape differences of adjacent Gaussian ellipsoids, thereby realizing dynamic adjustment and optimization of the point cloud.

[0120] Among them, posture estimation is mainly reflected in the use of deep learning models to extract features from 3D point clouds and predict the 3D posture of objects. For example, a layered approach can be used to learn local features of point clouds or the Transformer's self-attention mechanism can be applied to network structures such as 3D point cloud data.

[0121] Among them, multimodal tactile sensing mainly includes force sensors, temperature sensors, vibration sensors, capacitive sensors, etc.

[0122] Among them, the force sensor is mainly reflected in the use of microstructure-enhanced flexible force sensors, which are made of transparent PDMS and black graphite / PDMS composite materials to improve sensitivity and spatial resolution.

[0123] Among them, the temperature sensor is mainly embodied in an integrated flexible thermistor or thermocouple, which is used to measure the surface temperature of an object.

[0124] Among them, vibration sensors are mainly embodied in integrated piezoelectric materials or micro-electromechanical systems (MEMS) accelerometers, which are used to sense the surface texture and vibration characteristics of objects.

[0125] Among them, capacitive sensors are mainly integrated capacitive sensors, which are used to measure the capacitance characteristics of objects to distinguish different materials.

[0126] In some embodiments, force, temperature, vibration, and capacitance sensors can be integrated onto the same flexible substrate to form a compact tactile sensing array.

[0127] Among them, signal processing is mainly reflected in the development of corresponding circuits and algorithms to process, filter and calibrate multimodal tactile signals.

[0128] Among them, deep learning models are mainly reflected in technologies such as multimodal feature extraction, feature fusion, multi-task learning, model training, data processing and fusion, and system integration.

[0129] Among them, multimodal feature extraction is mainly reflected in the use of independent CNN models to extract features from multispectral images, 3D point clouds and data from various tactile sensors.

[0130] Among them, feature fusion is mainly reflected in the use of attention mechanism or gating network to adaptively fuse features from different modalities to obtain a more comprehensive feature representation.

[0131] Among them, multi-task learning is mainly reflected in the use of multi-task learning methods to simultaneously predict the posture, position, force, temperature, surface texture and material category of an object.

[0132] Among them, model training mainly involves using a large amount of experimental data to train deep learning models so that they can accurately predict the properties and states of objects.

[0133] Among them, data processing and fusion are mainly reflected in data registration, multimodal data fusion, and data enhancement.

[0134] Among them, data registration is mainly reflected in the registration of multispectral images, 3D point clouds and multimodal tactile sensor data to achieve synchronization in time and space.

[0135] Among them, multimodal data fusion is mainly reflected in early fusion, late fusion and dynamic fusion.

[0136] Among them, early fusion is mainly reflected in the fusion of multimodal data before feature extraction, such as channel splicing or weighted summation.

[0137] Among them, late fusion is mainly reflected in the extraction of multimodal features separately, and then the fusion is carried out using attention mechanism, gating network or decision fusion method.

[0138] Among them, dynamic fusion is mainly reflected in the dynamic adjustment of the weights of different modal information according to different tasks and scenarios.

[0139] Among them, data enhancement is mainly reflected in the use of data enhancement methods to increase the diversity of training data and improve the generalization ability of the model.

[0140] Among them, system integration is mainly reflected in hardware integration and software integration.

[0141] Hardware integration mainly consists of integrating a multispectral camera, a 3D point cloud sensor, and a multimodal tactile sensing array into a compact sensing unit.

[0142] Software integration is mainly reflected in the development of a unified software framework to achieve the collection, processing, analysis, fusion and visualization of multimodal data.

[0143] The technical effects of the above method mainly include more accurate posture recognition, enhanced object recognition capability, safer grasping operation, higher grasping success rate, wider application scenarios and the ability to realize the recognition and safe grasping of special materials.

[0144] More accurate posture recognition is mainly reflected in the fact that posture recognition based on 3D point cloud can more accurately obtain the three-dimensional information of the object, reduce the recognition errors caused by factors such as lighting changes and occlusion, and make the grasping posture more accurate.

[0145] The enhanced object recognition capability is mainly reflected in the improvement of the recognition accuracy of the object material, temperature and surface texture by fusing multimodal information. For example, it can distinguish between metal and plastic, high-temperature objects and normal-temperature objects, and surfaces with different textures.

[0146] Safer grasping operations mainly rely on the use of multimodal tactile feedback to achieve safe grasping of fragile objects, high-temperature objects, or objects with special surface textures.

[0147] The higher grasping success rate is mainly reflected in the fact that more comprehensive perception and control capabilities can improve the grasping success rate in complex scenarios.

[0148] The wider application scenarios are mainly reflected in the fact that the system can be applied to a wider range of fields, such as intelligent manufacturing, medical robots, home service robots and wearable electronic devices.

[0149] The recognition and safe grasping of special materials is mainly reflected in the fact that by combining multimodal tactile feedback, special materials can be recognized, such as high-temperature objects, fragile objects or objects with special surface textures, to ensure safe grasping.

[0150] The specific algorithm process includes: data acquisition, preprocessing, feature extraction, data fusion, multi-task prediction and output.

[0151] Data collection mainly involves using multispectral cameras to collect visible light, infrared and ultraviolet images of materials, using 3D point cloud sensors to collect three-dimensional point cloud data of materials, and using multimodal tactile sensor arrays to collect contact force, temperature, vibration and capacitance data.

[0152] Preprocessing mainly involves preprocessing of multispectral images, 3D point clouds and multimodal tactile data, such as denoising, enhancement, filtering and calibration.

[0153] Feature extraction mainly involves using independent CNN models to extract features from multispectral images, 3D point clouds, and data from various tactile sensors.

[0154] Data fusion is mainly reflected in the use of attention mechanisms or gating networks to adaptively fuse features from different modalities.

[0155] Multi-task prediction mainly involves using deep learning models to identify material postures and simultaneously predict contact positions, force magnitudes, temperatures, surface textures, and material categories.

[0156] The output is mainly reflected in the output material's posture information, force feedback information, temperature information, surface texture information and material category.

[0157] In some embodiments, multiple carrier lifting components 10, multiple pallet conveying components 20, multiple push plate mechanism components 30, use value-based reinforcement learning and strategy-based reinforcement learning to optimize the moving path, and multiple head components 40 use a spatiotemporal graph neural network to integrate time information, spatial information and material information to dynamically predict the optimal placement path of the material on the target pallet; wherein the time information includes the arrival time of the material, and the spatial information includes the loading status of the pallet.

[0158] In some embodiments, the intelligent adaptive multi-layer flexible material arrangement system 100 of the present application can realize intelligent path optimization, which is specifically embodied in the use of reinforcement learning algorithm and genetic algorithm.

[0159] For example, the use of hybrid reinforcement learning mainly combines traditional value-based reinforcement learning with strategy-based reinforcement learning to achieve faster and more stable path optimization, enabling it to better adapt to dynamic environments.

[0160] For example, the use of spatiotemporal graph neural networks is mainly reflected in the introduction of spatiotemporal graph neural networks, which comprehensively consider time, space and material information, dynamically predict the optimal placement path, reduce mechanical movement and waiting time, improve the overall efficiency of the system, and cope with complex and changeable placement needs.

[0161] For example, the use of prediction-based dynamic planning is mainly reflected in the fact that the prediction-based dynamic planning algorithm can pre-plan the optimal placement path according to the arrival time of the materials and the loading status of the pallets, achieve early optimization, avoid congestion, and maximize efficiency.

[0162] Among them, multiple head assemblies 40 are also used to classify defects of materials in the flexible vibration disk 60; wherein, material information can be queried in the blockchain traceability system.

[0163] In some embodiments, the intelligent adaptive multi-layer flexible material tray system 100 of the present application can realize integrated quality control, which is specifically embodied in the use of a high-precision three-dimensional visual inspection system, automatic sorting RFID technology, and two-dimensional code technology.

[0164] For example, AI-driven defect classification is mainly reflected in the defect classification algorithm based on deep learning, which can identify and classify different types of material defects, such as cracks, scratches, deformation, surface contamination, etc., providing more comprehensive information for quality control.

[0165] For example, the use of a blockchain traceability system mainly involves using blockchain technology to build a safe and reliable production traceability system, ensuring that data cannot be tampered with, making it convenient for users to query material production information, improving the transparency and reliability of the traceability system, and ensuring product quality.

[0166] For example, the use of edge computing is mainly reflected in the fact that some computing tasks are placed on the device side, reducing data transmission delays, improving real-time performance, and reducing dependence on cloud platforms. Edge computing can quickly process and feedback detection results, improving detection and sorting efficiency.

[0167] In some embodiments, the intelligent adaptive multi-layer flexible material display system 100 also includes an adaptive wireless energy transmission module, which is used to dynamically adjust the wireless energy transmission power and frequency according to the equipment positions and load conditions of multiple carrier lifting components 10, multiple pallet conveying components 20, multiple push plate mechanism components 30, and multiple head components 40; wherein, time-sensitive network communication is adopted between the multiple carrier lifting components 10, multiple pallet conveying components 20, multiple push plate mechanism components 30, and multiple head components 40, and the multiple carrier lifting components 10, multiple pallet conveying components 20, multiple push plate mechanism components 30, and multiple head components 40 are self-assembly mechanisms.

[0168] In some embodiments, the intelligent adaptive multi-layer flexible material tray system 100 of the present application can achieve wireless and flexible deployment, which is specifically embodied in the use of wireless energy transmission, 5G wireless communication, modular structure design, and quick connection mechanism.

[0169] For example, the use of adaptive wireless energy transmission is mainly reflected in the use of adaptive wireless energy transmission technology, which dynamically adjusts the wireless energy transmission power and frequency according to the device location and load conditions, ensures efficient and reliable wireless power supply, and realizes collaborative energy management among multiple devices.

[0170] For example, the use of time-sensitive networking (TSN) is mainly reflected in upgrading 5G wireless communications to time-sensitive networks to ensure the real-time and reliability of data transmission, especially in scenarios that require precise synchronization control.

[0171] For example, the use of modular self-assembly mechanism is mainly reflected in the upgrading of modular design to self-assembly mechanism. The equipment can be automatically assembled and disassembled according to needs, which simplifies the deployment process, shortens the deployment time, and improves the usability of the equipment.

[0172] The intelligent adaptive multi-layer flexible material tray system of the present application can have the following technical effects:

[0173] 1. Greatly improve equipment compatibility: automatically adapt to pallets of different sizes, expand the scope of equipment application, and improve equipment utilization.

[0174] 2. Significantly improve placement efficiency: multi-layer synchronous placement and intelligent path optimization greatly shorten the production cycle.

[0175] 3. Improve material adaptability: Flexible suction nozzle and force feedback gripping significantly improve the gripping ability and placement accuracy of special-shaped and fragile materials.

[0176] 4. Realize intelligent control: AI path optimization, automatic sorting and material traceability reduce manual intervention and improve the level of production intelligence.

[0177] 5. Improve system flexibility: Wireless deployment and modular design allow the equipment to be quickly deployed and flexibly adjusted to meet different production needs.

[0178] 6. Improve production quality and management level: Accurate material detection, sorting and pallet traceability functions ensure product quality and improve production management level.

[0179] 7. Achieve energy saving and remote monitoring: Energy recovery, wireless power supply, and remote monitoring and maintenance are adopted to achieve energy saving and high reliability operation of equipment.

[0180] In the several embodiments provided in this application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device implementation described above is only illustrative, for example, the division of the modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0181] If the integrated units in the above other embodiments are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) or a processor (processor) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, RandomAccess Memory), disk or optical disk and other media that can store program codes.

[0182] The above description is only an implementation method of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly used in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. An intelligent adaptive multi-layer flexible material tray system, characterized in that: The intelligent self-adaptive multi-layer flexible material tray system comprises: A plurality of carrier lifting assemblies, each of the carrier lifting assemblies being used to carry a basket assembly, wherein the basket assembly is used to accommodate a tray; A plurality of tray conveying assemblies, each of the tray conveying assemblies being disposed on a first side of the corresponding basket assembly; A plurality of push plate mechanism assemblies, each of which is disposed on the second side of the corresponding basket assembly, and is used to push the target pallet in the basket assembly to the corresponding pallet conveying assembly; wherein the pallet conveying assembly is used to convey the target pallet to the material placement position, and the pallet conveying assembly is configured to adjust the track width in the pallet conveying assembly according to the size of the target pallet to adapt to the target pallet; A plurality of head assemblies, each of which is used to absorb materials from a corresponding flexible vibration plate, and move to the material placement position to place the materials in the target tray; After the materials are placed in the target pallet, the push plate unit in the pallet conveying assembly pushes the target pallet into the basket assembly.

2. The intelligent adaptive multi-layer flexible material tray system according to claim 1 is characterized in that: The intelligent self-adaptive multi-layer flexible material tray system also includes: A first multispectral image acquisition component, used to acquire a multispectral image of the target pallet; wherein the multispectral image includes a visible light image, an infrared image, and an ultraviolet image; A processing component is coupled to the first multispectral image acquisition component, and is used to fuse the multispectral images, identify the target pallet from the fused multispectral images, and control the pallet conveying component to adjust the track width in the pallet conveying component according to the identification result.

3. The intelligent adaptive multi-layer flexible material tray system according to claim 2 is characterized in that: The processing component is also used to extract image features of the visible light image, the infrared image and the ultraviolet image respectively, and fuse the image features to obtain multi-spectral fusion features, and use a low-rank metric model based on orthogonal learning to reduce the dimension and denoise the multi-spectral fusion features to obtain a low-rank feature representation, and identify the target pallet based on the low-rank feature representation.

4. The intelligent adaptive multi-layer flexible material tray system according to claim 2 is characterized in that: The processing component is also used to compensate for the system error corresponding to the first multi-spectral image acquisition component according to a preset period.

5. The intelligent adaptive multi-layer flexible material tray system according to any one of claims 2 to 4, characterized in that: The processing component is also used to perform multi-axis coordinated control of the motors in the carrier lifting component, the pallet conveying component, the push plate mechanism component and the head component.

6. The intelligent adaptive multi-layer flexible material tray system according to claim 1 is characterized in that: The material strength of the carrier lifting assembly is greater than a threshold value, and the material weight is less than a threshold value, wherein the carrier lifting assembly is used to dynamically adjust the operating speed according to the weight of the basket assembly; and each of the carrier lifting assemblies is configured with an independent control unit.

7. The intelligent adaptive multi-layer flexible material tray system according to claim 1 is characterized in that: Each of the head components is provided with a flexible material suction nozzle, and the flexible material suction nozzle is a vacuum cavity structure, and each of the vacuum cavity is configured to adjust the vacuum degree according to the shape and size of the material, and each of the head components is used to collect a multispectral image corresponding to the flexible vibration disk; wherein the multispectral image includes a visible light image, an infrared image and an ultraviolet image; the multispectral images are fused, and the material posture, surface texture, material, predicted contact position, adsorption force, and temperature in the flexible vibration disk are identified from the fused multispectral images, and the material is adsorbed from the flexible vibration disk according to the material posture, the surface texture, the material, the predicted contact position, the adsorption force, and the temperature.

8. The intelligent adaptive multi-layer flexible material tray system according to claim 1, characterized in that: Multiple carrier lifting components, multiple pallet conveying components, multiple push plate mechanism components, use value-based reinforcement learning and strategy-based reinforcement learning to optimize the moving path, and multiple head components use spatiotemporal graph neural networks to integrate time information, spatial information and material information to dynamically predict the optimal placement path of materials on the target pallet; among them, the time information includes the arrival time of the material, and the spatial information includes the loading status of the pallet.

9. The intelligent adaptive multi-layer flexible material tray system according to claim 1, characterized in that: Multiple head components are also used to classify defects in materials in the flexible vibration plate; among them, material information can be queried in the blockchain traceability system.

10. The intelligent adaptive multi-layer flexible material tray system according to claim 1, characterized in that: The intelligent adaptive multi-layer flexible material display system also includes an adaptive wireless energy transmission module, which is used to dynamically adjust the wireless energy transmission power and frequency according to the equipment positions and load conditions of multiple carrier lifting components, multiple pallet conveying components, multiple push plate mechanism components, and multiple head components; wherein time-sensitive network communication is adopted between the multiple carrier lifting components, multiple pallet conveying components, multiple push plate mechanism components, and multiple head components, and the multiple carrier lifting components, multiple pallet conveying components, multiple push plate mechanism components, and multiple head components are self-assembly mechanisms.

Images