Intelligent robot spraying path automatic generation system and method

Abstract

The invention relates to the technical field of intelligent control, and particularly discloses an automatic generation system and method for a spraying path of an intelligent robot, and the system employs an artificial intelligence technology based on deep learning to carry out the analysis and processing of the acceleration and angular velocity data, collected by an accelerometer and a gyroscope, of the movement of a robot nozzle. Movement track information in the spraying process of the robot is excavated, and the acceleration and the angular speed of a spray head are adjusted in a self-adaptive mode in combination with surface state information of a sprayed product. In this way, the spraying track of the robot can be optimized, automatic generation of the spraying path is achieved, and therefore the efficiency and quality of spraying operation are improved.

Description

Technical Field

[0001] The present application relates to the field of intelligent control technology, and more specifically, to an intelligent robot spray path automatic generation system and method. Background Art

[0002] With the rapid development of industrial automation and intelligent manufacturing technology, spraying operations are an indispensable part of the product manufacturing process, and their efficiency and quality have a significant impact on the appearance and performance of the final product. Traditional spraying operations usually rely on manual operations, which are not only inefficient, but also easily affected by human factors, resulting in unstable spraying quality.

[0003] With the continuous development of artificial intelligence technology, more and more companies are trying to apply intelligent robots to spraying operations to improve spraying efficiency and quality. However, the existing intelligent robot spraying system can only operate according to the preset spraying path, and cannot make adaptive adjustments according to the actual situation of the product surface, resulting in unsatisfactory spraying effect. Therefore, an optimized intelligent robot spraying path automatic generation system and method are expected. Summary of the invention

[0004] In order to solve the above technical problems, this application is proposed. The embodiment of the present application provides an intelligent robot spray path automatic generation system and method, which uses artificial intelligence technology based on deep learning to analyze and process the acceleration and angular velocity data of the robot nozzle movement collected by the accelerometer and gyroscope, dig out the motion trajectory information of the robot during the spraying process, and combine the surface state information of the sprayed product to adaptively adjust the acceleration and angular velocity of the nozzle. In this way, the robot's spraying trajectory can be optimized, and the automatic generation of the spraying path can be achieved, thereby improving the efficiency and quality of the spraying operation.

[0005] Accordingly, according to one aspect of the present application, there is provided an intelligent robot spraying path automatic generation system, which comprises:

[0006] A robot spraying monitoring module, which is used for the surface state image of the sprayed product at the current time point collected by the industrial camera deployed on the intelligent robot, and the time series of acceleration values ​​and angular velocity values ​​within a predetermined time period including the current time point collected by the accelerometer and gyroscope deployed on the spray head of the intelligent robot;

[0007] A multi-scale spray state feature extraction module is used to extract multi-scale spray state features from the spray product surface state image to obtain a multi-scale product surface spray state feature vector;

[0008] A spraying trajectory feature extraction module, used to extract spraying trajectory features from the time series of the acceleration value and the time series of the angular velocity value to obtain a robot spraying trajectory feature vector;

[0009] An association encoding module, used for fine-tuning the inter-domain association of the multi-scale product surface spraying state feature vector and the robot spraying trajectory feature vector based on the feature selection space to obtain a spraying trajectory-product surface state association interaction feature vector;

[0010] The robot spraying path control module is used to determine the robot control instructions based on the spraying trajectory-product surface state associated interactive feature vector.

[0011] According to another aspect of the present application, a method for automatically generating a spraying path of an intelligent robot is provided, which comprises:

[0012] The surface state image of the sprayed product at the current time point collected by the industrial camera deployed on the intelligent robot, and the time series of acceleration values ​​and angular velocity values ​​within a predetermined time period including the current time point collected by the accelerometer and gyroscope deployed on the spray head of the intelligent robot;

[0013] Performing multi-scale spray state feature extraction on the spray product surface state image to obtain a multi-scale product surface spray state feature vector;

[0014] Extracting spraying trajectory features from the time series of the acceleration values ​​and the time series of the angular velocity values ​​to obtain a robot spraying trajectory feature vector;

[0015] Performing inter-domain correlation fine-tuning on the multi-scale product surface spraying state feature vector and the robot spraying trajectory feature vector based on feature selection space to obtain a spraying trajectory-product surface state correlation interaction feature vector;

[0016] Based on the spraying trajectory-product surface state associated interactive feature vector, the control instructions of the robot are determined.

[0017] Compared with the prior art, the intelligent robot spray path automatic generation system and method provided by the present application uses artificial intelligence technology based on deep learning to analyze and process the acceleration and angular velocity data of the robot spray head movement collected by the accelerometer and gyroscope, dig out the motion trajectory information of the robot during the spraying process, and combine the surface state information of the sprayed product to adaptively adjust the acceleration and angular velocity of the spray head. In this way, the robot's spraying trajectory can be optimized, and the automatic generation of the spray path can be realized, thereby improving the efficiency and quality of the spraying operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0019] Figure 1 It is a block diagram of an intelligent robot spray path automatic generation system according to an embodiment of the present application.

[0020] Figure 2 Schematic diagram of the architecture of the intelligent robot spray path automatic generation system according to an embodiment of the present application.

[0021] Figure 3 It is a block diagram of a spray trajectory feature extraction module in the intelligent robot spray path automatic generation system according to an embodiment of the present application.

[0022] Figure 4 The present invention is a flowchart of a method for automatically generating a spraying path for an intelligent robot according to an embodiment of the present application. DETAILED DESCRIPTION

[0023] Below, the embodiments of the present application will be described in more detail in conjunction with the accompanying drawings, and the above and other purposes, features and advantages of the present application will become more apparent. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the example embodiments described herein.

[0024] Figure 1 It is a block diagram of an intelligent robot spray path automatic generation system according to an embodiment of the present application.

[0025] Figure 2 FIG. 1 is a schematic diagram of the architecture of the automatic generation system of the spraying path of the intelligent robot according to the embodiment of the present application. Figure 1 and Figure 2As shown, according to the embodiment of the present application, the intelligent robot spraying path automatic generation system 100 includes: a robot spraying monitoring module 110, which is used to collect the surface state image of the sprayed product at the current time point by the industrial camera deployed on the intelligent robot, and the time series of acceleration values ​​and angular velocity values ​​within a predetermined time period including the current time point collected by the accelerometer and gyroscope deployed on the nozzle of the intelligent robot; a multi-scale spraying state feature extraction module 120, which is used to extract the multi-scale spraying state features of the surface state image of the sprayed product to obtain the multi-scale product surface spraying state feature. state feature vector; a spray trajectory feature extraction module 130, used to extract spray trajectory features from the time series of the acceleration value and the time series of the angular velocity value to obtain a robot spray trajectory feature vector; an association encoding module 140, used to perform inter-domain association fine-tuning on the multi-scale product surface spray state feature vector and the robot spray trajectory feature vector based on the feature selection space to obtain a spray trajectory-product surface state association interaction feature vector; a robot spray path control module 150, used to determine the robot control instructions based on the spray trajectory-product surface state association interaction feature vector.

[0026] As mentioned in the above background technology, the existing intelligent robot spraying system can often only operate according to the preset spraying path, and cannot be adaptively adjusted according to the actual situation of the product surface, resulting in unsatisfactory spraying effect. In response to the above technical problems, the technical concept of this application is to use artificial intelligence technology based on deep learning to analyze and process the acceleration and angular velocity data of the robot nozzle movement collected by the accelerometer and gyroscope, to mine the motion trajectory information of the robot during the spraying process, and to combine the surface state information of the sprayed product to adaptively adjust the acceleration and angular velocity of the nozzle. In this way, the robot's spraying trajectory can be optimized, and the automatic generation of the spraying path can be realized, thereby improving the efficiency and quality of the spraying operation.

[0027] In the above-mentioned intelligent robot spraying path automatic generation system 100, the robot spraying monitoring module 110 is used for the surface state image of the sprayed product at the current time point collected by the industrial camera deployed on the intelligent robot, and the time series of acceleration values ​​and angular velocity values ​​within a predetermined time period including the current time point collected by the accelerometer and gyroscope deployed on the nozzle of the intelligent robot. It should be understood that the industrial camera deployed on the intelligent robot can capture the state image of the surface of the sprayed product in real time, so as to identify the uniformity of the spraying, the thickness of the coating, the presence or absence of leakage or overspray, and other problems, so as to adaptively adjust the spraying path according to this information. At the same time, the accelerometer and gyroscope are deployed on the robot nozzle, which can collect the acceleration and angular velocity data of the nozzle movement in real time, reflecting the dynamic change information of the nozzle during the spraying process. At the same time, the motion trajectory of the nozzle has an important correlation effect on the spraying effect of the product surface. Therefore, the movement trajectory of the nozzle can be inferred based on the acceleration and angular velocity data during the movement of the nozzle, so as to predict and optimize the future spraying path according to the associated impact of the historical spraying trajectory on the spraying state of the product surface.

[0028] In the above-mentioned intelligent robot spray path automatic generation system 100, the multi-scale spray state feature extraction module 120 is used to perform multi-scale spray state feature extraction on the spray product surface state image to obtain a multi-scale product surface spray state feature vector. In a specific example of the present application, the encoding method of performing multi-scale spray state feature extraction on the spray product surface state image to obtain a multi-scale product surface spray state feature vector is to pass the spray product surface state image through a spray state multi-scale feature extractor based on a pyramid network to obtain the multi-scale product surface spray state feature vector. It should be understood that considering that the spray state of the product surface has multiple scale feature representations, including microscopic coating thickness, uniformity, and macroscopic spray area, shape, etc. Therefore, in order to fully obtain the spray state information of the product surface, in the technical solution of the present application, a spray state multi-scale feature extractor based on a pyramid network is used to extract and fuse image features of different scales on the spray product surface state image to obtain multiple levels of spray state information. Specifically, the pyramid network creates a series of image pyramids of different resolutions by multiple downsampling of the surface state image of the sprayed product, thereby extracting features in image feature spaces of different scales, decomposing and representing the surface state image of the sprayed product at multiple scales, and fusing the extracted image feature representations of different scales to form a multi-scale product surface spray state feature vector, which provides rich feature representation for the subsequent correlation analysis between the spraying trajectory and the spraying state.

[0029] In the above-mentioned intelligent robot spraying path automatic generation system 100, the spraying trajectory feature extraction module 130 is used to extract the spraying trajectory features from the time series of the acceleration value and the time series of the angular velocity value to obtain the robot spraying trajectory feature vector. Figure 3 FIG. 4 is a block diagram of a spraying trajectory feature extraction module in an intelligent robot spraying path automatic generation system according to an embodiment of the present application. Figure 3 As shown, the spray trajectory feature extraction module 130 includes: a data regularization unit 131, which is used to arrange the time series of the acceleration values ​​and the time series of the angular velocity values ​​into a nozzle movement parameter timing input matrix according to the time dimension and the parameter sample dimension; a robot spray trajectory pattern recognition unit 132, which is used to perform spray trajectory pattern recognition on the nozzle movement parameter timing input matrix to obtain the robot spray trajectory feature vector.

[0030] Specifically, the data regularization unit 131 is used to arrange the time series of the acceleration value and the time series of the angular velocity value into a nozzle movement parameter time series input matrix according to the time dimension and the parameter sample dimension. It should be understood that during the spraying process, the motion trajectory of the nozzle may be very complex, including multiple motion forms such as linear motion, curved motion, and rotation. The motion form of the nozzle will be reflected in its acceleration and angular velocity data. At the same time, since the acceleration and angular velocity changes of the nozzle during the spraying process are continuously changed over time, that is, the two have a time series correlation dependency characteristic in the time dimension. Therefore, in order to ensure that the time continuity and correlation of the data are maintained, the dynamic characteristics of the nozzle during the spraying process are more accurately reflected. Before extracting the spraying trajectory features, the time series data of the acceleration and angular velocity of the nozzle are further regularized, that is, the time series of the acceleration value and the time series of the angular velocity value are arranged in a matrix form according to the time dimension and the parameter sample dimension, so as to facilitate subsequent feature extraction and pattern recognition.

[0031] Specifically, the robot spraying trajectory pattern recognition unit 132 is used to perform spraying trajectory pattern recognition on the nozzle movement parameter timing input matrix to obtain the robot spraying trajectory feature vector. In a specific example of the present application, the encoding method of performing spraying trajectory pattern recognition on the nozzle movement parameter timing input matrix to obtain the robot spraying trajectory feature vector is to pass the nozzle movement parameter timing input matrix through a spraying trajectory feature extractor based on a convolutional neural network model to obtain the robot spraying trajectory feature vector. It should be understood that convolutional neural networks have powerful automatic feature learning capabilities and can effectively extract useful feature representations from raw data. Therefore, in the technical solution of the present application, a spraying trajectory feature extractor based on a convolutional neural network model is used to perform spraying trajectory pattern recognition of the nozzle. Specifically, the spraying trajectory feature extractor processes the nozzle movement parameter time series input matrix based on network structures such as convolutional layers and pooling layers to effectively fuse the acceleration and angular velocity information of different dimensions, automatically learn the movement mode of the nozzle during the spraying process, and capture the main characteristics and changing trends of the nozzle movement trajectory, thereby obtaining the robot spraying trajectory feature vector, which provides an effective feature representation for the subsequent correlation analysis between the spraying trajectory and the spraying state.

[0032] In the above-mentioned intelligent robot spray path automatic generation system 100, the association coding module 140 is used to perform inter-domain association fine-tuning on the multi-scale product surface spray state feature vector and the robot spray trajectory feature vector based on the feature selection space to obtain the spray trajectory-product surface state association interaction feature vector. It should be understood that the spray product surface state image provides visual information of the spraying effect, while the accelerometer and gyroscope data provide dynamic information of the nozzle movement. By fusing the multi-scale product surface spray state feature vector and the robot spray trajectory feature vector, the association relationship between the spray trajectory and the product surface state can be established, revealing the influence of the spray trajectory on the product surface spray effect, thereby providing strong support for subsequent path planning.

[0033] In particular, considering that the multi-scale product surface spraying state feature vector and the robot spraying trajectory feature vector may be distributed in different feature spaces, this is because the two are derived from completely different sensor data types - image and motion parameter time series. Image data is captured by industrial cameras and carries rich visual information, such as color, texture and shape, which is suitable for identifying objects and understanding the semantic content of scenes; while the time series of acceleration values ​​and angular velocity values ​​provided by accelerometers and gyroscopes record the dynamic behavior of the nozzle, mainly reflecting the geometric structure and spatial layout of the physical world, especially for accurately describing the movement path of the nozzle. When these raw data are processed by their respective feature extractors, although they can obtain feature representations in their respective fields, they still retain the characteristics of the original sensor data. The multi-scale product surface spraying state feature vector focuses more on visual pattern recognition and scene understanding, emphasizing the position, size and relationship of visible objects in the environment; in contrast, the robot spraying trajectory feature vector reflects more the spatiotemporal characteristics of the nozzle movement and focuses on the kinematic spraying behavior. Since these two feature vectors focus on information at the visual level and the geometric level respectively, even if they are used to describe different aspects of the same spraying process, they have differences in numerical representation and carry different types of data distribution and statistical properties. Therefore, when directly migrating the knowledge of one field (for example, a model trained only on image data) to another field (i.e., a new environment that integrates image and motion data), problems of systematic deviations will be encountered. In order to solve this problem, in the technical solution of the present application, the multi-scale product surface spraying state feature vector and the robot spraying trajectory feature vector are fine-tuned based on the inter-domain correlation of the feature selection space to obtain the spraying trajectory-product surface state correlation interaction feature vector.

[0034] Specifically, in the embodiment of the present application, the association coding module includes: a feature vector intrinsic decomposition unit, which is used to map the multi-scale product surface spraying state feature vector and the robot spraying trajectory feature vector to the intrinsic decomposition space to obtain the multi-scale product surface spraying state intrinsic feature vector and the robot spraying trajectory intrinsic feature vector; an intrinsic feature amplitude tuning unit, which is used to perform target domain driven feature amplitude tuning on the multi-scale product surface spraying state intrinsic feature vector and the robot spraying trajectory intrinsic feature vector to obtain the multi-scale product surface spraying state modulated intrinsic feature vector and the robot spraying trajectory. A spray trajectory-product surface state associated interactive feature generating unit is used to fuse the multi-scale product surface spray state modulated intrinsic feature vector and the robot spray trajectory modulated intrinsic feature vector based on the spray trajectory-product surface state fine-grained semantic information field to obtain the spray trajectory-product surface state associated interactive feature vector.

[0035] More specifically, in an embodiment of the present application, the fine-grained semantic information field construction unit is used to: multiply the transposed vector of the modulated intrinsic feature vector of the multi-scale product surface spraying state and the modulated intrinsic feature vector of the robot spraying trajectory, and then divide the transposed vector by the square root of the scale of the modulated intrinsic feature vector of the robot spraying trajectory to obtain a fine-grained interaction matrix of the spraying trajectory-product surface state; and perform convolution encoding on the fine-grained interaction matrix of the spraying trajectory-product surface state to obtain the fine-grained semantic information field of the spraying trajectory-product surface state.

[0036] More specifically, in an embodiment of the present application, the spray trajectory-product surface state associated interactive feature generation unit is used to: multiply the spray trajectory-product surface state fine-grained semantic information field by the multi-scale product surface spray state modulated intrinsic feature vector to obtain the multi-scale product surface spray state feature vector after information field mapping; multiply the spray trajectory-product surface state fine-grained semantic information field by the robot spray trajectory modulated intrinsic feature vector to obtain the robot spray trajectory feature vector after information field mapping; perform weighted fusion on the multi-scale product surface spray state feature vector after information field mapping and the robot spray trajectory feature vector after information field mapping to obtain the spray trajectory-product surface state associated interactive feature vector.

[0037] In the embodiment of the present application, specifically, the multi-scale product surface spraying state feature vector and the robot spraying trajectory feature vector are fine-tuned based on the inter-domain correlation of the feature selection space to obtain the spraying trajectory-product surface state correlation interaction feature vector, including: processing the multi-scale product surface spraying state feature vector and the robot spraying trajectory feature vector according to the following formula to obtain the spraying trajectory-product surface state correlation interaction feature vector; wherein the formula is:

[0038]

[0039] v c =αv kt +βv st

[0040] Among them, V1 represents the characteristic vector of the spraying state of the multi-scale product surface, V2 represents the characteristic vector of the robot spraying trajectory, PCA(·) represents the principal component analysis, U1 represents the sequence of the intrinsic decomposition vector of the spraying state of the multi-scale product surface, Λ1 represents the diagonal matrix of the spraying state of the multi-scale product surface, λ 11 and λ 1m They represent the eigenvalues ​​of the first and mth positions of the diagonal matrix of the multi-scale product surface spraying state, T represents the transpose of the vector, concat{·;·} represents the vector cascade, and v 11 、v 12 、v 1m They represent the first, second and mth eigenvectors of the sequence of eigendecomposition vectors of the multi-scale product surface spraying state, V k represents the intrinsic eigenvector of the multi-scale product surface spraying state, U2 represents the sequence of the robot spraying trajectory intrinsic decomposition vector, Λ2 represents the diagonal matrix of the multi-scale product surface spraying state, λ 21 and λ 2m They represent the eigenvalues ​​of the first and mth positions of the robot spraying trajectory diagonal matrix, v 21 、v 22 、v 2m They represent the first, second and mth eigenvectors of the sequence of eigendecomposition vectors of the robot spraying trajectory, V s represents the intrinsic feature vector of the robot spraying trajectory, v i represents the i-th eigenvalue of the intrinsic eigenvector of the multi-scale product surface spraying state, v j represents the jth eigenvalue of the robot spraying trajectory eigenvector, represents the square of the vector norm, exp represents the natural exponential function, V' k Represents the intrinsic eigenvector after the multi-scale product surface spray state is modulated, V' srepresents the intrinsic eigenvector of the robot spraying trajectory after modulation, represents matrix multiplication, S represents the scale of the vector, M x Represents the fine-grained interaction matrix between spraying trajectory and product surface state, Conv 3×3 (M x ) indicates that M x A convolutional encoding operation based on a convolution kernel of 3×3 is performed, Ω represents the fine-grained semantic information field of the spraying trajectory-product surface state, and v kt represents the multi-scale product surface spraying state feature vector after information field mapping, v st represents the feature vector of the robot spraying trajectory after information field mapping, α and β represent weighted hyperparameters, and v c Represents the spraying trajectory-product surface state correlation interaction feature vector.

[0041] That is, in the technical solution of the present application, the multi-scale product surface spraying state feature vector and the robot spraying trajectory feature vector are fine-tuned based on the inter-domain correlation of the feature selection space. The process first maps the multi-scale product surface spraying state feature vector and the robot spraying trajectory feature vector to the intrinsic decomposition space to obtain the multi-scale product surface spraying state intrinsic feature vector and the robot spraying trajectory intrinsic feature vector. Those of ordinary skill in the art should know that the process of mapping to the intrinsic decomposition space is a combination of dimensionality reduction and feature extraction. In a specific example, the eigenvalue decomposition technique of principal component analysis (PCA) is used to find the intrinsic decomposition space, and the intrinsic decomposition space mapping is performed. The multi-scale product surface spraying state intrinsic feature vector and the robot spraying trajectory intrinsic feature vector after information field mapping not only represent the projection of the original feature vector in the new coordinate system, but also reflect the main direction of change of the data, which helps to avoid the "dimensionality disaster", that is, the problem that the sample density drops sharply as the number of features increases.

[0042] Next, after mapping the feature vector to the intrinsic decomposition space, it is necessary to perform target domain driven feature amplitude tuning on the multi-scale product surface spray state intrinsic feature vector and the robot spray trajectory intrinsic feature vector to adjust the source domain feature vector to better match the data distribution of the target domain. This operation aims to capture and compensate for the systematic deviation between the two domains to reduce the hidden disturbance of the class expression of the local feature distribution of the multi-scale product surface spray state intrinsic feature vector and the robot spray trajectory intrinsic feature vector to the overall class expression. After modulation, the modulated intrinsic feature vector of the multi-scale product surface spray state and the modulated intrinsic feature vector of the robot spray trajectory can retain the source domain characteristics while being more in line with the actual needs of the target domain, enhancing the model's understanding and application of the target domain data.

[0043] After completing the scale modulation of the feature vector, a spray trajectory-product surface state fine-grained semantic information field is constructed between the multi-scale product surface spray state modulated intrinsic feature vector and the robot spray trajectory modulated intrinsic feature vector. The spray trajectory-product surface state fine-grained semantic information field is used to describe a more detailed relationship between feature vectors. The introduction of the spray trajectory-product surface state fine-grained semantic information field enables the model to go beyond simple point-to-point comparison and consider more complex patterns and contextual information, thereby enhancing the model's sensitivity to subtle differences and improving the quality of feature characterization. Finally, based on the spray trajectory-product surface state fine-grained semantic information field, the multi-scale product surface spray state modulated intrinsic feature vector and the robot spray trajectory modulated intrinsic feature vector are fused to obtain the spray trajectory-product surface state associated interactive feature vector.

[0044] In the above-mentioned intelligent robot spraying path automatic generation system 100, the robot spraying path control module 150 is used to determine the control instructions of the robot based on the spraying trajectory-product surface state associated interactive feature vector. Specifically, in an embodiment of the present application, the robot spraying path control module is used to: pass the spraying trajectory-product surface state associated interactive feature vector through a decoder-based spraying path planner to obtain the control instruction, and the control instruction represents the acceleration and angular velocity of the robot's movement at the current time point. It should be understood that the spraying trajectory-product surface state associated interactive feature vector combines the spraying trajectory characteristics and the product surface state characteristics, and can accurately describe the robot spraying process. In order to further dynamically adjust the motion trajectory of the nozzle according to the spraying trajectory information and product surface state information contained in the spraying trajectory-product surface state associated interactive feature vector, in the technical solution of the present application, a decoder-based spraying path planner is used to decode and regress the spraying trajectory-product surface state associated interactive feature vector. That is, by utilizing the powerful nonlinear mapping capability of the decoder model, the spray trajectory-product surface state associated interactive feature vector is mapped to the control instruction numerical space to guide the robot's running acceleration and angular velocity, and automatically adjust the spray trajectory, thereby achieving uniform and efficient spraying of the product surface.

[0045] In summary, the automatic generation system of the spray path of the intelligent robot according to the embodiment of the present application is explained, which uses artificial intelligence technology based on deep learning to analyze and process the acceleration and angular velocity data of the robot nozzle movement collected by the accelerometer and gyroscope, dig out the motion trajectory information of the robot during the spraying process, and combine the surface state information of the sprayed product to adaptively adjust the acceleration and angular velocity of the nozzle. In this way, the spray trajectory of the robot can be optimized, and the automatic generation of the spray path can be realized, thereby improving the efficiency and quality of the spraying operation.

[0046] Figure 4 Flow chart of the method for automatically generating a spraying path for an intelligent robot according to an embodiment of the present application. Figure 4 As shown, according to the embodiment of the present application, the method for automatically generating a spraying path of an intelligent robot includes the following steps: S110, a surface state image of a sprayed product at a current time point collected by an industrial camera deployed on the intelligent robot, and a time series of acceleration values ​​and a time series of angular velocity values ​​within a predetermined time period including the current time point collected by an accelerometer and a gyroscope deployed on the nozzle of the intelligent robot; S120, multi-scale spraying state feature extraction of the surface state image of the sprayed product to obtain a multi-scale product surface spraying state feature vector; S130, extracting spraying trajectory features from the time series of acceleration values ​​and the time series of angular velocity values ​​to obtain a robot spraying trajectory feature vector; S140, fine-tuning the multi-scale product surface spraying state feature vector and the robot spraying trajectory feature vector based on the inter-domain correlation of the feature selection space to obtain a spraying trajectory-product surface state correlation interaction feature vector; S150, determining the control instructions of the robot based on the spraying trajectory-product surface state correlation interaction feature vector.

[0047] Here, those skilled in the art can understand that the specific operations of each step in the above-mentioned intelligent robot spray path automatic generation method have been referred to above. Figures 1 to 3 The invention has been introduced in detail in the description of the intelligent robot spray path automatic generation system, and therefore, its repeated description will be omitted.

[0048] The basic principle of the present invention is described above in conjunction with specific embodiments. However, it should be pointed out that the advantages, strengths, effects, etc. mentioned in the present invention are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. must be possessed by each embodiment of the present invention. In addition, the specific details of the above embodiments are only for the purpose of illustration and facilitation of understanding, rather than limitation, and the above details do not limit the present invention to being implemented by adopting the above specific details.

[0049] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, please refer to the relevant description of other embodiments. In the several embodiments provided by the present invention, it should be understood that the disclosed system and method can be implemented in other ways. For example, the system embodiment described above is only schematic. For example, the module division is only a logical function division, and there may be other division methods in actual implementation. The modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment.

[0050] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0051] In addition, each functional module in each embodiment of the present invention may be integrated into one processing unit, each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional modules.

[0052] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference to a figure in a claim should not be considered as limiting the claim to which it relates.

[0053] In addition, it is obvious that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. The multiple units stated in the system claims can also be implemented by one unit through software or hardware.

[0054] Finally, it should be noted that the above description has been given for the purpose of illustration and description. In addition, the above embodiments are only used to illustrate the technical solution of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present invention.

Claims

1. An intelligent robot spraying path automatic generation system, characterized in that: include: A robot spraying monitoring module, which is used for the surface state image of the sprayed product at the current time point collected by the industrial camera deployed on the intelligent robot, and the time series of acceleration values ​​and angular velocity values ​​within a predetermined time period including the current time point collected by the accelerometer and gyroscope deployed on the spray head of the intelligent robot; A multi-scale spray state feature extraction module is used to extract multi-scale spray state features from the spray product surface state image to obtain a multi-scale product surface spray state feature vector; A spraying trajectory feature extraction module, used to extract spraying trajectory features from the time series of the acceleration value and the time series of the angular velocity value to obtain a robot spraying trajectory feature vector; An association encoding module, used for fine-tuning the inter-domain association of the multi-scale product surface spraying state feature vector and the robot spraying trajectory feature vector based on the feature selection space to obtain a spraying trajectory-product surface state association interaction feature vector; The robot spraying path control module is used to determine the robot control instructions based on the spraying trajectory-product surface state associated interactive feature vector.

2. The intelligent robot spraying path automatic generation system according to claim 1 is characterized in that: The multi-scale spray state feature extraction module is used to: The spray product surface state image is passed through a spray state multi-scale feature extractor based on a pyramid network to obtain the multi-scale product surface spray state feature vector.

3. The intelligent robot spraying path automatic generation system according to claim 2 is characterized in that: The spraying trajectory feature extraction module comprises: A data regularization unit, used for arranging the time series of the acceleration values ​​and the time series of the angular velocity values ​​into a nozzle movement parameter timing input matrix according to the time dimension and the parameter sample dimension; The robot spraying trajectory pattern recognition unit is used to perform spraying trajectory pattern recognition on the nozzle movement parameter timing input matrix to obtain the robot spraying trajectory feature vector.

4. The intelligent robot spraying path automatic generation system according to claim 3 is characterized in that: The robot spraying trajectory pattern recognition unit is used to: The spray head movement parameter time sequence input matrix is ​​passed through a spray trajectory feature extractor based on a convolutional neural network model to obtain the robot spray trajectory feature vector.

5. The intelligent robot spraying path automatic generation system according to claim 4 is characterized in that: The associated coding module comprises: A feature vector eigendecomposition unit, used for mapping the multi-scale product surface spraying state feature vector and the robot spraying trajectory feature vector to an eigendecomposition space to obtain a multi-scale product surface spraying state eigenfeature vector and a robot spraying trajectory eigenfeature vector; An intrinsic characteristic amplitude tuning unit is used to perform target domain driven characteristic amplitude tuning on the intrinsic characteristic vector of the multi-scale product surface spraying state and the intrinsic characteristic vector of the robot spraying trajectory to obtain the modulated intrinsic characteristic vector of the multi-scale product surface spraying state and the modulated intrinsic characteristic vector of the robot spraying trajectory; A fine-grained semantic information field construction unit, used to construct a spraying trajectory-product surface state fine-grained semantic information field between the multi-scale product surface spraying state modulated intrinsic feature vector and the robot spraying trajectory modulated intrinsic feature vector; A spraying trajectory-product surface state associated interactive feature generation unit is used to fuse the multi-scale product surface spray state modulated intrinsic feature vector and the robot spraying trajectory modulated intrinsic feature vector based on the spraying trajectory-product surface state fine-grained semantic information field to obtain the spraying trajectory-product surface state associated interactive feature vector.

6. The intelligent robot spraying path automatic generation system according to claim 5 is characterized in that: The fine-grained semantic information field construction unit is used to: The multi-scale product surface spray state modulated intrinsic feature vector and the robot spray trajectory modulated intrinsic feature vector are multiplied, and then divided by the square root of the scale of the robot spray trajectory modulated intrinsic feature vector to obtain a spray trajectory-product surface state fine-grained interaction matrix; The spraying trajectory-product surface state fine-grained interaction matrix is ​​convolutionally encoded to obtain the spraying trajectory-product surface state fine-grained semantic information field.

7. The intelligent robot spraying path automatic generation system according to claim 6 is characterized in that: The spraying trajectory-product surface state correlation interaction feature generation unit is used to: Multiplying the spraying trajectory-product surface state fine-grained semantic information field by the multi-scale product surface spraying state modulated intrinsic feature vector to obtain a multi-scale product surface spraying state feature vector after information field mapping; Multiplying the spraying trajectory-product surface state fine-grained semantic information field by the modulated intrinsic feature vector of the robot spraying trajectory to obtain a robot spraying trajectory feature vector after information field mapping; The multi-scale product surface spraying state feature vector after the information field mapping and the robot spraying trajectory feature vector after the information field mapping are weighted fused to obtain the spraying trajectory-product surface state associated interactive feature vector.

8. The intelligent robot spraying path automatic generation system according to claim 7, characterized in that: The robot spraying path control module is used to: pass the spraying trajectory-product surface state associated interactive feature vector through a decoder-based spraying path planner to obtain the control instruction, and the control instruction represents the acceleration and angular velocity of the robot's movement at the current time point.

9. A method for automatically generating a spraying path for an intelligent robot, characterized in that: include: The surface state image of the sprayed product at the current time point collected by the industrial camera deployed on the intelligent robot, and the time series of acceleration values ​​and angular velocity values ​​within a predetermined time period including the current time point collected by the accelerometer and gyroscope deployed on the spray head of the intelligent robot; Performing multi-scale spray state feature extraction on the spray product surface state image to obtain a multi-scale product surface spray state feature vector; Extracting spraying trajectory features from the time series of the acceleration values ​​and the time series of the angular velocity values ​​to obtain a robot spraying trajectory feature vector; Performing inter-domain correlation fine-tuning on the multi-scale product surface spraying state feature vector and the robot spraying trajectory feature vector based on feature selection space to obtain a spraying trajectory-product surface state correlation interaction feature vector; Based on the spraying trajectory-product surface state associated interactive feature vector, the control instructions of the robot are determined.

10. The method for automatically generating a spraying path for an intelligent robot according to claim 9, characterized in that: Performing multi-scale spray state feature extraction on the spray product surface state image to obtain a multi-scale product surface spray state feature vector, including: The spray product surface state image is passed through a spray state multi-scale feature extractor based on a pyramid network to obtain the multi-scale product surface spray state feature vector.

Images