A robotic arm control method, system, device, and medium

Abstract

The application discloses a kind of mechanical arm control method, system, equipment and medium, it is related to mechanical arm control technical field, response control request, determine the mechanical arm of control request and obtain corresponding acquisition image through mechanical arm, feature extraction is carried out to acquisition image, generate initial SpaFD feature data, initial SpaFD feature data is fitted using covariance matrix adaptive evolution strategy algorithm, generate target SpaFD feature data, target SpaFD feature data is searched in predetermined standard SpaFD feature model library, determine operation information, according to operation information control mechanical arm operation;Solved the existing mechanical arm identification control needs to build complex network structure, there is limitation under the scene needing quick response, cannot be adapted to different scenes, there is the technical problem that precision identification and control cannot be quickly carried out to the object of grabbing.

Description

Technical Field

[0001] This invention relates to the field of robotic arm control technology, and in particular to a robotic arm control method, system, device and medium. Background Technology

[0002] In modern industrial production, robotic arms play a vital role, performing various complex tasks such as assembly, handling, and precision operations. To improve the autonomous recognition capabilities of robotic arms and enable them to adapt to different scenarios and workpieces, image recognition technology for objects grasped by robotic arms has become particularly important.

[0003] Currently, most existing robotic arm recognition and control systems employ deep learning methods to build network models. However, building such models requires designing complex network structures and extensive hyperparameter tuning to achieve good performance. Traditional machine learning methods also require feature selection and optimization based on specific data and tasks. Therefore, existing robotic arm recognition and control systems, which require complex network structures, have limitations in scenarios requiring rapid response, cannot adapt to different scenarios, and suffer from technical problems in quickly and accurately recognizing and controlling grasped objects. Summary of the Invention

[0004] This invention provides a robotic arm control method, system, device, and medium, which solves the technical problems of existing robotic arm recognition and control systems that require the construction of complex network structures, have limitations in scenarios requiring rapid response, cannot adapt to different scenarios, and cannot quickly and accurately identify and control the grasped object.

[0005] The first aspect of this invention provides a robotic arm control method, comprising:

[0006] In response to a control request, the robotic arm corresponding to the control request is identified, and the corresponding image is acquired through the robotic arm.

[0007] Feature extraction is performed on the acquired images to generate initial SpaFD feature data;

[0008] The initial SpaFD feature data is fitted using a covariance matrix adaptive evolution strategy algorithm to generate target SpaFD feature data;

[0009] The target SpaFD feature data is used to retrieve a preset standard SpaFD feature model library to determine the operation information;

[0010] The robotic arm is controlled to operate based on the operation information.

[0011] Optionally, the construction process of the standard SpaFD feature model library includes:

[0012] Collect sample images of various types of objects to be grasped in different scenarios;

[0013] The feature extraction is performed on each of the sampled images to generate standard SpaFD feature data associated with each of the sampled images;

[0014] Sampling images of each of the grasped objects from different angles are acquired, and the fixed-point coordinate data of each of the grasped objects are obtained using the structured light method;

[0015] The database of preset operation action key-value pairs is retrieved using each of the grasped objects, and the operation action data associated with each of the grasped objects is matched.

[0016] A standard SpaFD feature model library is constructed using all the aforementioned standard SpaFD feature data, the fixed-point coordinate data, and the operation action data.

[0017] Optionally, the feature extraction process includes:

[0018] Obtain feature data for each band of the image;

[0019] The image is subjected to a v-order two-dimensional fractional differential mask to generate initial v-order feature data corresponding to each band;

[0020] The expression for the initial v-order feature data is:

[0021]

[0022] In the formula, This represents the initial v-order feature data of the Z-th band, where Z = 1, 2, ..., m, S represents the mask size, and M represents the initial v-order feature data of the Z-th band. (v) (i, j) represents the center coordinates of the mask, i represents the x-coordinate of the image pixel value, j represents the y-coordinate of the image pixel value, v represents the order, and Z represents the band.

[0023] Using the extracted feature data and the v-order feature data associated with each band, the target v-order feature data corresponding to each band is determined;

[0024] The expression for the target v-order feature data is:

[0025]

[0026] In the formula, f represents the target's v-order feature data in the Z-th band. Z (x0, y0) represents the extracted feature data of the Z-th band;

[0027] All the target v-order feature data are superimposed in the original band order to generate SpaFD feature data;

[0028] The expression for the SpaFD feature data is:

[0029]

[0030] In the formula, F represents the SpaFD feature data.

[0031] Optionally, the step of fitting the initial SpaFD feature data with the covariance matrix adaptive evolution strategy algorithm to generate the target SpaFD feature data includes:

[0032] The initial SpaFD feature data is continuously upsampled until the upsampling result is within the range of the preset standard deviation vector, at which point the iteration stops.

[0033] In this process, the parameters used in each iteration are updated;

[0034] The upsampling results within the preset standard deviation vector range are used as the optimal SpaFD feature data to generate target SpaFD feature data.

[0035] Optionally, the parameters include the mean vector, covariance matrix, directional correlation matrix, and standard deviation vector.

[0036] Optionally, the step of retrieving a preset standard SpaFD feature model library using the target SpaFD feature data to determine the operation information includes:

[0037] The target key is generated using the target SpaFD feature data;

[0038] Input the target key into a preset standard SpaFD feature model library and match it with standard SpaFD feature data that is consistent with the target SpaFD feature data.

[0039] Extract the operational information associated with the standard SpaFD feature data;

[0040] The operation information includes fixed-point coordinate data and operation action data.

[0041] Optionally, the step of controlling the operation of the robotic arm based on the operation information includes:

[0042] Control the robotic arm to grasp the fixed point coordinates associated with the fixed point coordinate data;

[0043] The robotic arm is used to perform the operation actions associated with the operation action data.

[0044] A second aspect of the present invention provides a robotic arm control system, comprising:

[0045] The response module is used to respond to control requests, determine the robotic arm corresponding to the control request, and acquire the corresponding image through the robotic arm.

[0046] The feature extraction module is used to extract features from the acquired image and generate initial SpaFD feature data;

[0047] The fitting module is used to fit the initial SpaFD feature data using a covariance matrix adaptive evolution strategy algorithm to generate target SpaFD feature data.

[0048] The retrieval module is used to retrieve a preset standard SpaFD feature model library using the target SpaFD feature data to determine operation information;

[0049] The operation module is used to control the operation of the robotic arm based on the operation information.

[0050] A third aspect of the present invention provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the steps of the robotic arm control method as described in any of the preceding claims.

[0051] The fourth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed, implements the robotic arm control method as described in any of the preceding claims.

[0052] As can be seen from the above technical solutions, the present invention has the following advantages:

[0053] In response to a control request, the system identifies the corresponding robotic arm and acquires the corresponding image. Feature extraction is performed on the acquired image to generate initial SpaFD feature data. A covariance matrix adaptive evolution strategy algorithm is used to fit the initial SpaFD feature data to generate target SpaFD feature data. The target SpaFD feature data is then used to retrieve a pre-defined standard SpaFD feature model library to determine the operation information. Based on this operation information, the robotic arm is controlled to operate. This invention solves the technical problems of existing robotic arm recognition and control systems, which require complex network structures, have limitations in scenarios requiring rapid response, cannot adapt to different scenarios, and cannot quickly and accurately identify and control grasped objects. This invention uses image extraction with a certain degree of invariance to image scale changes. The extracted SpaFD features have excellent descriptive ability for image texture features, reflecting the image's texture structure. In robotic arm applications, it is often necessary to locate and track grasped objects. By using SpaFD features, the robotic arm can extract texture features from images and compare them with predefined texture features to determine the position and posture information of the grasped object. This helps the robotic arm accurately locate and track targets, thereby achieving precise operation and task completion. Meanwhile, this invention employs an adaptive evolutionary strategy algorithm based on the covariance matrix, which features rapid convergence and efficient optimization during the optimization process. By combining the adaptive evolutionary strategy algorithm based on the covariance matrix with SpaFD features, the optimal feature representation and parameter settings can be found more quickly, improving the training and inference efficiency of image recognition algorithms. Attached Figure Description

[0054] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0055] Figure 1 This is a flowchart of the steps of a robotic arm control method provided in Embodiment 1 of the present invention;

[0056] Figure 2 This is a flowchart of the steps of a robotic arm control method provided in Embodiment 2 of the present invention;

[0057] Figure 3 This is a structural block diagram of a robotic arm control system provided in Embodiment 3 of the present invention. Detailed Implementation

[0058] This invention provides a robotic arm control method, system, device, and medium to address the technical problems of existing robotic arm recognition and control systems, which require the construction of complex network structures, have limitations in scenarios requiring rapid response, cannot adapt to different scenarios, and cannot quickly and accurately identify and control grasped objects.

[0059] To make the objectives, features, and advantages of this invention more apparent and understandable, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described below are only some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0060] Please see Figure 1 , Figure 1 The flowchart illustrates the steps of a robotic arm control method provided in Embodiment 1 of the present invention.

[0061] The present invention provides a robotic arm control method, comprising:

[0062] Step 101: Respond to the control request, determine the robotic arm corresponding to the control request, and acquire the corresponding image through the robotic arm.

[0063] A control request refers to a request message for controlling the robotic arm;

[0064] The main components of a robotic arm are a base, joints, kinematic chains, means, and hyperspectral cameras.

[0065] Image acquisition refers to the acquisition of hyperspectral images using a hyperspectral camera mounted on a robotic arm.

[0066] In this embodiment of the invention, in response to a received control request, the robotic arm that needs to be controlled is determined, and an image is acquired by a camera mounted on the robotic arm.

[0067] Step 102: Extract features from the acquired images to generate initial SpaFD feature data.

[0068] Feature extraction refers to the extraction of pixel spatial fractional derivative (SpaFD) features from hyperspectral images based on a two-dimensional fractional derivative mask.

[0069] In this embodiment of the invention, feature extraction is performed on the acquired image to generate initial pixel spatial fractional derivative (SpaFD) features.

[0070] Step 103: Use the covariance matrix adaptive evolution strategy algorithm to fit the initial SpaFD feature data to generate the target SpaFD feature data.

[0071] The Covariance Matrix Adaptation Evolution Strategy (CMA-ES) is an efficient optimization algorithm.

[0072] The CMA-ES algorithm is an evolutionary strategy (ES) that optimizes the objective function by simulating the process of natural evolution. The CMA-ES algorithm is adaptive, automatically adjusting its search strategy to suit the characteristics of the problem.

[0073] The core of the CMA-ES algorithm is the adaptive updating of the covariance matrix. The covariance matrix is ​​a crucial parameter that describes the distribution of solutions in the search space. By continuously updating the covariance matrix, the CMA-ES algorithm makes the search process more efficient.

[0074] Therefore, this invention employs an adaptive evolutionary strategy algorithm based on the covariance matrix, which features fast convergence and efficient optimization during the optimization process. By combining the adaptive evolutionary strategy algorithm based on the covariance matrix with SpaFD features, the optimal feature representation and parameter settings can be found more quickly, improving the training and inference efficiency of the image recognition algorithm.

[0075] In this embodiment of the invention, the covariance matrix adaptive evolution strategy algorithm is used to fit the initial SpaFD feature data, output the optimal solution, and the initial SpaFD feature data associated with the optimal fitting result is used as the target SpaFD feature data.

[0076] Step 104: Use the target SpaFD feature data to retrieve the preset standard SpaFD feature model library and determine the operation information.

[0077] In this embodiment of the invention, the target SpaFD feature data is used to retrieve a preset standard SpaFD feature model library, match the standard SpaFD feature data that is consistent with the target SpaFD feature data, and extract the operation information associated with the standard SpaFD feature data.

[0078] Step 105: Control the operation of the robotic arm according to the operation information.

[0079] In this embodiment of the invention, the operation information includes fixed-point coordinate data and operation action data, controlling the robotic arm to grasp the fixed-point coordinates associated with the fixed-point coordinate data; and using the robotic arm to execute the operation actions associated with the operation action data.

[0080] In this invention, in response to a control request, the corresponding robotic arm is identified, and the robotic arm acquires the corresponding image. Feature extraction is performed on the acquired image to generate initial SpaFD feature data. A covariance matrix adaptive evolution strategy algorithm is used to fit the initial SpaFD feature data to generate target SpaFD feature data. The target SpaFD feature data is then used to retrieve a pre-defined standard SpaFD feature model library to determine operation information. Based on this operation information, the robotic arm is controlled to operate. This invention solves the technical problems of existing robotic arm recognition and control systems, which require complex network structures, have limitations in scenarios requiring rapid response, cannot adapt to different scenarios, and cannot quickly and accurately identify and control grasped objects. This invention uses image extraction with features that exhibit a certain degree of invariance to image scale changes. The extracted SpaFD features have excellent descriptive ability for image texture features, reflecting the image's texture structure. In robotic arm applications, it is often necessary to locate and track grasped objects. By using SpaFD features, the robotic arm can extract texture features from images and compare them with predefined texture features to determine the position and posture information of the grasped object. This helps the robotic arm accurately locate and track targets, thereby achieving precise operation and task completion. Meanwhile, this invention employs an adaptive evolutionary strategy algorithm based on the covariance matrix, which features rapid convergence and efficient optimization during the optimization process. By combining the adaptive evolutionary strategy algorithm based on the covariance matrix with SpaFD features, the optimal feature representation and parameter settings can be found more quickly, improving the training and inference efficiency of image recognition algorithms.

[0081] Please see Figure 2 , Figure 2 This is a flowchart of a robotic arm control method provided in Embodiment 2 of the present invention.

[0082] The present invention provides a robotic arm control method, comprising:

[0083] Step 201: Respond to the control request, determine the robotic arm corresponding to the control request, and acquire the corresponding image through the robotic arm.

[0084] In this embodiment of the invention, the specific implementation process of step 201 is similar to that of step 101, and will not be repeated here.

[0085] Step 202: Extract features from the acquired images to generate initial SpaFD feature data.

[0086] In this embodiment of the invention, the specific implementation process of step 202 is similar to that of step 102, and will not be repeated here.

[0087] Furthermore, the feature extraction process includes:

[0088] Obtain feature data for each band of the image;

[0089] In this embodiment of the invention, the extracted feature data is f. Z (x0,y0), f Z (x0,y0) represents the value of the two-dimensional image f(x,y) in the Z-band of the hyperspectral image (sampled image and acquired image) at (x0,y0).

[0090] The image is subjected to a v-order two-dimensional fractional differential mask to generate initial v-order feature data for each band.

[0091] The expression for the initial v-order feature data is:

[0092]

[0093] In the formula, This represents the initial v-order feature data of the Z-th band, where Z = 1, 2, ..., m, S represents the mask size, and M represents the initial v-order feature data of the Z-th band. (v) (i,j) represents the center coordinates of the mask, i represents the x-coordinate of the image pixel value, j represents the y-coordinate of the image pixel value, v represents the order, and Z represents the band.

[0094] By using the extracted feature data and v-order feature data associated with each band, the target v-order feature data corresponding to each band is determined;

[0095] The expression for the target v-order feature data is:

[0096]

[0097] In the formula, f represents the target v-order feature data of the Z-th band. Z (x0,y0) represents the extracted feature data of the Z-th band;

[0098] All target v-order feature data are superimposed in the original band order to generate SpaFD feature data;

[0099] The expression for SpaFD feature data is:

[0100]

[0101] In the formula, F represents SpaFD feature data.

[0102] It is worth mentioning that all target v-order feature data are superimposed in the original band order to enhance texture details and obtain SpaFD feature data.

[0103] It should be noted that in the feature extraction process, "image" refers to either the sampled image or the acquired image. The extraction process for both is the same and will not be elaborated here.

[0104] Step 203: Use the covariance matrix adaptive evolution strategy algorithm to fit the initial SpaFD feature data to generate the target SpaFD feature data.

[0105] Furthermore, step 203 may include the following sub-steps:

[0106] S11. Continuously upsample the initial SpaFD feature data until the upsampling result is within the range of the preset standard deviation vector, then stop the iteration;

[0107] In this process, the parameters used in each iteration are updated;

[0108] In this embodiment of the invention, a sampling unit is used.

[0109] In this embodiment of the invention, the initial SpaFD feature data is continuously upsampled using a sampling unit.

[0110] S12. The upsampling results within the preset standard deviation vector range are used as the optimal SpaFD feature data to generate the target SpaFD feature data.

[0111] Furthermore, the parameters include the mean vector, covariance matrix, directional correlation matrix, and standard deviation vector.

[0112] For ease of understanding, the following is the fitting process of the covariance matrix adaptive evolution strategy algorithm:

[0113] 1) Initialize the covariance matrix, mean vector m, covariance matrix C, directional correlation matrix B, and standard deviation vector σ of the adaptive evolution strategy algorithm, and preset that the iteration stops when the standard deviation vector σ reaches a certain range.

[0114] 2) Convert the acquired image to a grayscale image, divide the grayscale image into several small blocks, each usually called a pixel block. For each pixel block, use the grayscale value of each pixel in the pixel block as a component of the sample vector. This generates N sample vectors x. i Let i = 1, 2, ..., N, where these vectors correspond to different values ​​of the image parameters. Sample vectors are generated using a multivariate Gaussian distribution N(m, C) based on the current mean vector m and covariance matrix C.

[0115] 3) Convert each sample vector x i When applied to an image, the corresponding function value f(x) is calculated. i );

[0116] f(xi )=ΣLB·x i

[0117] Where LB represents the filter coefficients, which are user-defined.

[0118] 4) Based on the function value f(x) i Sort the sample vectors in descending order and select the top m. u m sample vectors are used as parent solutions. u The number of individuals to select is set according to the actual situation.

[0119] 5) Calculate the evolutionary path vector p;

[0120]

[0121] Among them, w i represents the sample weights, and w represents the sum of the weights.

[0122] 6) Update the mean vector m 更 ;

[0123] m 更 =(m+α) p )×p

[0124] Where, α p is the evolutionary path step size coefficient, and m is the current mean vector.

[0125] 7) Calculate the sample variation vector y i ;

[0126] y i =B·D(x i )

[0127] Where B is the direction correlation matrix, and D is a diagonal matrix with diagonal elements σ. i , that is, the elements of the standard deviation vector.

[0128] 8) Update the covariance matrix C;

[0129]

[0130] Where, α C It is the learning rate of the covariance matrix.

[0131] 9) Calculate the standard deviation adjustment vector C;

[0132]

[0133] Where, μ eff It is the number of individuals selected effectively, n eff It is the effective dimension of the optimization variables.

[0134] 10) Calculate and update the standard deviation update vector Δσ;

[0135]

[0136] σ=σ·exp(Δσ)

[0137] If the termination condition is met, stop the iteration and output the current optimal solution x*, which is the optimal fitting result.

[0138] The SpaFD feature data associated with the best fit result is used as the target SpaFD feature data and output.

[0139] Step 204: Use the target SpaFD feature data to retrieve the preset standard SpaFD feature model library and determine the operation information.

[0140] Furthermore, the construction process of the standard SpaFD feature model library includes:

[0141] Collect sample images of various types of objects to be grasped in different scenarios;

[0142] In this embodiment of the invention, since the working scene of the robotic arm is not fixed and there are many types of objects to be grasped, it is necessary to acquire sampled images of various types of objects in different scenes and then construct a feature model library.

[0143] Feature extraction is performed on each sampled image to generate standard SpaFD feature data associated with each sampled image;

[0144] In this embodiment of the invention, the feature extraction process here is the same as the feature extraction process in step 202 above, and will not be repeated here.

[0145] Collect sampling images of each grasped object from different angles, and use structured light method to obtain the fixed-point coordinate data of each grasped object;

[0146] In this embodiment of the invention, by acquiring images of the object to be grasped from multiple angles, and using a structured light method, a camera or other sensor records the light changes on the object's surface after the projection light pattern is applied, thereby obtaining the three-dimensional point data of the object to be grasped. This allows for the determination of the fixed-point coordinate data of the object, which is the coordinate point to be grasped by the robotic arm. This is a conventional coordinate transformation and will not be elaborated further.

[0147] The database of preset operation action key-value pairs is searched for each grasped object, and the operation action data associated with each grasped object is matched.

[0148] The preset operation action key-value pair database refers to the key-value pair database established by the association between the grasped object and the preset required operation action, where the grasped object is the key and the operation action is the value.

[0149] The required actions include, but are not limited to, reaching out and rotating.

[0150] In this embodiment of the invention, each grasped object is used to retrieve a preset operation action key-value pair database and match the operation action data associated with each grasped object.

[0151] A standard SpaFD feature model library was constructed using all standard SpaFD feature data, fixed-point coordinate data, and operation action data.

[0152] In this embodiment of the invention, the associated standard SpaFD feature data, fixed-point coordinate data and operation action data are classified according to different grasping objects. It can be understood that one grasping object corresponds to this set of grasping data, and the grasping data includes standard SpaFD feature data, fixed-point coordinate data and operation action data.

[0153] Furthermore, step 204 may include the following sub-steps:

[0154] S21. Generate the target key using the target SpaFD feature data;

[0155] S22. Input the target key into the preset standard SpaFD feature model library and match the standard SpaFD feature data that is consistent with the target SpaFD feature data.

[0156] S23. Extract operational information associated with standard SpaFD feature data;

[0157] Operational information includes fixed-point coordinate data and operational action data.

[0158] The standard SpaFD feature model library is a key-value pair database. It is a database that establishes the association between target SpaFD feature data and standard SpaFD feature data, fixed-point coordinate data and operation action data. In this database, target SpaFD feature data is used as the key, and standard SpaFD feature data, fixed-point coordinate data and operation action data are used as the value.

[0159] In this embodiment of the invention, the target SpaFD feature data is used as the target key and input into a preset standard SpaFD feature model library. Standard SpaFD feature data that is consistent with the target SpaFD feature data is retrieved, and the standard SpaFD feature data is associated with fixed-point coordinate data and operation action data and extracted as operation information.

[0160] Step 205: Control the robotic arm to grasp the fixed coordinates associated with the fixed coordinate data.

[0161] In this embodiment of the invention, the robotic arm is controlled to grasp the fixed coordinates associated with the fixed coordinate data.

[0162] Step 206: Use a robotic arm to perform the operation actions associated with the operation action data.

[0163] In this embodiment of the invention, a robotic arm is used to perform operation actions associated with operation action data.

[0164] In this invention, in response to a control request, the corresponding robotic arm is identified, and the robotic arm acquires the corresponding image. Feature extraction is performed on the acquired image to generate initial SpaFD feature data. A covariance matrix adaptive evolution strategy algorithm is used to fit the initial SpaFD feature data to generate target SpaFD feature data. The target SpaFD feature data is then used to retrieve a pre-defined standard SpaFD feature model library to determine operation information. Based on this operation information, the robotic arm is controlled to operate. This invention solves the technical problems of existing robotic arm recognition and control systems, which require complex network structures, have limitations in scenarios requiring rapid response, cannot adapt to different scenarios, and cannot quickly and accurately identify and control grasped objects. This invention uses image extraction with features that exhibit a certain degree of invariance to image scale changes. The extracted SpaFD features have excellent descriptive ability for image texture features, reflecting the image's texture structure. In robotic arm applications, it is often necessary to locate and track grasped objects. By using SpaFD features, the robotic arm can extract texture features from images and compare them with predefined texture features to determine the position and posture information of the grasped object. This helps the robotic arm accurately locate and track targets, thereby achieving precise operation and task completion. Meanwhile, this invention employs an adaptive evolutionary strategy algorithm based on the covariance matrix, which features rapid convergence and efficient optimization during the optimization process. By combining the adaptive evolutionary strategy algorithm based on the covariance matrix with SpaFD features, the optimal feature representation and parameter settings can be found more quickly, improving the training and inference efficiency of image recognition algorithms.

[0165] Please see Figure 3 , Figure 3 This is a structural block diagram of a robotic arm control system provided in Embodiment 3 of the present invention.

[0166] The present invention provides a robotic arm control system, comprising:

[0167] The response module 301 is used to respond to control requests, determine the robotic arm corresponding to the control request, and acquire the corresponding image through the robotic arm.

[0168] Feature extraction module 302 is used to extract features from the acquired image and generate initial SpaFD feature data;

[0169] The fitting module 303 is used to fit the initial SpaFD feature data using the covariance matrix adaptive evolution strategy algorithm to generate the target SpaFD feature data.

[0170] The retrieval module 304 is used to retrieve a preset standard SpaFD feature model library using the target SpaFD feature data to determine the operation information;

[0171] The operation module 305 is used to control the operation of the robotic arm based on the operation information.

[0172] Furthermore, the construction process of the standard SpaFD feature model library includes:

[0173] Collect sample images of various types of objects to be grasped in different scenarios;

[0174] Feature extraction is performed on each sampled image to generate standard SpaFD feature data associated with each sampled image;

[0175] Collect sampling images of each grasped object from different angles, and use structured light method to obtain the fixed-point coordinate data of each grasped object;

[0176] The database of preset operation action key-value pairs is searched for each grasped object, and the operation action data associated with each grasped object is matched.

[0177] A standard SpaFD feature model library was constructed using all standard SpaFD feature data, fixed-point coordinate data, and operation action data.

[0178] Furthermore, the feature extraction process includes:

[0179] Obtain feature data for each band of the image;

[0180] The image is subjected to a v-order two-dimensional fractional differential mask to generate initial v-order feature data for each band.

[0181] The expression for the initial v-order feature data is:

[0182]

[0183] In the formula, This represents the initial v-order feature data of the Z-th band, where Z = 1, 2, ..., m, S represents the mask size, and M represents the initial v-order feature data of the Z-th band. (v) (i,j) represents the center coordinates of the mask, i represents the x-coordinate of the image pixel value, j represents the y-coordinate of the image pixel value, v represents the order, and Z represents the band.

[0184] By using the extracted feature data and v-order feature data associated with each band, the target v-order feature data corresponding to each band is determined;

[0185] The expression for the target v-order feature data is:

[0186]

[0187] In the formula, f represents the target v-order feature data of the Z-th band. Z (x0,y0) represents the extracted feature data of the Z-th band;

[0188] All target v-order feature data are superimposed in the original band order to generate SpaFD feature data;

[0189] The expression for SpaFD feature data is:

[0190]

[0191] In the formula, F represents SpaFD feature data.

[0192] Furthermore, the fitting module 303 includes:

[0193] The iterative submodule is used to continuously upsample the initial SpaFD feature data until the upsampling result is within the range of the preset standard deviation vector, at which point the iteration stops.

[0194] In this process, the parameters used in each iteration are updated;

[0195] The target SpaFD feature data submodule is used to generate target SpaFD feature data by taking the upsampling results within the preset standard deviation vector range as the optimal SpaFD feature data.

[0196] Furthermore, the parameters include the mean vector, covariance matrix, directional correlation matrix, and standard deviation vector.

[0197] Furthermore, the retrieval module 304 includes:

[0198] The target key submodule is used to generate target keys using target SpaFD feature data;

[0199] The standard SpaFD feature data submodule is used to input the target key into a preset standard SpaFD feature model library and match the standard SpaFD feature data that is consistent with the target SpaFD feature data.

[0200] The operation information submodule is used to extract operation information associated with standard SpaFD feature data;

[0201] Operational information includes fixed-point coordinate data and operational action data.

[0202] Furthermore, the operating module 305 includes:

[0203] The fixed-point coordinate submodule is used to control the robotic arm to grasp objects at the fixed-point coordinates associated with the fixed-point coordinate data.

[0204] The operation action submodule is used to execute operation actions associated with operation action data using a robotic arm.

[0205] In this invention, in response to a control request, the corresponding robotic arm is identified, and the robotic arm acquires the corresponding image. Feature extraction is performed on the acquired image to generate initial SpaFD feature data. A covariance matrix adaptive evolution strategy algorithm is used to fit the initial SpaFD feature data to generate target SpaFD feature data. The target SpaFD feature data is then used to retrieve a pre-defined standard SpaFD feature model library to determine operation information. Based on this operation information, the robotic arm is controlled to operate. This invention solves the technical problems of existing robotic arm recognition and control systems, which require complex network structures, have limitations in scenarios requiring rapid response, cannot adapt to different scenarios, and cannot quickly and accurately identify and control grasped objects. This invention uses image extraction with features that exhibit a certain degree of invariance to image scale changes. The extracted SpaFD features have excellent descriptive ability for image texture features, reflecting the image's texture structure. In robotic arm applications, it is often necessary to locate and track grasped objects. By using SpaFD features, the robotic arm can extract texture features from images and compare them with predefined texture features to determine the position and posture information of the grasped object. This helps the robotic arm accurately locate and track targets, thereby achieving precise operation and task completion. Meanwhile, this invention employs an adaptive evolutionary strategy algorithm based on the covariance matrix, which features rapid convergence and efficient optimization during the optimization process. By combining the adaptive evolutionary strategy algorithm based on the covariance matrix with SpaFD features, the optimal feature representation and parameter settings can be found more quickly, improving the training and inference efficiency of image recognition algorithms.

[0206] An electronic device according to an embodiment of the present invention includes: a memory and a processor, wherein the memory stores a computer program; when the computer program is executed by the processor, the processor performs a robotic arm control method as described in any of the above embodiments.

[0207] The memory can be an electronic memory such as flash memory, EEPROM (Electrically Erasable Programmable Read-Only Memory), EPROM, hard disk, or ROM. The memory has storage space for program code used to perform any of the method steps described above. For example, the storage space for program code may include individual program codes for implementing the various steps in the methods described above. This program code can be read from or written to one or more computer program products. These computer program products include program code carriers such as hard disks, compact discs (CDs), memory cards, or floppy disks. The program code may be compressed, for example, in a suitable form. When run by a computing processing device, this code causes the computing processing device to perform the various steps in the methods described above.

[0208] This invention provides a computer-readable storage medium storing a computer program thereon, which, when executed, implements a robotic arm control method as described in any embodiment of this invention.

[0209] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0210] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between apparatuses or units through some interfaces, and may be electrical, mechanical, or other forms.

[0211] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0212] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0213] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0214] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A robotic arm control method, characterized in that, include: In response to a control request, the robotic arm corresponding to the control request is identified, and the corresponding image is acquired through the robotic arm. Feature extraction is performed on the acquired images to generate initial SpaFD feature data; SpaFD feature data refers to the pixel spatial fractional differential features extracted from hyperspectral images based on a two-dimensional fractional differential mask. The initial SpaFD feature data is fitted using a covariance matrix adaptive evolution strategy algorithm to generate target SpaFD feature data; Specifically, the covariance matrix adaptive evolution strategy algorithm is a covariance matrix adaptive evolution algorithm, which refers to an evolution strategy that optimizes the search by adaptively updating the covariance matrix. The target SpaFD feature data is used to retrieve a preset standard SpaFD feature model library to determine the operation information; The step of retrieving a preset standard SpaFD feature model library using the target SpaFD feature data to determine the operation information includes: The target key is generated using the target SpaFD feature data; Input the target key into a preset standard SpaFD feature model library and match it with standard SpaFD feature data that is consistent with the target SpaFD feature data. Extract the operational information associated with the standard SpaFD feature data; The operation information includes fixed-point coordinate data and operation action data; The standard SpaFD feature model library is a key-value pair database, which is established by associating the target SpaFD feature data with the standard SpaFD feature data, the fixed-point coordinate data and the operation action data. The target SpaFD feature data is used as the key, and the standard SpaFD feature data, the fixed-point coordinate data and the operation action data are used as the value. The robotic arm is controlled to operate based on the operation information.

2. The robotic arm control method according to claim 1, characterized in that, The construction process of the standard SpaFD feature model library includes: Collect sample images of various types of objects to be grasped in different scenarios; The feature extraction is performed on each of the sampled images to generate standard SpaFD feature data associated with each of the sampled images; Sampling images of each of the grasped objects from different angles are acquired, and the fixed-point coordinate data of each of the grasped objects are obtained using the structured light method; The database of preset operation action key-value pairs is retrieved using each of the grasped objects, and the operation action data associated with each of the grasped objects is matched. A standard SpaFD feature model library is constructed using all the aforementioned standard SpaFD feature data, the fixed-point coordinate data, and the operation action data.

3. The robotic arm control method according to claim 1 or 2, characterized in that, The feature extraction process includes: Obtain feature data for each band of the image; The image is subjected to a v-order two-dimensional fractional differential mask to generate initial v-order feature data corresponding to each band; The expression for the initial v-order feature data is: In the formula, Indicates the first The initial v-order feature data of the band, wherein , Indicates the mask size. Indicates the coordinates of the mask center. The x-coordinate represents the pixel value of the image. The vertical axis represents the pixel value of the image. Indicates the order, Indicates the band; Using the extracted feature data and the v-order feature data associated with each band, the target v-order feature data corresponding to each band is determined; The expression for the target v-order feature data is: In the formula, Indicates the first The target v-order feature data of the band, Indicates the first The extracted feature data of the band; All the target v-order feature data are superimposed in the original band order to generate SpaFD feature data; The expression for the SpaFD feature data is: In the formula, This represents the SpaFD feature data.

4. The robotic arm control method according to claim 1, characterized in that, The step of fitting the initial SpaFD feature data using the covariance matrix adaptive evolution strategy algorithm to generate the target SpaFD feature data includes: The initial SpaFD feature data is continuously upsampled until the upsampling result is within the range of the preset standard deviation vector, at which point the iteration stops. In this process, the parameters used in each iteration are updated; The upsampling results within the preset standard deviation vector range are used as the optimal SpaFD feature data to generate target SpaFD feature data.

5. The robotic arm control method according to claim 4, characterized in that, The parameters include the mean vector, covariance matrix, directional correlation matrix, and standard deviation vector.

6. The robotic arm control method according to claim 1, characterized in that, The step of controlling the operation of the robotic arm according to the operation information includes: Control the robotic arm to grasp the fixed point coordinates associated with the fixed point coordinate data; The robotic arm is used to perform the operation actions associated with the operation action data.

7. A robotic arm control system, characterized in that, The robotic arm control system is used to implement the robotic arm control method as described in any one of claims 1-6, and the robotic arm control system includes: The response module is used to respond to control requests, determine the robotic arm corresponding to the control request, and acquire the corresponding image through the robotic arm. The feature extraction module is used to extract features from the acquired image and generate initial SpaFD feature data; The fitting module is used to fit the initial SpaFD feature data using a covariance matrix adaptive evolution strategy algorithm to generate target SpaFD feature data. The retrieval module is used to retrieve a preset standard SpaFD feature model library using the target SpaFD feature data to determine operation information; The operation module is used to control the operation of the robotic arm based on the operation information.

8. An electronic device, characterized in that, The system includes a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor causes the processor to perform the steps of the robotic arm control method as described in any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed, it implements the robotic arm control method as described in any one of claims 1-6.

Images