An AI-based Visual Perception Method and System

Abstract

The present invention relates to the field of visual perception technology, and specifically discloses a visual perception method and system based on AI. The method includes obtaining the motion boundary of the robotic arm, determining the motion area according to the motion boundary, and synchronously constructing a motion instruction table; obtaining in real time the robotic arm image with time tags of the robotic arm, identifying the robotic arm image, and positioning the working point; sorting the working points according to the time tags to obtain the working trajectory; inserting the working trajectory into the motion area, displaying the motion area containing the working trajectory, and receiving the supplementary points input by the user; querying the instruction in the motion instruction table based on the supplementary points as a reference instruction. The present invention introduces an AI visual perception solution, simplifies the recognition process, determines the working area that the actuator of the robotic arm can reach, receives the user's target position within the working area, and then quickly queries the corresponding instruction group, greatly reducing the complexity of task adjustment of the robotic arm.

Description

Technical Field

[0001] The present invention relates to the technical field of visual perception, and specifically to an AI-based visual perception method and system. Background Art

[0002] During the working process of a robotic arm, it will at least move from one working station to another. During this process, the movement of the working end of the robotic arm is a trajectory, and there are many areas it can reach. Workstations can be added in these areas. However, for each added workstation, a multi-departmental joint programming is required, which is very cumbersome. How to simplify the difficulty of adding workstations is the technical problem that the technical solution of the present invention aims to solve. Summary of the Invention

[0003] The purpose of the present invention is to provide an AI-based visual perception method and system to solve the problems raised in the above background art.

[0004] To achieve the above purpose, the present invention provides the following technical solutions:

[0005] An AI-based visual perception method, the method includes:

[0006] Obtain the movement boundary of the robotic arm, determine the movement area according to the movement boundary, and synchronously construct a movement instruction table; the movement instruction table includes a coordinate item and an instruction item;

[0007] Obtain the robotic arm images with time tags of the robotic arm in three mutually perpendicular directions in real time, identify the robotic arm images, and locate the working points;

[0008] Sort the working points according to the time tags to obtain a working trajectory;

[0009] Insert the working trajectory into the movement area, display the movement area containing the working trajectory, and receive the supplementary points input by the user;

[0010] Query the instructions in the movement instruction table based on the supplementary points as reference instructions.

[0011] As a further solution of the present invention: the step of obtaining the movement boundary of the robotic arm, determining the movement area according to the movement boundary, and synchronously constructing a movement instruction table includes:

[0012] Construct a three-dimensional Cartesian coordinate system based on a certain corner point of the fixed seat of the robotic arm;

[0013] Locate the transmission pairs in the robotic arm and determine the movement range of each transmission pair;

[0014] Select movement points within the movement range of each transmission pair according to a preset detection density to obtain the movement point set of each transmission pair;

[0015] Select and only select one moving point from the set of moving points of each transmission pair for combination;

[0016] Determine the working point coordinates of each combination in the simulation software, and determine the instruction set based on the synchronization of the moving points; among them, the value of each element in the instruction set is a control instruction, and the serial number of each element corresponds to the transmission pair one by one;

[0017] Count all the working point coordinates and determine the motion area;

[0018] Count the working point coordinates and the corresponding instruction sets of all combinations to obtain the motion instruction table.

[0019] As a further solution of the present invention: the steps of obtaining the robotic arm image with time tags in three mutually perpendicular directions and identifying the robotic arm image to locate the working point include:

[0020] Take the x-axis, y-axis, and z-axis as the three shooting directions, and obtain the robotic arm image with time tags of the robotic arm in real time;

[0021] Based on the AI module, identify the robotic arm image, locate the corner point as the origin in the image, and synchronously determine the three two-dimensional coordinates of the working point; the working point is the center position of the executing part;

[0022] Register the three two-dimensional coordinates to obtain the three-dimensional coordinates of the working point;

[0023] Insert the time tag of the robotic arm image into the three-dimensional coordinates.

[0024] As a further solution of the present invention: the steps of sorting the working points according to the time tags to obtain the working trajectory include:

[0025] Read the three-dimensional coordinates with time tags;

[0026] Sort the three-dimensional coordinates according to the time sequence to obtain a set of three-dimensional coordinates;

[0027] Fit the working trajectory based on the set of three-dimensional coordinates;

[0028] The fitting process is:

[0029]

[0030] In the formula, B(t) is the parametric form of the working trajectory, B i,n represents the Bessel basis function, is the binomial coefficient, P iRepresents the $i$-th three-dimensional coordinate in the set of three-dimensional coordinates. The total number of three-dimensional coordinates in the set of three-dimensional coordinates is set to $(n + 1)$; where $t$ is the independent variable. When $t$ takes zero, it represents the first three-dimensional coordinate in the working trajectory. When $t$ takes one, it represents the last three-dimensional coordinate in the working trajectory.

[0031] As a further solution of the present invention: The steps of inserting the working trajectory into the motion area, displaying the motion area containing the working trajectory, and receiving the supplementary points input by the user include:

[0032] Insert the working trajectory into the motion area;

[0033] Extend the working trajectory based on the robotic arm model;

[0034] In the motion area, remove the extended area as the optional area;

[0035] Display the optional area and receive the supplementary points input by the user.

[0036] As a further solution of the present invention: The method further includes:

[0037] Use the motion instruction table as a sample set to train the neural network model; The input of the neural network model is coordinates, and the output is instructions;

[0038] When the neural network model is trained, receive the supplementary points based on the neural network model and output instructions.

[0039] The technical solution of the present invention also provides a vision perception system based on AI. The system includes:

[0040] An instruction table construction module, used to obtain the motion boundary of the robotic arm, determine the motion area according to the motion boundary, and synchronously construct a motion instruction table; The motion instruction table includes a coordinate item and an instruction item;

[0041] A working point positioning module, used to obtain the robotic arm image with time tags of the robotic arm in three mutually perpendicular directions in real time, identify the robotic arm image, and locate the working point;

[0042] A working trajectory generation module, used to sort the working points according to the time tags to obtain the working trajectory;

[0043] A supplementary point receiving module, used to insert the working trajectory into the motion area, display the motion area containing the working trajectory, and receive the supplementary points input by the user;

[0044] An instruction simulation module, used to query the instructions in the motion instruction table based on the supplementary points as reference instructions.

[0045] As a further solution of the present invention: The instruction table construction module includes:

[0046] A coordinate system construction unit, configured to construct a three-dimensional Cartesian coordinate system based on a certain corner point of the fixed seat of the robotic arm;

[0047] A transmission pair positioning unit, configured to position the transmission pairs in the robotic arm and determine the motion range of each transmission pair;

[0048] A motion point selection unit, configured to select motion points within the motion range of each transmission pair according to a preset detection density to obtain a set of motion points for each transmission pair;

[0049] A motion point combination unit, configured to select and only select one motion point from each set of motion points of the transmission pairs for combination;

[0050] An instruction group determination unit, configured to determine the working point coordinates of each combination in the simulation software and determine the instruction group based on the synchronous motion points; wherein, the value of each element in the instruction group is a control instruction, and the sequence number of each element corresponds to the transmission pair one by one;

[0051] A region determination unit, configured to count all the working point coordinates and determine the motion region;

[0052] A data statistics unit, configured to count the working point coordinates and the corresponding instruction groups of all combinations to obtain a motion instruction table.

[0053] As a further solution of the present invention: The working point positioning module includes:

[0054] An image acquisition unit, configured to use the x-axis, y-axis, and z-axis as three shooting directions to acquire a robotic arm image with a time tag of the robotic arm in real time;

[0055] An image recognition unit, configured to recognize the robotic arm image based on the AI module, locate the corner point serving as the origin in the image, and synchronously determine the three two-dimensional coordinates of the working point; the working point is the central position of the actuator;

[0056] A coordinate registration unit, configured to register the three two-dimensional coordinates to obtain the three-dimensional coordinates of the working point;

[0057] A time tag insertion unit, configured to insert the time tag of the robotic arm image into the three-dimensional coordinates.

[0058] As a further solution of the present invention: The working trajectory generation module includes:

[0059] A three-dimensional coordinate reading unit, configured to read the three-dimensional coordinates with a time tag;

[0060] A three-dimensional coordinate sorting unit, which is used to sort three-dimensional coordinates according to the time sequence to obtain a set of three-dimensional coordinates;

[0061] Fit the working trajectory based on the set of three-dimensional coordinates;

[0062] The fitting process is as follows:

[0063]

[0064] In the formula, B(t) is the parametric form of the working trajectory, and B i,n represents the Bessel basis function, is the binomial coefficient, P i represents the i-th three-dimensional coordinate in the set of three-dimensional coordinates. The total number of three-dimensional coordinates in the set of three-dimensional coordinates is set to (n + 1); among them, t is the independent variable. When t takes zero, it represents the first three-dimensional coordinate in the working trajectory. When t takes one, it represents the last three-dimensional coordinate in the working trajectory.

[0065] Compared with the prior art, the beneficial effects of the present invention are as follows: The present invention introduces an AI visual perception scheme, simplifies the recognition process, determines the working area that the execution part of the robotic arm can reach, calibrates several points in the working area, and at the same time determines the corresponding instruction set. When receiving the user's demand for adding a work station, the corresponding instruction set can be quickly queried according to the position of the added work station, greatly reducing the complexity of task adjustment of the robotic arm. Description of the Drawings

[0066] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings in the following description are only some embodiments of the present invention.

[0067] Figure 1 It is a flow chart of the AI-based visual perception method.

[0068] Figure 2 It is the first sub-flow chart of the AI-based visual perception method.

[0069] Figure 3 It is the second sub-flow chart of the AI-based visual perception method.

[0070] Figure 4 It is the third sub-flow chart of the AI-based visual perception method.

[0071] Figure 5 It is the fourth sub-flow chart of the AI-based visual perception method.

[0072] Figure 6 It is the composition structure diagram of the AI-based visual perception system. Specific Embodiments

[0073] In order to make the technical problems, technical solutions and beneficial effects to be solved by the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0074] Figure 1 As a flowchart of a vision perception method based on AI, in an embodiment of the present invention, a vision perception method based on AI, the method includes:

[0075] Step S100: Obtain the motion boundary of the robotic arm, determine the motion area according to the motion boundary, and synchronously construct a motion instruction table; the motion instruction table includes a coordinate item and an instruction item;

[0076] This application is applied to the technical field of robotic arm control. There are multiple transmission mechanisms in the robotic arm, and each of these transmission mechanisms has an independent limit position. By counting these limit positions, the motion boundary of the robotic arm can be obtained. The motion boundary is the farthest position that the robotic arm can reach, and the interior of the motion boundary is the motion area; at a preset interval, some points are evenly selected in the motion area, and it is obtained what kind of instruction is required for the robotic arm to move to this point. The coordinates and instructions of the points are grouped into one set, and multiple sets of data obtained by statistics are used to obtain the motion instruction table.

[0077] Step S200: Real-time obtain the robotic arm images with time tags of the robotic arm in three mutually perpendicular directions, and identify the robotic arm images to locate the working point;

[0078] The robotic arm images of the robotic arm are obtained in real time. When obtaining the robotic arm images, the acquisition time also needs to be recorded and stored in the form of time tags; by identifying the robotic arm images, the area in the robotic arm that contacts the working station can be located, and points are selected in this area, which is called the working point; when the executing part of the robotic arm is a fixture, generally the center of the fixture is selected as the working point; in addition, the robotic arm images are two-dimensional data, while the motion of the robotic arm is three-dimensional, and there is a certain dimensional difference. Therefore, it is necessary to obtain the robotic arm images in three mutually perpendicular directions respectively, and then synthesize the recognition results of the robotic arm images at each angle to obtain the final three-dimensional coordinates; in practical applications, when the robotic arm is installed, the x-axis, y-axis and z-axis will be synchronously confirmed. These three axes are perpendicular to each other. Therefore, taking these three axes as the shooting directions and installing cameras can obtain the robotic arm images.

[0079] Step S300: Sort the working points according to the time tags to obtain the working trajectory;

[0080] Since the robotic arm image contains time tags, and the identified working points also have time tags, the time sequence is determined according to the time tags, and the working points are connected and fitted to obtain the working trajectory.

[0081] Step S400: Insert the working trajectory into the motion area, display the motion area containing the working trajectory, and receive the supplementary points input by the user;

[0082] Inserting the working trajectory into the determined motion area and displaying it can enable the user to intuitively observe which areas the robotic arm has passed through in actual work and which areas it can still reach; opening a point receiving port to receive the points selected by the user in the areas that can still be reached, which are called supplementary points.

[0083] Step S500: Query the instructions in the motion instruction table based on the supplementary points as reference instructions;

[0084] Supplementary points are generally represented in the form of coordinates. After receiving the supplementary points input by the user, match the corresponding instructions in the motion instruction table according to the supplementary points, which are called reference instructions, and feedback them to the user; this means that the user can quickly obtain the approximate instructions for the robotic arm to reach any position, which is extremely convenient when adding workstations.

[0085] Figure 2 It is the first sub-process block diagram of the AI-based visual perception method. The steps of obtaining the motion boundary of the robotic arm, determining the motion area according to the motion boundary, and synchronously constructing the motion instruction table include:

[0086] Step S101: Construct a three-dimensional Cartesian coordinate system based on a corner point of the fixed seat of the robotic arm;

[0087] Step S102: Locate the transmission pairs in the robotic arm and determine the motion range of each transmission pair;

[0088] Step S103: Select motion points within the motion range of each transmission pair according to the preset detection density to obtain the motion point set of each transmission pair;

[0089] Step S104: Select and only select one motion point from the motion point set of each transmission pair for combination;

[0090] Step S105: Determine the working point coordinates of each combination in the simulation software and synchronously determine the instruction group based on the motion points; where each element value in the instruction group is a control instruction, and the serial number of each element corresponds to the transmission pair one by one;

[0091] Step S106: Count all the working point coordinates and determine the motion area;

[0092] Step S107: Count the working point coordinates and corresponding instruction groups of all combinations to obtain a motion instruction table.

[0093] When installing the robotic arm, it is necessary to synchronously determine a coordinate system to define the x-axis, y-axis, and z-axis. Generally, a certain corner point of the fixed base of the robotic arm is taken as the origin, and then three directions of the fixed base are respectively selected as the x-axis, y-axis, and z-axis to obtain a three-dimensional Cartesian coordinate system. Locate the transmission pairs in the robotic arm. The transmission pair is the core part of the transmission mechanism in the robotic arm. There are limits to the relative positions of the two components of each transmission pair, and the range of allowable relative motion is the motion range. For a sliding pair, it is the relative sliding distance, and for a rotating pair, it is the relative rotation angle.

[0094] Select motion points within the motion range of each transmission pair according to a preset detection density. The detection density refers to how many motion points are selected within a unit range. After selection, at least one motion point can be obtained within the motion range corresponding to each transmission pair, and the obtained motion points are counted as a motion point set. Then, select and only select one motion point from each motion point set of the transmission pairs for combination. Each motion point is actually a position condition. After the states of all transmission pairs are determined, the entire robotic arm is in a stable state. Read the working point coordinates of the robotic arm in the stable state, and at the same time obtain the instruction groups required when each transmission pair is at the corresponding motion point (one instruction corresponds to one transmission pair), and statistically organize the working point coordinates and instruction groups in one-to-one correspondence to obtain a motion instruction table. With the working point coordinates as the center, determine a spherical area, count the spherical areas corresponding to all working point coordinates, then calculate the sum of all spherical areas and fit the boundary to obtain a large area, which is called the motion area.

[0095] Figure 3 It is the second sub-process block diagram of the AI-based visual perception method. The steps of obtaining the robotic arm image with time tags of the robotic arm in three mutually perpendicular directions and identifying the robotic arm image to locate the working point include:

[0096] Step S201: Use the x-axis, y-axis, and z-axis as three shooting directions to obtain the robotic arm image with time tags of the robotic arm in real time.

[0097] Step S202: Based on the AI module, identify the robotic arm image, locate the corner point serving as the origin in the image, and synchronously determine the three two-dimensional coordinates of the working point. The working point is the central position of the executing part.

[0098] Step S203: Register the three two-dimensional coordinates to obtain the three-dimensional coordinates of the working point.

[0099] Step S204: Insert the time tag of the robotic arm image into the three-dimensional coordinates.

[0100] In an example of the technical solution of the present invention, the positioning process of the working point is specifically described. The x-axis, y-axis, and z-axis are used as three shooting directions, and the robotic arm images with time tags of the robotic arm are obtained in real time. Based on the AI module, the robotic arm images are recognized, and the corner point serving as the origin is located in the image. Then, with the corner point as the origin, the two-dimensional coordinates of the working point are determined. For the robotic arm images at three angles at the same moment, three two-dimensional coordinates of the working point can be obtained, which are (x, y), (y, z), and (x, z) respectively. It should be noted that there may be a certain difference in the positioning results of the working point of the robotic arm images at three angles. For example, the x coordinate can be obtained in two images. At this time, registration is required. There are many registration methods, and taking the average value is sufficient. Finally, the time tags of the three robotic arm images analyzed together are used as the time tags of the three-dimensional coordinates.

[0101] It should be noted that it is best that the frequencies of obtaining the robotic arm images with time tags of the robotic arm in the three directions are the same. If they are not the same, then time registration is required, and the three robotic arm images with sufficiently close times (the time difference is less than a preset threshold) are used as a group of images for analysis.

[0102] Figure 4 It is the third sub-process block diagram of the vision perception method based on AI. The steps of sorting the working points according to the time tags to obtain the working trajectory include:

[0103] Step S301: Read the three-dimensional coordinates with time tags;

[0104] Step S302: Sort the three-dimensional coordinates according to the time sequence to obtain a set of three-dimensional coordinates;

[0105] Step S303: Fit the working trajectory based on the set of three-dimensional coordinates.

[0106] The working trajectory is connected and fitted by three-dimensional coordinates. Read the three-dimensional coordinates with time tags, sort the three-dimensional coordinates according to the time sequence to obtain a set of three-dimensional coordinates. The obtained set of three-dimensional coordinates is actually an array, and the elements in it have an order, not disordered. Then, fit the working trajectory based on the set of three-dimensional coordinates. The fitting process is as follows:

[0107]

[0108] In the formula, B(t) is the parametric form of the working trajectory, and B i,n represents the Bessel basis function, is the binomial coefficient, P iRepresents the i-th three-dimensional coordinate in the set of three-dimensional coordinates. The total number of three-dimensional coordinates in the set of three-dimensional coordinates is set to (n + 1); among them, t is the independent variable. When t takes zero, it represents the first three-dimensional coordinate in the working trajectory. When t takes one, it represents the last three-dimensional coordinate in the working trajectory.

[0109] In an example of the technical solution of the present invention, the fitting process of the working trajectory is described. The Bezier fitting method is adopted to fit the set of three-dimensional coordinates. The independent variable uses values in the range from zero to one. Zero represents the first three-dimensional coordinate, and one represents the last three-dimensional coordinate; actually, the value of t is continuous. It is just an independent variable. Its physical meaning is that assuming the length of the working trajectory is one (100%), what are the three-dimensional coordinates at each position, and each position is controlled by t. It is also feasible to understand it as the percentage of the working trajectory.

[0110] Figure 5 It is the fourth sub-process block diagram of the AI-based visual perception method. The steps of inserting the working trajectory into the motion area, displaying the motion area containing the working trajectory, and receiving the supplementary points input by the user include:

[0111] Step S401: Insert the working trajectory into the motion area;

[0112] Step S402: Extend the working trajectory based on the robotic arm model;

[0113] Step S403: Exclude the extended area in the motion area as the optional area;

[0114] Step S404: Display the optional area and receive the supplementary points input by the user.

[0115] The above content provides a specific interaction solution. Insert the working trajectory into the motion area. During the insertion process, it is necessary to extend the working trajectory. The way of extension is to use the working point as the calibration point to calibrate the volume occupied by the robotic arm in the motion area. This is equivalent to expanding a point into a robotic arm model. Each point on the working trajectory expands the model once, and the area that the robotic arm body can occupy in the motion area can be obtained. Exclude the extended area in the motion area, and the remaining area is the area where there will definitely be no robotic arm. Use it as the optional area and receive the supplementary points input by the user. The supplementary points indicate the positions where workstations can be added.

[0116] It is worth mentioning that in the process of expanding the working point into a robot model, it is necessary to obtain the tangent of the working point at the working trajectory, and then use the tangent as a reference to expand the robot model. The robot model itself is very complex, and proportional expansion will be very complicated. Therefore, the present application will obtain the base position of the robot (a component at a fixed position), and determine a regular area with the same size as the robot according to the base position and the tangent, such as a set of some rectangles, and use it directly as the robot model, thereby simplifying the expansion process; in addition, there is another solution, that is, while taking an image of the robot and identifying the working point, the robot model can be identified based on the AI ​​module (the accuracy requirement is not high), and a corresponding relationship between it and the working point can be established. At this time, when expansion is required, the robot model can be directly read and inserted into the corresponding position; since the image recognition process of the present application is completed by the AI ​​module, both the identification of the working point and the identification of the robot model are not complicated, and only some samples need to be established in advance to obtain a localized model that is sufficient for the recognition work.

[0117] As an example of the technical solution of the present invention, the method further includes:

[0118] The motion instruction table is used as a sample set to train a neural network model; the input of the neural network model is the coordinates, and the output is the instruction;

[0119] After the neural network model is trained, it receives supplementary points and outputs instructions based on the neural network model.

[0120] The motion instruction table is a mapping relationship between coordinates and instruction groups. The number of elements in the motion instruction table is limited. The greater the detection density, the more elements in the motion instruction table, and the more computing resources required (there is a combination process, and the increase is very large). For this, the present application uses the motion instruction table as a sample set to train a neural network model as a mapping relationship between coordinates and the motion instruction table. At this time, after the user enters the coordinates, there is no need to match them in the motion instruction table to get the corresponding high-precision instructions. The user can then make fine adjustments to get the most accurate instructions.

[0121] It should be noted that when the robotic arm is being developed, since the position of the working point is related to the transmission pair, the transmission pair itself will undergo kinematic analysis. Therefore, there is indeed a mapping relationship between the position of the working point and each instruction, and this mapping relationship can be calculated by the R&D personnel. However, when there are more transmission pairs, the mechanical analysis and kinematic analysis processes involved are extremely numerous. Therefore, the neural network model training process provided in this application can eliminate the extremely complex analysis process and thereby improve efficiency.

[0122] Further, when the executing member of the robotic arm is replaced, the working point will also change accordingly. If the most accurate mapping relationship is determined by means of mechanical analysis and kinematic analysis, it will consume a lot of time of professionals. However, by using the above method of training the neural network model, a relatively accurate mapping relationship can be quickly obtained by the machine. Before the reference instruction is put into use, professionals only need to make fine-tuning, which is very time-saving and labor-saving.

[0123] Figure 6 FIG. 4 is a block diagram of the composition structure of an AI-based visual perception system. In an embodiment of the present invention, an AI-based visual perception system, the system 10 includes:

[0124] An instruction table construction module 11, configured to obtain the motion boundary of the robotic arm, determine the motion area according to the motion boundary, and synchronously construct a motion instruction table; the motion instruction table includes a coordinate item and an instruction item;

[0125] A working point positioning module 12, configured to obtain a robotic arm image with a time tag of the robotic arm in three mutually perpendicular directions in real time, identify the robotic arm image, and position the working point;

[0126] A working trajectory generation module 13, configured to sort the working points according to the time tag to obtain a working trajectory;

[0127] A supplementary point receiving module 14, configured to insert the working trajectory into the motion area, display the motion area containing the working trajectory, and receive supplementary points input by the user;

[0128] An instruction simulation module 15, configured to query an instruction in the motion instruction table based on the supplementary point as a reference instruction.

[0129] Further, the instruction table construction module 11 includes:

[0130] A coordinate system construction unit, configured to construct a three-dimensional Cartesian coordinate system based on a certain corner point of the fixed seat of the robotic arm;

[0131] A transmission pair positioning unit, configured to position the transmission pair in the robotic arm and determine the motion range of each transmission pair;

[0132] A motion point selection unit, configured to select motion points within the motion range of each transmission pair according to a preset detection density to obtain a motion point set of each transmission pair;

[0133] A motion point combination unit, configured to select and only select one motion point from the motion point set of each transmission pair for combination;

[0134] An instruction group determination unit is used to determine the working point coordinates of each combination in simulation software and determine the instruction group based on the synchronization of moving points; wherein, the value of each element in the instruction group is a control instruction, and the serial number of each element corresponds to a transmission pair one by one;

[0135] A region determination unit is used to count all the working point coordinates and determine the motion region;

[0136] A data statistics unit is used to count the working point coordinates of all combinations and the corresponding instruction groups to obtain a motion instruction table.

[0137] Specifically, the working point positioning module 12 includes:

[0138] An image acquisition unit is used to use the x-axis, y-axis, and z-axis as three shooting directions to acquire the robotic arm image with time tags of the robotic arm in real time;

[0139] An image recognition unit is used to recognize the robotic arm image based on the AI module, locate the corner point as the origin in the image, and synchronously determine the three two-dimensional coordinates of the working point; the working point is the central position of the executing part;

[0140] A coordinate registration unit is used to register the three two-dimensional coordinates to obtain the three-dimensional coordinates of the working point;

[0141] A time tag insertion unit is used to insert the time tag of the robotic arm image into the three-dimensional coordinates.

[0142] Furthermore, the working trajectory generation module 13 includes:

[0143] A three-dimensional coordinate reading unit is used to read the three-dimensional coordinates with time tags;

[0144] A three-dimensional coordinate sorting unit is used to sort the three-dimensional coordinates according to the time sequence to obtain a three-dimensional coordinate set;

[0145] Fit the working trajectory based on the three-dimensional coordinate set;

[0146] The fitting process is as follows:

[0147]

[0148] In the formula, B(t) is the parametric form of the working trajectory, B i,n represents the Bessel basis function, is the binomial coefficient, P iRepresents the i-th three-dimensional coordinate in a set of three-dimensional coordinates, and the total number of three-dimensional coordinates in the set of three-dimensional coordinates is set to (n + 1); among them, t is the independent variable. When t is zero, it represents the first three-dimensional coordinate in the working trajectory, and when t is one, it represents the last three-dimensional coordinate in the working trajectory.

[0149] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, and improvements made within the spirit and principles of the present invention shall be included within the protection scope of the present invention.

Claims

1. A visual perception method based on AI, characterized in that: The method comprises: Acquire the motion boundary of the robot arm, determine the motion area according to the motion boundary, and simultaneously construct a motion instruction table; the motion instruction table includes coordinate items and instruction items; Acquire the robot arm images with time tags in real time in three mutually perpendicular directions, identify the robot arm images, and locate the working point; Sort the work points according to the time labels to obtain the work trajectory; Inserting the working track into the motion area, displaying the motion area containing the working track, and receiving a supplementary point input by a user; Based on the supplementary point, query the instruction in the motion instruction table as a reference instruction; The steps of obtaining the motion boundary of the robot arm, determining the motion area according to the motion boundary, and synchronously constructing the motion instruction table include: A three-dimensional Cartesian coordinate system is constructed based on a corner point of the fixed seat of the robot arm; Locate the transmission pairs in the robot arm and determine the range of motion of each transmission pair; Selecting motion points within the motion range of each transmission pair according to a preset detection density to obtain a motion point set of each transmission pair; Select and only select one motion point from the set of motion points of each transmission pair for combination; Determine the coordinates of the working points of each combination in the simulation software, and determine the instruction group based on the motion point synchronization; wherein the value of each element in the instruction group is a control instruction, and the serial number of each element corresponds to the transmission pair one by one; Count the coordinates of all working points and determine the motion area; Count all the combined working point coordinates and corresponding instruction groups to obtain the motion instruction table.

2. The AI-based visual perception method according to claim 1, characterized in that: The steps of acquiring the robot arm images containing time tags in real time in three mutually perpendicular directions, identifying the robot arm images, and locating the working point include: The x-axis, y-axis and z-axis are used as three shooting directions to obtain the robot arm images with time tags in real time; Based on the AI ​​module, the robot arm image is recognized, the corner point as the origin is located in the image, and the three two-dimensional coordinates of the working point are simultaneously determined; the working point is the center position of the actuator; The three 2D coordinates are registered to obtain the 3D coordinates of the working point; Insert time tags of robot images into 3D coordinates.

3. The AI-based visual perception method according to claim 2, characterized in that: The step of sorting the working points according to the time tags to obtain the working trajectory includes: Read the three-dimensional coordinates with time tags; Sort the three-dimensional coordinates according to the time sequence to obtain a three-dimensional coordinate set; Fitting the working trajectory based on the three-dimensional coordinate set; The fitting process is: ； ； In the formula, is the parameterized form of the working trajectory, represents the Bessel basis function, is the binomial coefficient, ; Represents the first 3D coordinates, the total number of 3D coordinates in the 3D coordinate set is set to Among them, is the independent variable, When it is zero, it indicates the first three-dimensional coordinate in the working trajectory. When 1 is taken, it represents the last three-dimensional coordinate in the working trajectory.

4. The AI-based visual perception method according to claim 1, characterized in that: The steps of inserting the working track into the motion area, displaying the motion area containing the working track, and receiving the supplementary points input by the user include: inserting the working trajectory into the motion area; Extend the working trajectory based on the robot arm model; Eliminate the extended area in the motion area as an optional area; Displays an optional area and receives additional points entered by the user.

5. The AI-based visual perception method according to claim 1, characterized in that: The method further comprises: The motion instruction table is used as a sample set to train a neural network model; the input of the neural network model is the coordinates, and the output is the instruction; After the neural network model is trained, it receives supplementary points and outputs instructions based on the neural network model.

6. An AI-based visual perception system, characterized in that: The system comprises: An instruction table construction module is used to obtain the motion boundary of the robot arm, determine the motion area according to the motion boundary, and simultaneously construct a motion instruction table; the motion instruction table includes coordinate items and instruction items; A working point positioning module is used to obtain the robot arm images with time tags in real time in three mutually perpendicular directions, identify the robot arm images, and locate the working point; A work trajectory generation module is used to sort the work points according to the time tags to obtain the work trajectory; A supplementary point receiving module, used for inserting the working track into the motion area, displaying the motion area containing the working track, and receiving supplementary points input by the user; An instruction simulation module, used for searching instructions in a motion instruction table based on the supplementary point as a reference instruction; The instruction table building module includes: A coordinate system building unit, used for building a three-dimensional Cartesian coordinate system based on a corner point of a fixed seat of the robot arm; The transmission pair positioning unit is used to position the transmission pair in the robot arm and determine the motion range of each transmission pair; A motion point selection unit, used to select motion points within the motion range of each transmission pair according to a preset detection density, to obtain a motion point set of each transmission pair; A motion point combination unit, used for selecting and only selecting one motion point from the motion point set of each transmission pair for combination; An instruction group determination unit is used to determine the working point coordinates of each combination in the simulation software, and to determine the instruction group based on the motion point synchronization; wherein the value of each element in the instruction group is a control instruction, and the serial number of each element corresponds to the transmission pair one by one; The area determination unit is used for counting the coordinates of all working points and determining the motion area; The data statistics unit is used to count the coordinates of all combined working points and the corresponding instruction groups to obtain a motion instruction table.

7. The AI-based visual perception system according to claim 6, characterized in that: The working point positioning module comprises: An image acquisition unit, used to use the x-axis, y-axis and z-axis as three shooting directions to acquire a robotic arm image containing a time tag of the robotic arm in real time; An image recognition unit, used to recognize the image of the robot arm based on the AI ​​module, locate the corner point as the origin in the image, and simultaneously determine the three two-dimensional coordinates of the working point; the working point is the center position of the actuator; A coordinate registration unit is used to register three two-dimensional coordinates to obtain the three-dimensional coordinates of the working point; The time tag insertion unit is used to insert the time tag of the robot arm image into the three-dimensional coordinates.

8. The AI-based visual perception system according to claim 7, characterized in that: The work trajectory generation module includes: A three-dimensional coordinate reading unit, used for reading the three-dimensional coordinates containing a time tag; A three-dimensional coordinate sorting unit, used for sorting the three-dimensional coordinates according to the time sequence to obtain a three-dimensional coordinate set; Fitting the working trajectory based on the three-dimensional coordinate set; The fitting process is: ； ； In the formula, is the parameterized form of the working trajectory, represents the Bessel basis function, is the binomial coefficient, ; Represents the first 3D coordinates, the total number of 3D coordinates in the 3D coordinate set is set to Among them, is the independent variable, When it is zero, it indicates the first three-dimensional coordinate in the working trajectory. When 1 is taken, it represents the last three-dimensional coordinate in the working trajectory.

Images