AI-based visual perception method and system

Abstract

The invention relates to the technical field of visual perception, and particularly discloses an AI-based visual perception method and system, and the method comprises the steps: obtaining a motion boundary of a mechanical arm, determining a motion region according to the motion boundary, and synchronously constructing a motion instruction list; a mechanical arm image, containing a time label, of the mechanical arm is obtained in real time, the mechanical arm image is recognized, and a working point is positioned; sorting the working points according to the time labels to obtain a working track; inserting the working track into the motion area, displaying the motion area containing the working track, and receiving a supplementary point position input by a user; and querying an instruction in a motion instruction table based on the supplementary point as a reference instruction. The AI visual perception scheme is introduced, the recognition process is simplified, the working area where the execution part of the mechanical arm can reach is determined, the target position of the user is received in the working area, then the corresponding instruction set is quickly inquired, and the task adjustment complexity of the mechanical arm is greatly reduced.

Description

Technical Field

[0001] The present invention relates to the field of visual perception technology, and specifically to an AI-based visual perception method and system. Background Art

[0002] During the working process, the robot arm will at least move from one workstation to another. During this process, the movement of the working end of the robot arm is a trajectory, and there are many areas that it can reach. Additional workstations can be added in these areas. However, each additional workstation requires a multi-department joint program design, which is very cumbersome. How to simplify the difficulty of adding workstations is the technical problem that the technical solution of the present invention wants to solve. Summary of the invention

[0003] The purpose of the present invention is to provide an AI-based visual perception method and system to solve the problems raised in the above-mentioned background technology.

[0004] To achieve the above object, the present invention provides the following technical solutions:

[0005] A visual perception method based on AI, the method comprising:

[0006] Acquire the motion boundary of the robot arm, determine the motion area according to the motion boundary, and simultaneously construct a motion instruction table; the motion instruction table includes coordinate items and instruction items;

[0007] Acquire the robot arm images with time tags in real time in three mutually perpendicular directions, identify the robot arm images, and locate the working point;

[0008] Sort the work points according to the time labels to obtain the work trajectory;

[0009] Inserting the working track into the motion area, displaying the motion area containing the working track, and receiving a supplementary point input by a user;

[0010] Based on the supplementary point, an instruction is searched in the motion instruction table as a reference instruction.

[0011] As a further solution of the present invention: the steps of obtaining the motion boundary of the robot arm, determining the motion area according to the motion boundary, and synchronously constructing the motion instruction table include:

[0012] A three-dimensional Cartesian coordinate system is constructed based on a corner point of the fixed seat of the robot arm;

[0013] Locate the transmission pairs in the robot arm and determine the range of motion of each transmission pair;

[0014] Selecting motion points within the motion range of each transmission pair according to a preset detection density to obtain a motion point set of each transmission pair;

[0015] Select and only select one motion point from the set of motion points of each transmission pair for combination;

[0016] Determine the coordinates of the working points of each combination in the simulation software, and determine the instruction group based on the motion point synchronization; wherein the value of each element in the instruction group is a control instruction, and the serial number of each element corresponds to the transmission pair one by one;

[0017] Count the coordinates of all working points and determine the motion area;

[0018] Count all the combined working point coordinates and corresponding instruction groups to obtain the motion instruction table.

[0019] As a further solution of the present invention: the steps of acquiring the robot arm images containing time tags in real time in three mutually perpendicular directions, identifying the robot arm images, and locating the working point include:

[0020] The x-axis, y-axis and z-axis are used as three shooting directions to obtain the robot arm images with time tags in real time;

[0021] Based on the AI ​​module, the robot arm image is recognized, the corner point as the origin is located in the image, and the three two-dimensional coordinates of the working point are simultaneously determined; the working point is the center position of the actuator;

[0022] The three 2D coordinates are registered to obtain the 3D coordinates of the working point;

[0023] Insert time tags of robot images into 3D coordinates.

[0024] As a further solution of the present invention: the step of sorting the working points according to the time tags to obtain the working trajectory includes:

[0025] Read the three-dimensional coordinates with time tags;

[0026] Sort the three-dimensional coordinates according to the time sequence to obtain a three-dimensional coordinate set;

[0027] Fitting the working trajectory based on the three-dimensional coordinate set;

[0028] The fitting process is:

[0029]

[0030] Where B(t) is the parameterized form of the working trajectory, B i,n represents the Bessel basis function, is the binomial coefficient, P irepresents the i-th three-dimensional coordinate in the three-dimensional coordinate set, and the total number of three-dimensional coordinates in the three-dimensional coordinate set is set to (n+1); where t is an independent variable, when t is zero, it represents the first three-dimensional coordinate in the working trajectory, and when t is one, it represents the last three-dimensional coordinate in the working trajectory.

[0031] As a further solution of the present invention: the step of inserting the working track into the motion area, displaying the motion area containing the working track, and receiving the supplementary point input by the user comprises:

[0032] inserting the working trajectory into the motion area;

[0033] Extend the working trajectory based on the robot arm model;

[0034] Eliminate the extended area in the motion area as an optional area;

[0035] Displays an optional area and receives additional points entered by the user.

[0036] As a further solution of the present invention: the method further comprises:

[0037] The motion instruction table is used as a sample set to train a neural network model; the input of the neural network model is the coordinates, and the output is the instruction;

[0038] After the neural network model is trained, it receives supplementary points and outputs instructions based on the neural network model.

[0039] The technical solution of the present invention also provides an AI-based visual perception system, the system comprising:

[0040] An instruction table construction module is used to obtain the motion boundary of the robot arm, determine the motion area according to the motion boundary, and simultaneously construct a motion instruction table; the motion instruction table includes coordinate items and instruction items;

[0041] A working point positioning module is used to obtain the robot arm images with time tags in real time in three mutually perpendicular directions, identify the robot arm images, and locate the working point;

[0042] A work trajectory generation module is used to sort the work points according to the time tags to obtain the work trajectory;

[0043] A supplementary point receiving module, used for inserting the working track into the motion area, displaying the motion area containing the working track, and receiving supplementary points input by the user;

[0044] The instruction simulation module is used to query instructions in the motion instruction table based on the supplementary point as reference instructions.

[0045] As a further solution of the present invention: the instruction table building module includes:

[0046] A coordinate system building unit, used for building a three-dimensional Cartesian coordinate system based on a corner point of a fixed seat of the robot arm;

[0047] The transmission pair positioning unit is used to position the transmission pair in the robot arm and determine the motion range of each transmission pair;

[0048] A motion point selection unit, used to select motion points within the motion range of each transmission pair according to a preset detection density, to obtain a motion point set of each transmission pair;

[0049] A motion point combination unit, used for selecting and only selecting one motion point from the motion point set of each transmission pair for combination;

[0050] An instruction group determination unit is used to determine the working point coordinates of each combination in the simulation software, and to determine the instruction group based on the motion point synchronization; wherein the value of each element in the instruction group is a control instruction, and the serial number of each element corresponds to the transmission pair one by one;

[0051] The area determination unit is used for counting the coordinates of all working points and determining the motion area;

[0052] The data statistics unit is used to count the coordinates of all combined working points and the corresponding instruction groups to obtain a motion instruction table.

[0053] As a further solution of the present invention: the working point positioning module includes:

[0054] An image acquisition unit, used to use the x-axis, y-axis and z-axis as three shooting directions to acquire a robotic arm image containing a time tag of the robotic arm in real time;

[0055] An image recognition unit, used to recognize the image of the robot arm based on the AI ​​module, locate the corner point as the origin in the image, and simultaneously determine the three two-dimensional coordinates of the working point; the working point is the center position of the actuator;

[0056] A coordinate registration unit is used to register three two-dimensional coordinates to obtain the three-dimensional coordinates of the working point;

[0057] The time tag insertion unit is used to insert the time tag of the robot arm image into the three-dimensional coordinates.

[0058] As a further solution of the present invention: the work trajectory generation module includes:

[0059] A three-dimensional coordinate reading unit, used for reading the three-dimensional coordinates containing a time tag;

[0060] A three-dimensional coordinate sorting unit, used for sorting the three-dimensional coordinates according to the time sequence to obtain a three-dimensional coordinate set;

[0061] Fitting the working trajectory based on the three-dimensional coordinate set;

[0062] The fitting process is:

[0063]

[0064] Where B(t) is the parameterized form of the working trajectory, B i,n represents the Bessel basis function, is the binomial coefficient, P i represents the i-th three-dimensional coordinate in the three-dimensional coordinate set, and the total number of three-dimensional coordinates in the three-dimensional coordinate set is set to (n+1); where t is an independent variable, when t is zero, it represents the first three-dimensional coordinate in the working trajectory, and when t is one, it represents the last three-dimensional coordinate in the working trajectory.

[0065] Compared with the prior art, the beneficial effects of the present invention are: the present invention introduces an AI visual perception solution, simplifies the recognition process, determines the working area that the robot's actuator can reach, calibrates several points in the working area, and determines the corresponding instruction group at the same time. When the user's input of the need to add a workstation is received, the corresponding instruction group can be quickly queried according to the location of the added workstation, which greatly reduces the complexity of the robot's task adjustment. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention.

[0067] Figure 1 This is a flowchart of the AI-based visual perception method.

[0068] Figure 2 This is the first sub-process flowchart of the AI-based visual perception method.

[0069] Figure 3 This is a block diagram of the second sub-process of the AI-based visual perception method.

[0070] Figure 4 This is the third sub-process flowchart of the AI-based visual perception method.

[0071] Figure 5 This is the fourth sub-process flowchart of the AI-based visual perception method.

[0072] Figure 6 This is a structural block diagram of the AI-based visual perception system. DETAILED DESCRIPTION

[0073] In order to make the technical problems, technical solutions and beneficial effects to be solved by the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0074] Figure 1 The flowchart of the AI-based visual perception method is as follows. In an embodiment of the present invention, an AI-based visual perception method includes:

[0075] Step S100: obtaining a motion boundary of the robot arm, determining a motion area according to the motion boundary, and synchronously constructing a motion instruction table; the motion instruction table includes coordinate items and instruction items;

[0076] The present application is applied to the field of control technology of a robotic arm. There are multiple transmission mechanisms in the robotic arm. These transmission mechanisms all have independent extreme positions. By counting these extreme positions, the motion boundary of the robotic arm can be obtained. The motion boundary is the farthest position that the robotic arm can reach, and the inside of the motion boundary is the motion area. According to preset intervals, some points are evenly selected in the motion area to obtain what kind of instructions are needed for the robotic arm to move to the point, the coordinates of the point and the instructions are grouped together, and the multiple groups of data obtained are counted to obtain a motion instruction table.

[0077] Step S200: acquiring a robotic arm image containing a time tag in real time in three mutually perpendicular directions, identifying the robotic arm image, and locating a working point;

[0078] Acquire the robotic arm image of the robotic arm in real time. While acquiring the robotic arm image, it is also necessary to record the acquisition time and store it in the form of a time tag; recognize the robotic arm image to locate the area in the robotic arm that contacts the workstation, and select a point in this area, which is called a working point; when the actuator of the robotic arm is a fixture, the center of the fixture is generally selected as the working point; in addition, the robotic arm image is two-dimensional data, while the movement of the robotic arm is three-dimensional, which has a certain dimensionality difference. Therefore, it is necessary to acquire the robotic arm image in three mutually perpendicular directions respectively, and then combine the recognition results of the robotic arm image at each angle to obtain the final three-dimensional coordinates; in actual applications, when the robotic arm is installed, it will be synchronously confirmed as the x-axis, y-axis and z-axis, and these three axes are perpendicular to each other. Therefore, use these three axes as the shooting direction, install the camera, and acquire the robotic arm image.

[0079] Step S300: sorting the working points according to the time tags to obtain the working trajectory;

[0080] Since the robot image contains time tags, the identified working points also have time tags. The time sequence is determined according to the time tags, and the working points are connected and fitted to obtain the working trajectory.

[0081] Step S400: inserting the working track into the motion area, displaying the motion area containing the working track, and receiving a supplementary point input by a user;

[0082] Inserting the working trajectory into the determined motion area and displaying it can allow the user to intuitively observe which areas the robot arm can pass through in actual work and which areas it can still reach; opening the point receiving port to receive the points selected by the user in the areas that can still be reached, which are called supplementary points.

[0083] Step S500: searching for instructions in a motion instruction table based on the supplementary point as a reference instruction;

[0084] The supplementary points are generally expressed in the form of coordinates. After receiving the supplementary points input by the user, the corresponding instructions are matched in the motion instruction table according to the supplementary points, which are called reference instructions and fed back to the user. This means that the user can quickly obtain the approximate instructions for the robot arm to reach any position, which is extremely convenient when adding workstations.

[0085] Figure 2 The first sub-flow chart of the AI-based visual perception method, wherein the steps of obtaining the motion boundary of the robot arm, determining the motion area according to the motion boundary, and synchronously constructing the motion instruction table include:

[0086] Step S101: constructing a three-dimensional Cartesian coordinate system based on a corner point of a fixed base of the robot arm;

[0087] Step S102: positioning the transmission pair in the robot arm and determining the motion range of each transmission pair;

[0088] Step S103: selecting a motion point within the motion range of each transmission pair according to a preset detection density to obtain a motion point set of each transmission pair;

[0089] Step S104: selecting and only selecting one motion point from the motion point set of each transmission pair for combination;

[0090] Step S105: determining the working point coordinates of each combination in the simulation software, and determining the instruction group based on the motion point synchronization; wherein the value of each element in the instruction group is a control instruction, and the serial number of each element corresponds to the transmission pair one by one;

[0091] Step S106: Count the coordinates of all working points and determine the motion area;

[0092] Step S107: Count all the combined working point coordinates and corresponding instruction groups to obtain a motion instruction table.

[0093] When installing the robotic arm, it is necessary to synchronously determine a coordinate label to determine the x-axis, y-axis and z-axis. Generally, a corner point of the fixed seat of the robotic arm is used as the origin, and then the three directions of the fixed seat are selected as the x-axis, y-axis and z-axis respectively, and a three-dimensional Cartesian coordinate system can be obtained; the transmission pair is positioned in the robotic arm, and the transmission pair is the core part of the transmission mechanism in the robotic arm. The relative positions of the two components of each transmission pair are limited, and the range of relative motion allowed is the motion range; for the sliding pair, it is the relative sliding distance, and for the rotating pair, it is the relative rotation angle.

[0094] According to the preset detection density, motion points are selected within the motion range of each transmission pair. The detection density is the number of motion points selected within the unit range. After the selection, at least one motion point can be obtained within the motion range corresponding to each transmission pair, and the obtained motion points are counted as a motion point set; then, only one motion point is selected from the motion point set of each transmission pair for combination. Each motion point is actually a position condition. After the states of all transmission pairs are determined, the entire robotic arm is in a stable state. The working point coordinates of the robotic arm in the stable state are read, and the instruction groups required when each transmission pair is at the corresponding motion point are obtained at the same time (one instruction corresponds to one transmission pair), and the working point coordinates and the instruction groups are counted one by one to obtain a motion instruction table; with the working point coordinates as the center, a spherical area is determined, the spherical areas corresponding to all working point coordinates are counted, and then the sum of all spherical areas is calculated and the boundaries are fitted to obtain a large area, which is called a motion area.

[0095] Figure 3 The second sub-flow diagram of the AI-based visual perception method, wherein the steps of acquiring the robot arm images containing time tags in real time in three mutually perpendicular directions, identifying the robot arm images, and locating the working point include:

[0096] Step S201: using the x-axis, y-axis and z-axis as three shooting directions, acquiring a robotic arm image containing a time tag of the robotic arm in real time;

[0097] Step S202: recognizing the image of the robot arm based on the AI ​​module, locating the corner point as the origin in the image, and synchronously determining the three two-dimensional coordinates of the working point; the working point is the center position of the actuator;

[0098] Step S203: aligning the three two-dimensional coordinates to obtain the three-dimensional coordinates of the working point;

[0099] Step S204: inserting the time tag of the robot arm image into the three-dimensional coordinates.

[0100] In an example of the technical solution of the present invention, the positioning process of the working point is described in detail. The x-axis, y-axis and z-axis are used as three shooting directions, and a robotic arm image with a time tag is obtained in real time. The robotic arm image is recognized based on the AI ​​module, and the corner point as the origin is located in the image. Then, the two-dimensional coordinates of the working point are determined with the corner point as the origin. The robotic arm images at three angles at the same time can obtain the three two-dimensional coordinates of the working point, which are (x, y), (y, z) and (x, z). It should be noted that the working point positioning results of the robotic arm images at three angles may have certain differences. For example, the x coordinate will be obtained in two images. At this time, alignment is required. There are many ways to align, and the average can be taken. Finally, the time tags of the three robotic arm images analyzed together are used as the time tags of the three-dimensional coordinates.

[0101] It should be noted that the frequency of acquiring the robotic arm images with time tags in three directions is preferably the same. If not, it is necessary to align the time and analyze the three robotic arm images that are close enough in time (the time difference is less than a preset threshold) as the same group of images.

[0102] Figure 4 The third sub-flow chart of the AI-based visual perception method, wherein the step of sorting the working points according to the time labels to obtain the working trajectory includes:

[0103] Step S301: Read the three-dimensional coordinates containing the time tag;

[0104] Step S302: sorting the three-dimensional coordinates according to the time sequence to obtain a three-dimensional coordinate set;

[0105] Step S303: Fitting the working trajectory based on the three-dimensional coordinate set.

[0106] The working trajectory is formed by connecting and fitting the three-dimensional coordinates. The three-dimensional coordinates with time tags are read and sorted according to the time sequence to obtain a three-dimensional coordinate set. The obtained three-dimensional coordinate set is actually an array, in which the elements are in order and not disordered. Then, the working trajectory is fitted based on the three-dimensional coordinate set. The fitting process is:

[0107]

[0108] Where B(t) is the parameterized form of the working trajectory, B i,n represents the Bessel basis function, is the binomial coefficient, P irepresents the i-th three-dimensional coordinate in the three-dimensional coordinate set, and the total number of three-dimensional coordinates in the three-dimensional coordinate set is set to (n+1); where t is an independent variable, when t is zero, it represents the first three-dimensional coordinate in the working trajectory, and when t is one, it represents the last three-dimensional coordinate in the working trajectory.

[0109] In an example of the technical solution of the present invention, the fitting process of the working trajectory is described. The Bessel fitting method is used to fit the three-dimensional coordinate set. The independent variable uses a numerical value in the range of zero to one, zero represents the first three-dimensional coordinate, and one represents the last three-dimensional coordinate. In fact, the value of t is continuous, it is an independent variable, and its physical meaning is that assuming the length of the working trajectory is one (100%), the three-dimensional coordinates at each position are what they are, and each position is controlled by t. It is also feasible to understand it as a percentage of the working trajectory.

[0110] Figure 5 The fourth sub-flow chart of the AI-based visual perception method includes the steps of inserting the working track into the motion area, displaying the motion area containing the working track, and receiving the supplementary points input by the user:

[0111] Step S401: inserting the working trajectory into the motion area;

[0112] Step S402: Extending the working trajectory based on the robot arm model;

[0113] Step S403: Eliminate the extended area in the motion area as an optional area;

[0114] Step S404: displaying the optional area and receiving the supplementary points input by the user.

[0115] The above content provides a specific interactive solution, which inserts the work trajectory into the motion area. During the insertion process, the work trajectory needs to be extended. The extension method is to use the work point as the calibration point and calibrate the volume occupied by the robotic arm in the motion area. This is equivalent to expanding a point into a robotic arm model. Each point on the work trajectory expands the model once, and the area that the robotic arm body can occupy in the motion area can be obtained. The area after extension is eliminated in the motion area, and the remaining area is the area where the robotic arm will definitely not exist. It is used as an optional area to receive additional points input by the user. The additional points indicate the location where additional workstations can be added.

[0116] It is worth mentioning that in the process of expanding the working point into a robot model, it is necessary to obtain the tangent of the working point at the working trajectory, and then use the tangent as a reference to expand the robot model. The robot model itself is very complex, and proportional expansion will be very complicated. Therefore, the present application will obtain the base position of the robot (a component at a fixed position), and determine a regular area with the same size as the robot according to the base position and the tangent, such as a set of some rectangles, and use it directly as the robot model, thereby simplifying the expansion process; in addition, there is another solution, that is, while taking an image of the robot and identifying the working point, the robot model can be identified based on the AI ​​module (the accuracy requirement is not high), and a corresponding relationship between it and the working point can be established. At this time, when expansion is required, the robot model can be directly read and inserted into the corresponding position; since the image recognition process of the present application is completed by the AI ​​module, both the identification of the working point and the identification of the robot model are not complicated, and only some samples need to be established in advance to obtain a localized model that is sufficient for the recognition work.

[0117] As an example of the technical solution of the present invention, the method further includes:

[0118] The motion instruction table is used as a sample set to train a neural network model; the input of the neural network model is the coordinates, and the output is the instruction;

[0119] After the neural network model is trained, it receives supplementary points and outputs instructions based on the neural network model.

[0120] The motion instruction table is a mapping relationship between coordinates and instruction groups. The number of elements in the motion instruction table is limited. The greater the detection density, the more elements in the motion instruction table, and the more computing resources required (there is a combination process, and the increase is very large). For this, the present application uses the motion instruction table as a sample set to train a neural network model as a mapping relationship between coordinates and the motion instruction table. At this time, after the user enters the coordinates, there is no need to match them in the motion instruction table to get the corresponding high-precision instructions. The user can then make fine adjustments to get the most accurate instructions.

[0121] It should be noted that when the robotic arm is being developed, since the position of the working point is related to the transmission pair, the transmission pair itself will undergo kinematic analysis. Therefore, there is indeed a mapping relationship between the position of the working point and each instruction, and this mapping relationship can be calculated by the R&D personnel. However, when there are more transmission pairs, the mechanical analysis and kinematic analysis processes involved are extremely numerous. Therefore, the neural network model training process provided in this application can eliminate the extremely complex analysis process and thereby improve efficiency.

[0122] Furthermore, when the actuator of the robot arm is replaced, the working point will also change accordingly. If mechanical analysis and kinematic analysis are used to determine the most accurate mapping relationship, it will consume a lot of time of professionals. However, by using the above-mentioned method of training the neural network model, the machine can quickly obtain a more accurate mapping relationship. Before the reference instructions are put into use, professionals only need to make fine adjustments, which is very time-saving and labor-saving.

[0123] Figure 6 : is a structural block diagram of a visual perception system based on AI. In an embodiment of the present invention, a visual perception system based on AI, the system 10 includes:

[0124] The instruction table construction module 11 is used to obtain the motion boundary of the robot arm, determine the motion area according to the motion boundary, and simultaneously construct a motion instruction table; the motion instruction table includes coordinate items and instruction items;

[0125] A working point positioning module 12 is used to obtain a robotic arm image containing a time tag in real time in three mutually perpendicular directions, identify the robotic arm image, and locate the working point;

[0126] A work trajectory generation module 13 is used to sort the work points according to the time tags to obtain a work trajectory;

[0127] A supplementary point receiving module 14 is used to insert the working track into the motion area, display the motion area containing the working track, and receive supplementary points input by the user;

[0128] The instruction simulation module 15 is used to query instructions in the motion instruction table based on the supplementary point as reference instructions.

[0129] Furthermore, the instruction table construction module 11 includes:

[0130] A coordinate system building unit, used for building a three-dimensional Cartesian coordinate system based on a corner point of a fixed seat of the robot arm;

[0131] The transmission pair positioning unit is used to position the transmission pair in the robot arm and determine the motion range of each transmission pair;

[0132] A motion point selection unit, used to select motion points within the motion range of each transmission pair according to a preset detection density, to obtain a motion point set of each transmission pair;

[0133] A motion point combination unit, used for selecting and only selecting one motion point from the motion point set of each transmission pair for combination;

[0134] An instruction group determination unit is used to determine the working point coordinates of each combination in the simulation software, and to determine the instruction group based on the motion point synchronization; wherein the value of each element in the instruction group is a control instruction, and the serial number of each element corresponds to the transmission pair one by one;

[0135] The area determination unit is used for counting the coordinates of all working points and determining the motion area;

[0136] The data statistics unit is used to count the coordinates of all combined working points and the corresponding instruction groups to obtain a motion instruction table.

[0137] Specifically, the working point positioning module 12 includes:

[0138] An image acquisition unit, used to use the x-axis, y-axis and z-axis as three shooting directions to acquire a robotic arm image containing a time tag of the robotic arm in real time;

[0139] An image recognition unit, used to recognize the image of the robot arm based on the AI ​​module, locate the corner point as the origin in the image, and simultaneously determine the three two-dimensional coordinates of the working point; the working point is the center position of the actuator;

[0140] A coordinate registration unit is used to register three two-dimensional coordinates to obtain the three-dimensional coordinates of the working point;

[0141] The time tag insertion unit is used to insert the time tag of the robot arm image into the three-dimensional coordinates.

[0142] Furthermore, the work trajectory generation module 13 includes:

[0143] A three-dimensional coordinate reading unit, used for reading the three-dimensional coordinates containing a time tag;

[0144] A three-dimensional coordinate sorting unit, used for sorting the three-dimensional coordinates according to the time sequence to obtain a three-dimensional coordinate set;

[0145] Fitting the working trajectory based on the three-dimensional coordinate set;

[0146] The fitting process is:

[0147]

[0148] Where B(t) is the parameterized form of the working trajectory, B i,n represents the Bessel basis function, is the binomial coefficient, P irepresents the i-th three-dimensional coordinate in the three-dimensional coordinate set, and the total number of three-dimensional coordinates in the three-dimensional coordinate set is set to (n+1); where t is an independent variable, when t is zero, it represents the first three-dimensional coordinate in the working trajectory, and when t is one, it represents the last three-dimensional coordinate in the working trajectory.

[0149] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A visual perception method based on AI, characterized in that: The method comprises: Acquire the motion boundary of the robot arm, determine the motion area according to the motion boundary, and simultaneously construct a motion instruction table; the motion instruction table includes coordinate items and instruction items; Acquire the robot arm images with time tags in real time in three mutually perpendicular directions, identify the robot arm images, and locate the working point; Sort the work points according to the time labels to obtain the work trajectory; Inserting the working track into the motion area, displaying the motion area containing the working track, and receiving a supplementary point input by a user; Based on the supplementary point, an instruction is searched in the motion instruction table as a reference instruction.

2. The AI-based visual perception method according to claim 1, characterized in that: The steps of obtaining the motion boundary of the robot arm, determining the motion area according to the motion boundary, and synchronously constructing the motion instruction table include: A three-dimensional Cartesian coordinate system is constructed based on a corner point of the fixed seat of the robot arm; Locate the transmission pairs in the robot arm and determine the range of motion of each transmission pair; Selecting motion points within the motion range of each transmission pair according to a preset detection density to obtain a motion point set of each transmission pair; Select and only select one motion point from the set of motion points of each transmission pair for combination; Determine the coordinates of the working points of each combination in the simulation software, and determine the instruction group based on the motion point synchronization; wherein the value of each element in the instruction group is a control instruction, and the serial number of each element corresponds to the transmission pair one by one; Count the coordinates of all working points and determine the motion area; Count all the combined working point coordinates and corresponding instruction groups to obtain the motion instruction table.

3. The AI-based visual perception method according to claim 1, characterized in that: The steps of acquiring the robot arm images containing time tags in real time in three mutually perpendicular directions, identifying the robot arm images, and locating the working point include: The x-axis, y-axis and z-axis are used as three shooting directions to obtain the robot arm images with time tags in real time; Based on the AI ​​module, the robot arm image is recognized, the corner point as the origin is located in the image, and the three two-dimensional coordinates of the working point are simultaneously determined; the working point is the center position of the actuator; The three 2D coordinates are registered to obtain the 3D coordinates of the working point; Insert time tags of robot images into 3D coordinates.

4. The AI-based visual perception method according to claim 3, characterized in that: The step of sorting the working points according to the time tags to obtain the working trajectory includes: Read the three-dimensional coordinates with time tags; Sort the three-dimensional coordinates according to the time sequence to obtain a three-dimensional coordinate set; Fitting the working trajectory based on the three-dimensional coordinate set; The fitting process is: Where B(t) is the parameterized form of the working trajectory, B i,n represents the Bessel basis function, is the binomial coefficient, P i represents the i-th three-dimensional coordinate in the three-dimensional coordinate set, and the total number of three-dimensional coordinates in the three-dimensional coordinate set is set to (n+1); where t is an independent variable, when t is zero, it represents the first three-dimensional coordinate in the working trajectory, and when t is one, it represents the last three-dimensional coordinate in the working trajectory.

5. The AI-based visual perception method according to claim 1, characterized in that: The steps of inserting the working track into the motion area, displaying the motion area containing the working track, and receiving the supplementary points input by the user include: inserting the working trajectory into the motion area; Extend the working trajectory based on the robot arm model; Eliminate the extended area in the motion area as an optional area; Displays an optional area and receives additional points entered by the user.

6. The AI-based visual perception method according to claim 1, characterized in that: The method further comprises: The motion instruction table is used as a sample set to train a neural network model; the input of the neural network model is the coordinates, and the output is the instruction; After the neural network model is trained, it receives supplementary points and outputs instructions based on the neural network model.

7. An AI-based visual perception system, characterized in that: The system comprises: An instruction table construction module is used to obtain the motion boundary of the robot arm, determine the motion area according to the motion boundary, and simultaneously construct a motion instruction table; the motion instruction table includes coordinate items and instruction items; A working point positioning module is used to obtain the robot arm images with time tags in real time in three mutually perpendicular directions, identify the robot arm images, and locate the working point; A work trajectory generation module is used to sort the work points according to the time tags to obtain the work trajectory; A supplementary point receiving module, used for inserting the working track into the motion area, displaying the motion area containing the working track, and receiving supplementary points input by the user; The instruction simulation module is used to query instructions in the motion instruction table based on the supplementary point as reference instructions.

8. The AI-based visual perception system according to claim 7, characterized in that: The instruction table building module includes: A coordinate system building unit, used for building a three-dimensional Cartesian coordinate system based on a certain corner point of a fixed seat of the robot arm; The transmission pair positioning unit is used to position the transmission pair in the robot arm and determine the motion range of each transmission pair; A motion point selection unit, used to select motion points within the motion range of each transmission pair according to a preset detection density, to obtain a motion point set of each transmission pair; A motion point combination unit, used for selecting and only selecting one motion point from the motion point set of each transmission pair for combination; An instruction group determination unit is used to determine the working point coordinates of each combination in the simulation software, and to determine the instruction group based on the motion point synchronization; wherein the value of each element in the instruction group is a control instruction, and the serial number of each element corresponds to the transmission pair one by one; The area determination unit is used for counting the coordinates of all working points and determining the motion area; The data statistics unit is used to count the coordinates of all combined working points and the corresponding instruction groups to obtain a motion instruction table.

9. The AI-based visual perception system according to claim 7, characterized in that: The working point positioning module comprises: An image acquisition unit, used to use the x-axis, y-axis and z-axis as three shooting directions to acquire a robotic arm image containing a time tag of the robotic arm in real time; An image recognition unit, used to recognize the image of the robot arm based on the AI ​​module, locate the corner point as the origin in the image, and simultaneously determine the three two-dimensional coordinates of the working point; the working point is the center position of the actuator; A coordinate registration unit is used to register three two-dimensional coordinates to obtain the three-dimensional coordinates of the working point; The time tag insertion unit is used to insert the time tag of the robot arm image into the three-dimensional coordinates.

10. The AI-based visual perception system according to claim 9, characterized in that: The work trajectory generation module includes: A three-dimensional coordinate reading unit, used for reading the three-dimensional coordinates containing a time tag; A three-dimensional coordinate sorting unit, used for sorting the three-dimensional coordinates according to the time sequence to obtain a three-dimensional coordinate set; Fitting the working trajectory based on the three-dimensional coordinate set; The fitting process is: Where B(t) is the parameterized form of the working trajectory, B i,n represents the Bessel basis function, is the binomial coefficient, P i represents the i-th three-dimensional coordinate in the three-dimensional coordinate set, and the total number of three-dimensional coordinates in the three-dimensional coordinate set is set to (n+1); where t is an independent variable, when t is zero, it represents the first three-dimensional coordinate in the working trajectory, and when t is one, it represents the last three-dimensional coordinate in the working trajectory.

Images