A control method for a mobile workstation and related equipment

Abstract

The embodiment of the present application provides a control method for a mobile workstation and related equipment for improving the intelligence level of the mobile workstation, including: obtaining a map and determining a target work position on the map; receiving a first image sent by a first image acquisition device and extracting a first environmental mark in the first image; analyzing the first environmental mark to obtain first mark information; determining a first position of the mobile workstation on the map based on the first mark information; performing path planning to obtain a first target path from the first position to the target work position; and controlling a driving device to drive the mobile workstation to move along the first target path. In the embodiment of the present application, the intelligence level is greatly improved. The computer software capabilities are utilized to the greatest extent, the personnel participation cost is reduced, and the operation accuracy and reliability are improved. It is particularly suitable for automated operations based on mobile workstations or robots in biological laboratories.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of mobile workstations, and more particularly to a control method for a mobile workstation and related equipment. Background Art

[0002] Currently, much of the navigation for mobile workstations or robotic systems is instructional. A laboratory map is first drawn and given to the mobile workstation or robot, with the mobile workstation positioned at a certain initial position on the map. The path from the initial position to the target work location is then calculated, and the robot is instructed to move to the work location along this route. Once the mobile workstation is in place, the robotic arm is activated and, following a pre-defined operational trajectory, moves to a designated spatial position to perform the operation. During this operation, most tasks are pre-specified or require a pre-demonstration and recording of the relevant trajectory. The system's intelligence level is severely low, and errors cannot be corrected in real time. When the accumulated movement error is large, the robotic arm may not be accurately positioned. Furthermore, the mobile workstation cannot interact with the surrounding environment in real time, making it difficult to ensure efficient completion of operational tasks. Summary of the Invention

[0003] The embodiments of the present application provide a control method for a mobile workstation and related devices thereof, for improving the intelligence level of the mobile workstation.

[0004] A first aspect of an embodiment of the present application provides a method for controlling a mobile workstation, including:

[0005] Obtain a map and determine the target work location on the map;

[0006] receiving a first image sent by a first image acquisition device, and extracting a first environment mark in the first image;

[0007] Analyze the first environmental marker to obtain first marker information;

[0008] determining a first position of the mobile workstation on the map according to the first marking information;

[0009] Performing path planning to obtain a first target path from the first position to the target working position;

[0010] The driving device is controlled to drive the mobile workstation to move along the first target path.

[0011] In one implementation of the first aspect of the embodiments of the present application, after controlling the driving device to drive the mobile workstation to move along the first target path, the control method further includes:

[0012] receiving a second image sent by a second image acquisition device, and extracting a second environmental marker from the second image;

[0013] Analyzing the second environmental marker to obtain second marker information;

[0014] determining a second position of the mobile workstation on the map according to the second marking information;

[0015] Performing path planning to obtain a second target path from the second position to the target working position;

[0016] The driving device is controlled to drive the mobile workstation to move along the second target path.

[0017] In one implementation of the first aspect of the embodiments of the present application, after controlling the driving device to drive the mobile workstation to move along the first target path, the method further includes:

[0018] receiving a third image sent by a third image acquisition device, and extracting an object identifier from the third image;

[0019] using the object identifier to enhance the signal characteristics and segment the image region to determine the characteristic points of the third image;

[0020] Determine the three-dimensional structure of the operation object based on the feature points;

[0021] Determine the three-dimensional coordinates of the operation object based on the feature points;

[0022] According to the three-dimensional structure and three-dimensional coordinates, the robotic arm is controlled to operate the object.

[0023] In an implementation of the first aspect of the embodiments of the present application, the third image acquisition device includes a left camera and a right camera, and the third image includes a left image and a right image;

[0024] Receiving a third image sent by a third image acquisition device, extracting an object identifier in the third image; using the object identifier to enhance signal features and segment the image area, and determining feature points of the third image, specifically including:

[0025] Receive a left image sent by a left camera and a right image sent by a right camera, and extract object identifiers from the left image and the right image;

[0026] Object identification is used to enhance signal features and segment image regions to determine feature points of the left and right images.

[0027] In an implementation of the first aspect of the embodiments of the present application, the control method of the mobile workstation further includes:

[0028] Control the laser radar to scan the environment and obtain the relative position relationship between the mobile workstation and obstacles or operating objects;

[0029] Perform path planning, including:

[0030] Path planning is performed based on relative position relationships.

[0031] In one implementation of the first aspect of the embodiments of the present application, path planning is performed based on the relative position relationship, specifically including:

[0032] According to the relative position relationship between the mobile workstation and the obstacle, determine whether the obstacle is moving relative to the environment;

[0033] If the obstacle moves relative to the environment, determine whether the obstacle will collide with the mobile workstation:

[0034] If the obstacle is about to collide with the mobile workstation, the driving device is controlled to slow down or stop the mobile workstation.

[0035] In an implementation of the first aspect of the embodiment of the present application, before determining the target work location on the map, the control method further includes: receiving a task sent by the server;

[0036] Determining a target work location on a map specifically includes: determining the target work location on a map according to a task.

[0037] A second aspect of an embodiment of the present application provides a control device for a mobile workstation, including:

[0038] An acquisition module is used to acquire a map and determine the target work location on the map;

[0039] A receiving module, configured to receive a first image sent by a first image acquisition device, and extract a first environment mark from the first image;

[0040] An analysis module, configured to analyze the first environmental marker to obtain first marker information;

[0041] a determining module, configured to determine a first position of the mobile workstation on the map according to the first marking information;

[0042] A planning module, configured to perform path planning to obtain a first target path from a first position to a target working position;

[0043] The control module is used to control the driving device to drive the mobile workstation to move along the first target path.

[0044] In one implementation of the second aspect of the embodiments of the present application,

[0045] The receiving module is further configured to receive a second image sent by the second image acquisition device and extract a second environmental marker from the second image;

[0046] The analysis module is further used to analyze the second environmental mark to obtain second mark information;

[0047] The determination module is further configured to determine a second position of the mobile workstation on the map according to the second marking information;

[0048] The planning module is further used to perform path planning to obtain a second target path from the second position to the target working position;

[0049] The control module is also used to control the driving device to drive the mobile workstation to move along the second target path.

[0050] In one implementation of the second aspect of the embodiments of the present application,

[0051] The receiving module is further configured to receive a third image sent by the third image acquisition device and extract an object identifier from the third image;

[0052] The determination module is further configured to utilize the object identifier to enhance the signal characteristics and segment the image region to determine the characteristic points of the third image;

[0053] The determination module is further used to determine the three-dimensional structure of the operation object based on the feature points;

[0054] The determination module is further used to determine the three-dimensional coordinates of the operation object based on the feature points;

[0055] The control module is also used to control the robotic arm to operate the object according to the three-dimensional structure and three-dimensional coordinates.

[0056] In an implementation of the second aspect of the embodiments of the present application, the third image acquisition device includes a left camera and a right camera, and the third image includes a left image and a right image;

[0057] The receiving module is further used to receive the left image sent by the left camera and the right image sent by the right camera, and extract the object identifiers in the left image and the right image;

[0058] The determination module is further used to utilize the object identifier to enhance the signal features and segment the image area, and determine the feature points of the left image and the right image.

[0059] In one implementation of the second aspect of the embodiments of the present application,

[0060] The control module is also used to control the laser radar to scan the environment and obtain the relative position relationship between the mobile workstation and obstacles or operating objects;

[0061] The planning module is specifically used to plan paths based on relative position relationships.

[0062] In one implementation of the second aspect of the embodiments of the present application, the planning module includes:

[0063] The judgment submodule is used to judge whether the obstacle is moving relative to the environment based on the relative position relationship between the mobile workstation and the obstacle;

[0064] The judgment submodule is further configured to judge whether the obstacle will collide with the mobile workstation when the judgment submodule determines that the obstacle is moving relative to the environment:

[0065] The control submodule is used to control the driving device to slow down or stop the mobile workstation when the judgment submodule determines that the obstacle will collide with the mobile workstation.

[0066] In one implementation of the second aspect of the embodiments of the present application,

[0067] The receiving module is also used to receive tasks sent by the server;

[0068] The acquisition module is also used to determine the target work location on the map according to the task.

[0069] A third aspect of the embodiments of the present application provides a computer device, including:

[0070] CPU, memory, input and output interfaces, wired or wireless network interfaces, and power supply;

[0071] The memory is either transient or persistent storage;

[0072] The central processing unit is configured to communicate with the memory and execute instructions in the memory to perform the method of the first aspect.

[0073] A fourth aspect of an embodiment of the present application provides a computer-readable storage medium, which includes instructions. When the instructions are executed on a computer, the computer executes the method of the first aspect.

[0074] A fifth aspect of the embodiments of the present application provides a computer program product comprising instructions, which, when executed on a computer, causes the computer to execute the method of the first aspect.

[0075] The sixth aspect of an embodiment of the present application provides a chip system, which includes at least one processor and a communication interface, the communication interface and the at least one processor are interconnected through lines, and the at least one processor is used to run a computer program or instruction to execute the method of the first aspect.

[0076] A seventh aspect of an embodiment of the present application provides a mobile workstation, comprising: a robotic arm, a driving device, an image acquisition device, a laser radar, and the computer device of the third aspect.

[0077] In an embodiment of the present application, a first image transmitted by a first image acquisition device is received, a first environmental marker is extracted from the first image, the first environmental marker is analyzed to obtain first marker information, and a first position of the mobile workstation on a map is determined based on the first marker information. Significant improvements have been made to the navigation technology of the mobile workstation and the identification and spatial positioning technology of the experimental operation object, greatly enhancing the level of intelligence. This maximizes the use of computer software capabilities, reduces personnel participation costs, and improves operational accuracy and reliability. This system is particularly suitable for automated operations based on mobile workstations or robots in biological laboratories. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] Figure 1 This is a research report from the Center for Drug Development at Tufts University in the United States;

[0079] Figure 2 It is a laboratory of the control method of the mobile workstation of the embodiment of the present application;

[0080] Figure 3 This is a flow chart of a control method for a mobile workstation according to an embodiment of the present application;

[0081] Figure 4 is another flow chart of the control method of the mobile workstation according to an embodiment of the present application;

[0082] Figure 5 is a left image of an operation object of the control method of the mobile workstation according to an embodiment of the present application;

[0083] Figure 6 is a right image of an operation object of the control method of the mobile workstation according to an embodiment of the present application;

[0084] Figure 7 is a schematic diagram of a 3D structure of an operation object of a control method for a mobile workstation according to an embodiment of the present application;

[0085] Figure 8 2. It is a schematic diagram of the spatial positioning of the operation object of the control method of the mobile workstation according to an embodiment of the present application;

[0086] Figure 9 It is a schematic diagram of a control method of a mobile workstation according to an embodiment of the present application, in which a manipulator operates an operation object;

[0087] Figure 10 is another flow chart of the control method of the mobile workstation according to an embodiment of the present application;

[0088] Figure 11 This is a schematic diagram of navigation and operation of a mobile workstation in an indoor static environment in a control method of a mobile workstation according to an embodiment of the present application;

[0089] Figures 12 to 15This is a schematic diagram of a laser radar scanning environment for a control method of a mobile workstation according to an embodiment of the present application;

[0090] Figure 16 This is a schematic diagram of mobile workstation navigation and operation in an indoor dynamic environment in a control method of a mobile workstation according to an embodiment of the present application;

[0091] Figure 17 Schematic diagram of a mobile workstation laser radar scanning obstacle for a control method of a mobile workstation according to an embodiment of the present application;

[0092] Figure 18 is another flow chart of the control method of the mobile workstation according to an embodiment of the present application;

[0093] Figure 19 is a structural diagram of a control device of a mobile workstation according to an embodiment of the present application;

[0094] Figure 20 is a structural diagram of a computer device according to an embodiment of the present application;

[0095] 1. Work location; 2. Central server; 3. Inside the laboratory; 4. Company intranet; 5. Mobile workstation; 6. Operation object; 7. Laser line; 8. Obstacle; 9. Robotic arm. DETAILED DESCRIPTION

[0096] Biosynthesis and new drug discovery is a complex and multidisciplinary process, as Figure 1 Research reports from the Center for Drug Development at Tufts University in the United States show that it often takes a lot of time and money.

[0097] Biosynthesis and drug development involve several key steps: first, target discovery, typically proteins, enzymes, receptors, ion channels, and so on. Next, target confirmation is achieved through biochemical assays. Finally, optimized targets are screened. This involves arduous and repetitive labor and is prone to errors. Modern biological laboratories utilize automated and intelligent technologies to address these challenges. These technologies include mobile workstations, which enable mobile operation in indoor environments. Mobile workstations are also called intelligent mobile work platforms or robots.

[0098] In a biological laboratory environment, multiple mobile workstations are allowed to operate in parallel efficiently and complete their respective operational tasks according to the laboratory's predetermined work content.

[0099] Taking advantage of the lab's inherent ease of landmark placement, the placement of landmarks within the biological laboratory significantly enhances and improves the positioning and navigation capabilities of mobile workstations, further enhancing their intelligence. Landmarks are placed on the sides of various workstations, walls, columns, and other environmental features for easy visual identification. By recognizing these landmarks, the mobile workstation achieves autonomous navigation and positioning, enabling real-time obstacle avoidance and adaptive adaptation to the environment.

[0100] Taking advantage of the fact that markers are easily identified and located on the object being manipulated, object markers are placed on the object. The mobile workstation calculates the spatial trajectory of the robotic arm in real time based on the object markers, guiding the robotic arm to complete the manipulation task.

[0101] Landmarks include environmental markers and object markers. Landmarks can be physical materials that can be visually identified, such as biological agents, labels, or colors.

[0102] By placing markers on the lab environment and operational objects, and employing highly intelligent, adaptive navigation and target recognition technology based on marker recognition, the system can adaptively avoid obstacles in real time, autonomously plan routes, locate target work locations, identify operational objects, and plan the robotic arm's spatial path to complete the required operational tasks within a complex and variable indoor environment. This system also allows multiple mobile work platforms to collaborate within a single lab environment, efficiently and seamlessly completing the required tasks.

[0103] In one implementation, the laboratory layout is as follows Figure 2 As shown, the laboratory includes multiple workstations, multiple mobile workstations, and a central server. The central server is connected to computer terminals via the cloud. Typically, the central server connects to computer terminals within the company's intranet, but it can also connect to computer terminals on wide area networks such as the World Wide Web and the Internet. The central server communicates with the mobile workstations via wireless networks or Bluetooth. This laboratory can be called an intelligent automated laboratory. The mobile workstations and central server constitute a mobile platform system. The central server can also be simply referred to as the server.

[0104] exist Figure 2In the example, mobile workstation A has object X at its target work location, and mobile workstation B has object Y at its target work location. A central server receives work tasks from computer terminals and assigns them to mobile workstations A and B. Mobile workstation A autonomously navigates to the vicinity of object X, identifies the spatial location of object X, and guides its robotic arm to operate on it. Mobile workstation B autonomously navigates to the vicinity of object Y, identifies the spatial location of object Y, and guides its robotic arm to operate on it.

[0105] The control method of the mobile workstation provided in the embodiment of the present application has the following general process:

[0106] The server in the laboratory performs task distribution.

[0107] The mobile workstation receives tasks issued by the server and executes planning and scheduling according to the task requirements. The mobile workstation uses environmental markers to complete positioning and autonomous navigation.

[0108] The mobile workstation operates on the object, identifies the object's markings, and autonomously maneuvers the robotic arm based on task requirements to perform the operation.

[0109] After the mobile workstation completes a task, it accepts another task and the cycle repeats.

[0110] The control method of the mobile workstation provided in the embodiment of the present application is as follows: Figure 3 As shown, the specific process is:

[0111] S1. The laboratory central server receives the test task from the company's intranet and subdivides the test task into executable subtasks.

[0112] S2. The mobile workstation retrieves and queries the subtasks it can complete from the central server and accepts the subtasks.

[0113] S3. The mobile workstation determines the target work location according to the requirements of the subtask.

[0114] S4. The mobile workstation identifies and locates environmental markers, and moves to the target work location after planning the path and avoiding obstacles.

[0115] S5. At the target working position, the mobile workstation identifies the object mark on the operation object, determines whether the operation object meets the requirements of the subtask, and calculates the position of the operation object.

[0116] S6: The mobile workstation calculates the robot's spatial trajectory, controls it to reach the designated spatial position, and then operates the object. The entire process is unattended and fully automated.

[0117] Furthermore, information from mobile workstations is fed back to the central server in real time, which then notifies the company's intranet staff. If an error occurs, the central server issues a warning or pause message to notify the intranet staff.

[0118] S4 and S5 of the above-mentioned specific process are key to the embodiments of this application and are described in detail below. S4 and S5 involve important processes that are highly automated, so that when the laboratory environment is adjusted or objects are moved, there is no need to change the software and hardware systems, which can meet the needs of long-term and efficient operation of the laboratory.

[0119] Existing technology often uses a teach-in approach for mobile workstations or robots in indoor environments. This approach pre-determines a map, the initial position of the mobile workstation, and the target work location. Based on this map and these two locations, a search algorithm is then executed to calculate a planned path. This data is then transmitted to the chassis, where a motor drives the wheels to reach the target work location. However, this approach suffers from a low level of intelligence and an inability to adapt to environmental changes. If facilities are added or moved indoors, or if there are moving objects, the mobile workstation or robot cannot detect this, let alone avoid obstacles in real time. The need to set an initial position for the mobile workstation or robot requires external intervention at each startup. This is primarily due to the mobile workstation or robot being unable to identify its current location, necessitating manual intervention.

[0120] like Figure 4 As shown, the specific implementation of S4 can be:

[0121] S401: The first image acquisition device of the mobile workstation identifies and locates environmental markers to confirm the coordinate position of the robot.

[0122] S402. The laser radar of the mobile workstation scans the surrounding environment, calculates a real-time map, and realizes the navigation and positioning functions of the robot.

[0123] S403: Laser scanning can detect obstacles in real time. The scan data can be used to update the map in a timely manner. The shortest path algorithm is used to obtain the first target path. The target path can also be called movement path data.

[0124] S404: Control the driving device of the chassis according to the first target path to drive the wheels to move.

[0125] S405: Move to the target working position.

[0126] To achieve real-time obstacle avoidance, the following methods can be used:

[0127] LiDAR is used to measure distance and locate obstacles. Several environmental markers with known locations are identified and the mobile workstation's position is calculated based on their positions and the distances between them and the mobile platform. The obstacle's spatial coordinates are determined based on its direction and distance relative to the mobile workstation, as well as the mobile workstation's position.

[0128] The speed and direction of obstacles are calculated using lidar.

[0129] Consider the constraints between the moving speed of the mobile workstation, the moving speed of the obstacles, and the code operation time.

[0130] v1: The speed of the mobile workstation; v2: The speed of the obstacle; t: The code execution time. The code execution time is determined by the code calculation speed. The value of the code execution time can be obtained through simulation and becomes a constant.

[0131] The constraint relationship is summarized as follows: the obstacle will travel from point A to point B in n seconds in its direction of movement (in the same direction, opposite direction, or at an angle relative to the mobile workstation), and the actual movement time of the workstation during this period is nt seconds;

[0132] When planning the shortest path for a mobile workstation, it is necessary to consider that the obstacle has moved for t seconds at the moment when the code operation is completed and "passed to the chassis drive".

[0133] After a mobile workstation or robot navigates to its target work location, a robotic arm is needed to begin performing the manipulation task. Current technology typically pre-fixes the positions of the robotic arm and the object being manipulated. In this case, a spatial trajectory is calculated, and then the robotic arm is moved to the desired location according to the trajectory. This technology cannot identify and determine the spatial position of the object being manipulated in real time. Slight changes in the environment or relative position can lead to errors, making the operation inaccurate. A robotic arm can also be called a manipulator or a mechanical gripper.

[0134] In order to solve the above problems, there is a certain degree of difference in the coordinates of the target working position each time the mobile workstation reaches it (caused by various errors), and it is necessary to re-precisely locate the position of the robot by identifying environmental markers.

[0135] The object identifier of the manipulation object is identified and spatially located. The robot arm is guided in spatial direction based on the trajectory data. The robot arm reaches the designated spatial position and operates the manipulation object. This method of real-time recognition and trajectory calculation, mobilizing the robot arm to complete the operation, offers a higher level of intelligence. The manipulation object can also be referred to as the experimental object or test object.

[0136] In one implementation, a pair of industrial cameras mounted on a robotic arm constitute a stereoscopic vision acquisition device. The stereoscopic vision acquisition device detects the object being operated and calculates the spatial position of the operated object. The algorithm flow is as follows:

[0137] like Figure 5 and Figure 6 As shown in FIG, a stereo camera obtains a left image and a right image and performs rectification. The left image and the right image can be referred to as the left and right images for short.

[0138] like Figure 7 As shown, according to the image feature points, the feature points of the left and right images are calculated to determine the 3D structure, and the 3D structure graph is calculated to obtain the coordinates of the space point.

[0139] like Figure 8 As shown, the operation object is spatially positioned to determine the position coordinates and size of the operation object in space.

[0140] like Figure 9 As shown, the robotic arm moves along the spatially planned trajectory to grasp the object.

[0141] The stereo vision acquisition device can be a stereo camera, which can be divided into a left camera and a right camera.

[0142] like Figure 10 As shown, the specific implementation of S5 can be:

[0143] S501, receiving a left image sent by a left camera and a right image sent by a right camera, and extracting object identifiers in the left image and the right image;

[0144] S502: Utilize the object identifier to enhance the signal feature and segment the image area, and determine the feature points of the left image and the right image.

[0145] S503, determining the three-dimensional structure of the operation object according to the feature points;

[0146] S504, determining the three-dimensional coordinates of the operation object according to the feature points;

[0147] S505: Control the robotic arm to operate the operation object according to the three-dimensional structure and the three-dimensional coordinates.

[0148] The task allocation management system for the joint laboratory in the embodiment of the present application can realize the automation of the entire workflow, greatly reduce the errors caused by complex and repetitive labor, and greatly improve production efficiency and reliability.

[0149] The laboratory environment can be divided into static and dynamic environments. In a static environment, there are no moving obstacles, while in a dynamic environment, there are moving obstacles. The following examples illustrate each.

[0150] Navigation and operation of mobile workstations in indoor static environments.

[0151] exist Figure 11 In the laboratory environment shown, the mobile workstation A in the lower left corner receives tasks distributed by the central server through wireless communication and goes to the target work location Ta to operate the operation object.

[0152] First, it identifies and locates environmental markers to drive the wheels to the target location. The actual movement path is shown in red. Throughout this process, it encountered no obstacles. Mobile Station A's LiDAR can quickly scan the environment, updating the mobile station's position on the map in real time. The LiDAR scan of the environment is shown below:

[0153] like Figures 12 to 15 This is a diagram of a mobile workstation using LiDAR. The squares in the diagram represent the mobile workstation, the gray area represents the laser line, the white area represents a blank area, and the black area represents areas the laser line cannot reach. The jagged shape of the laser line occurs when it encounters an obstacle.

[0154] After the mobile workstation A reaches the target working position, the stereo vision camera begins to identify and detect the operation object and calculate the spatial position of the operation object.

[0155] Navigation and operation of mobile workstations in indoor dynamic environments.

[0156] exist Figure 16 In the lab environment shown, mobile workstations A (bottom left) and B (top left) receive tasks from a central server via a wireless network to travel to their respective destination locations, Ta and Tb, to perform operations. First, they identify and locate environmental markers, driving their wheels to move.

[0157] Mobile workstation B is at position P1 and detects that A is on its current path and a collision may occur. Mobile workstation B slows down and stops for a while. After A passes, it replans the trajectory and continues to move to the target work position Tb.

[0158] The mobile workstation uses LiDAR to scan for obstacles such as Figure 17 As shown in Figure 2, the mobile workstation detects the obstacle and updates the positions of the mobile workstation and the obstacle on the map in real time.

[0159] After mobile workstation A and mobile workstation B respectively arrive at the target working positions, the stereo vision camera begins to identify and detect the operation object target and calculate the spatial position of the operation object.

[0160] like Figure 18 As shown, an embodiment of the present application provides a control method for a mobile workstation, including:

[0161] 101. Obtain a map and determine the target work location on the map;

[0162] 102. Receive a first image sent by a first image acquisition device, and extract a first environment mark from the first image;

[0163] 103. Analyze the first environmental marker to obtain first marker information;

[0164] 104. Determine a first position of the mobile workstation on the map according to the first marking information;

[0165] 105. Perform path planning to obtain a first target path from the first position to the target working position;

[0166] 106. Control the driving device to drive the mobile workstation to move along the first target path.

[0167] In an embodiment of the present application, a first image transmitted by a first image acquisition device is received, a first environmental marker is extracted from the first image, the first environmental marker is analyzed to obtain first marker information, and a first position of the mobile workstation on a map is determined based on the first marker information. Significant improvements have been made to the navigation technology of the mobile workstation and the identification and spatial positioning technology of the experimental operation object, greatly enhancing the level of intelligence. This maximizes the use of computer software capabilities, reduces personnel participation costs, and improves operational accuracy and reliability. This system is particularly suitable for automated operations based on mobile workstations or robots in biological laboratories.

[0168] The first marking information includes the coordinates of the first environment marking on the map and may also include the height of the first environment marking. The second marking information includes the coordinates of the second environment marking on the map and may also include the height of the second environment marking.

[0169] In a feasible solution, the first target path is a path with the shortest distance or a path with the shortest time.

[0170] In a feasible solution, after controlling the driving device to drive the mobile workstation to move along the first target path, the control method further includes:

[0171] receiving a second image sent by a second image acquisition device, and extracting a second environmental marker from the second image;

[0172] Analyzing the second environmental marker to obtain second marker information;

[0173] determining a second position of the mobile workstation on the map according to the second marking information;

[0174] Performing path planning to obtain a second target path from the second position to the target working position;

[0175] The driving device is controlled to drive the mobile workstation to move along the second target path.

[0176] The first image acquisition device and the second image acquisition device can be the same image acquisition device or different image acquisition devices. The image acquisition device can be a camera, a video camera, etc.

[0177] In a feasible solution, after controlling the driving device to drive the mobile workstation to move along the first target path or the second target path, the method further includes:

[0178] receiving a third image sent by a third image acquisition device, and extracting an object identifier from the third image;

[0179] using the object identifier to enhance the signal characteristics and segment the image region to determine the characteristic points of the third image;

[0180] Determine the three-dimensional structure of the operation object based on the feature points;

[0181] Determine the three-dimensional coordinates of the operation object based on the feature points;

[0182] According to the three-dimensional structure and three-dimensional coordinates, the robotic arm is controlled to operate the object.

[0183] In a feasible solution, the third image acquisition device includes a left camera and a right camera, and the third image includes a left image and a right image;

[0184] In one possible solution, the method further includes:

[0185] Control the laser radar to scan the environment and obtain the relative position relationship between the mobile workstation and obstacles or operating objects;

[0186] Perform path planning, including:

[0187] Path planning is performed based on relative position relationships.

[0188] In one feasible solution, path planning is performed based on relative position relationships, specifically including:

[0189] According to the relative position relationship between the mobile workstation and the obstacle, determine whether the obstacle is moving relative to the environment;

[0190] If the obstacle moves relative to the environment, determine whether the obstacle will collide with the mobile workstation:

[0191] If the obstacle is about to collide with the mobile workstation, the driving device is controlled to slow down or stop the mobile workstation.

[0192] The environment refers to the laboratory where the mobile workstation is located.

[0193] In a feasible solution, before determining the target work location on the map, the control method further includes: receiving a task sent by the server;

[0194] Determining a target work location on a map specifically includes: determining the target work location on a map according to a task.

[0195] like Figure 19 As shown, an embodiment of the present application provides a control device for a mobile workstation, comprising:

[0196] An acquisition module 201 is used to acquire a map and determine a target work location on the map;

[0197] The receiving module 202 is configured to receive a first image sent by a first image acquisition device and extract a first environment marker from the first image;

[0198] An analysis module 203 is configured to analyze the first environmental marker to obtain first marker information;

[0199] A determination module 204 is configured to determine a first position of the mobile workstation on the map according to the first marking information;

[0200] A planning module 205 is used to perform path planning to obtain a first target path from the first position to the target working position;

[0201] The control module 206 is configured to control the driving device to drive the mobile workstation to move along the first target path.

[0202] In one implementation,

[0203] The receiving module 202 is further configured to receive a second image sent by a second image acquisition device and extract a second environmental marker from the second image;

[0204] The analysis module 203 is further configured to analyze the second environmental marker to obtain second marker information;

[0205] The determination module 204 is further configured to determine a second position of the mobile workstation on the map according to the second marking information;

[0206] The planning module 205 is further configured to perform path planning to obtain a second target path from the second position to the target working position;

[0207] The control module 206 is further configured to control the driving device to drive the mobile workstation to move along the second target path.

[0208] In one implementation,

[0209] The receiving module 202 is further configured to receive a third image sent by a third image acquisition device and extract an object identifier from the third image;

[0210] The determination module 204 is further configured to utilize the object identifier to enhance the signal feature and segment the image region to determine feature points of the third image;

[0211] The determination module 204 is further configured to determine the three-dimensional structure of the operation object based on the feature points;

[0212] The determination module 204 is further configured to determine the three-dimensional coordinates of the operation object based on the feature points;

[0213] The control module 206 is further configured to control the robot arm to operate the operation object according to the three-dimensional structure and the three-dimensional coordinates.

[0214] In one implementation, the third image acquisition device includes a left camera and a right camera, and the third image includes a left image and a right image;

[0215] The receiving module 202 is further configured to receive a left image sent by the left camera and a right image sent by the right camera, and extract object identifiers from the left image and the right image;

[0216] The determination module 204 is further configured to utilize the object identifier to enhance the signal feature and segment the image region, thereby determining feature points of the left image and the right image.

[0217] In one implementation,

[0218] The control module 206 is further used to control the laser radar to scan the environment and obtain the relative position relationship between the mobile workstation and the obstacle or operation object;

[0219] The planning module 205 is specifically used to perform path planning based on the relative position relationship.

[0220] In one implementation, the planning module 205 includes:

[0221] The judgment submodule is used to judge whether the obstacle is moving relative to the environment based on the relative position relationship between the mobile workstation and the obstacle;

[0222] The judgment submodule is further configured to judge whether the obstacle will collide with the mobile workstation when the judgment submodule determines that the obstacle is moving relative to the environment:

[0223] The control submodule is used to control the driving device to slow down or stop the mobile workstation when the judgment submodule determines that the obstacle will collide with the mobile workstation.

[0224] In one implementation,

[0225] The receiving module 202 is further used to receive tasks sent by the server;

[0226] The acquisition module 201 is further configured to determine a target work location on a map according to the task.

[0227] like Figure 20 As shown, the embodiment of the present application further provides a computer device 300, including:

[0228] CPU 301, memory 305, input / output interface 304, wired or wireless network interface 303 and power supply 302;

[0229] The memory 305 is a temporary storage memory or a permanent storage memory;

[0230] The CPU 301 is configured to communicate with the memory 305 and execute the instructions in the memory 305 to perform the following operations: Figures 2 to 18 The method in the embodiment shown.

[0231] The embodiment of the present application also provides a mobile workstation, including: a mechanical arm, a driving device, an image acquisition device, a laser radar and Figure 20 Computer equipment shown.

[0232] The embodiment of the present application further provides a computer-readable storage medium, which includes instructions. When the instructions are executed on a computer, the computer executes the following Figures 2 to 18 The method in the embodiment shown.

[0233] The present application also provides a computer program product comprising instructions, which, when executed on a computer, causes the computer to execute the following Figures 2 to 18 The method in the embodiment shown.

[0234] The embodiment of the present application also provides a chip system, which includes at least one processor and a communication interface, wherein the communication interface and the at least one processor are interconnected through a line, and the at least one processor is used to run a computer program or instruction to execute the following steps: Figures 2 to 18 The method in the embodiment shown.

[0235] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0236] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interface, device or unit, which can be electrical, mechanical or other forms.

[0237] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0238] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0239] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, read-only memory), random access memory (RAM, random access memory), disk or optical disk, and other media that can store program code.

Claims

1. A control method for a mobile workstation, characterized in that: include: obtaining a map, and determining a target work location on the map; receiving a first image sent by a first image acquisition device, and extracting a first environment mark in the first image; Analyzing the first environmental marker to obtain first marker information; determining a first position of the mobile workstation on the map according to the first marking information; Performing path planning to obtain a first target path from the first position to the target working position; Controlling the driving device to drive the mobile workstation to move along the first target path; After the control driving device drives the mobile workstation to move along the first target path, the method further includes: receiving a third image sent by a third image acquisition device, and extracting an object identifier from the third image; using the object identifier to enhance signal features and segment image regions to determine feature points of the third image; determining the three-dimensional structure of the operation object according to the feature points; Determining the three-dimensional coordinates of the operation object according to the feature points; According to the three-dimensional structure and the three-dimensional coordinates, the robot arm is controlled to operate the operation object.

2. The control method of the mobile workstation according to claim 1, characterized in that: After the control driving device drives the mobile workstation to move along the first target path, the control method further includes: receiving a second image sent by a second image acquisition device, and extracting a second environmental marker from the second image; Analyzing the second environmental mark to obtain second mark information; determining a second position of the mobile workstation on the map according to the second marking information; Performing path planning to obtain a second target path from the second position to the target working position; The driving device is controlled to drive the mobile workstation to move along the second target path.

3. The control method of the mobile workstation according to claim 1, characterized in that: The third image acquisition device includes a left camera and a right camera, and the third image includes a left image and a right image; receiving a third image sent by a third image acquisition device, and extracting an object identifier from the third image; Using the object identifier to enhance signal features and segment the image region to determine feature points of the third image specifically includes: receiving the left image sent by the left camera and the right image sent by the right camera, and extracting the object identifiers in the left image and the right image; The object identifier is used to enhance signal features and segment image regions, thereby determining feature points of the left image and the right image.

4. The control method of the mobile workstation according to claim 1, characterized in that: The method further comprises: Controlling the laser radar to scan the environment to obtain the relative position relationship between the mobile workstation and the obstacle or the operation object; The path planning specifically includes: Path planning is performed according to the relative position relationship.

5. The control method of the mobile workstation according to claim 4, characterized in that: The performing of path planning according to the relative position relationship specifically includes: determining whether the obstacle moves relative to the environment based on a relative positional relationship between the mobile workstation and the obstacle; If the obstacle moves relative to the environment, determine whether the obstacle will collide with the mobile workstation: If the obstacle is about to collide with the mobile workstation, the driving device is controlled to cause the mobile workstation to decelerate or stop.

6. The control method of the mobile workstation according to claim 1, characterized in that: Before determining the target working position on the map, the control method further includes: receiving a task sent by a server; The determining of the target working position on the map specifically includes: determining the target working position on the map according to the task.

7. A control device for a mobile workstation, characterized in that: include: An acquisition module, configured to acquire a map and determine a target working position on the map; A receiving module, configured to receive a first image sent by a first image acquisition device, and extract a first environment mark from the first image; An analysis module, configured to analyze the first environmental marker to obtain first marker information; a determining module, configured to determine a first position of the mobile workstation on the map according to the first marking information; A planning module, configured to perform path planning to obtain a first target path from the first position to the target working position; a control module, configured to control a driving device to drive the mobile workstation to move along the first target path; The receiving module is further configured to receive a third image sent by a third image acquisition device and extract an object identifier from the third image; The determination module is further configured to utilize the object identifier to enhance signal features and segment image regions to determine feature points of the third image; The determining module is further configured to determine the three-dimensional structure of the operation object based on the feature points; The determining module is further configured to determine the three-dimensional coordinates of the operation object based on the feature points; The control module is further used to control the robotic arm to operate the operation object according to the three-dimensional structure and the three-dimensional coordinates.

8. A computer device, characterized in that: include: CPU, memory, input and output interfaces, wired or wireless network interfaces, and power supply; The memory is either transient or persistent storage; The central processing unit is configured to communicate with the memory and execute instructions in the memory to perform the method according to any one of claims 1 to 6.

9. A mobile workstation, characterized in that: include: A robotic arm, a driving device, an image acquisition device, a laser radar, and a computer device as claimed in claim 8.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium comprises instructions, and when the instructions are executed on a computer, the computer is caused to perform the method according to any one of claims 1 to 6.

11. A computer program product comprising instructions, characterized in that When the computer program product is run on a computer, the computer is caused to perform the method according to any one of claims 1 to 6.

Images