A robot control method and system based on computer vision

Through the robot control method and system based on computer vision, using the Internet of Things and edge intelligent terminals to process environmental data in real time, the problems of high labor costs and limited sensor accuracy in robot mobile control are solved, the environment perception and adaptability are improved, and the environment is realized and adaptable. Task completion efficiency and safety.

CN118456439BActive Publication Date: 2024-11-08深圳辰迈机器人有限公司
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410735991.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-06-07
Publication Date
2024-11-08
Estimated Expiration
2044-06-07

AI Technical Summary

Technical Problem

The existing robot movement control methods have problems such as high labor costs and limited sensor detection accuracy, which leads to the robot frequently hitting obstacles or colliding with pedestrians, affecting personal safety.

Method used

Adopting a robot control method and system based on computer vision, through the central processor, vision module, positioning module, inertial measurement module, lidar module and communication module, combined with the Internet of Things server and edge smart terminal, environmental data is obtained and processed in real time and adjusted Behavior control model to optimize robot operation behavior.

Benefits of technology

It improves the robot's environmental perception range and adaptability, enhances task completion efficiency, saves resource expenditure, reduces operation costs, and optimizes task management and operation efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118456439B_ABST
    Figure CN118456439B_ABST
Patent Text Reader

Abstract

The present invention proposes a robot control method and system based on computer vision, the method comprising: the robot obtains location information, determines the Internet of Things server according to preset conditions, and sends the robot attribute information, visual data and surrounding environment data to the Internet of Things server; the Internet of Things server determines the behavior control model according to the robot attribute information; judges whether the robot's visual recognition ability meets the requirements according to the visual data and the environment; if it does, sends the behavior control model to control the robot behavior; otherwise, determines the visual reinforcement requirements, obtains the surrounding environment data from the edge intelligent terminal; adjusts the behavior control model according to the data and visual reinforcement requirements, and sends it to the robot; the robot operates autonomously according to the behavior control model. The scheme of the present invention can improve the robot's environmental perception, self-adaptation, task efficiency, learning and collaboration capabilities, save resources, improve scalability, optimize task management, improve operating efficiency, and reduce costs.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of robot technology, and in particular to a robot control method and system based on computer vision. Background Art

[0002] A robot is a mechanical device that automatically performs tasks. It can accept human instructions, execute preset programs, or act according to the principles of artificial intelligence technology. The task of robots is to assist or replace human work, and they play an important role in the fields of industry, medicine, agriculture, services, construction, and military. With the deepening understanding of the intelligence of robot technology, robot technology continues to penetrate into various fields of human activities. According to the characteristics of different fields, people have developed various special robots and intelligent robots with various perception, decision-making, action and interaction capabilities.

[0003] In practical applications, robots often need to perform mobile operations, such as food delivery robots and guide robots. There are currently two main methods for controlling robot movement. One is to rely on manual command, control the movement of the robot through a remote control device, and avoid obstacles when encountering obstacles, but the labor cost is high. The other is preset program control. The robot is equipped with sensors to detect surrounding obstacles. Once an obstacle is detected, the robot stops or avoids it. However, the sensor detection accuracy is limited, and it is impossible to fully understand the obstacle situation, causing the robot to frequently collide with obstacles or collide with pedestrians, affecting personal safety. Summary of the invention

[0004] The present invention is based on the above-mentioned problems and proposes a robot control method and system based on computer vision. Through the scheme of the present invention, the robot's environmental perception range and level can be improved, the robot's adaptive ability can be enhanced, the task completion efficiency can be improved, the learning and collaboration ability can be improved, resource expenditure can be saved, scalability can be improved, task management can be optimized, operation efficiency can be improved and operating costs can be reduced.

[0005] In view of this, one aspect of the present invention proposes a robot control method based on computer vision, which is applied to a robot control system based on computer vision. The robot control system based on computer vision includes a first robot, multiple edge intelligent terminals, multiple Internet of Things servers and a cloud server; the first robot is equipped with a central processing unit, a vision module, a positioning module, an inertial measurement module, a laser radar module, a communication module and an execution module; the robot control method based on computer vision includes:

[0006] The first robot obtains first position information of itself;

[0007] The first robot determines a first Internet of Things server from the Internet of Things servers according to the first position information and a first preset condition;

[0008] The first robot establishes a communication connection with the first Internet of Things server, and sends first robot attribute information of the first robot to the first Internet of Things server;

[0009] The first robot acquires first visual data and first surrounding environment data, and sends the first visual data and the first surrounding environment data to the first Internet of Things server;

[0010] The first Internet of Things server determines a corresponding first behavior control model according to the first robot attribute information;

[0011] The first IoT server determines whether the visual recognition capability of the first robot meets the current environmental conditions according to the first visual data and the first surrounding environment data;

[0012] If satisfied, sending the first behavior control model to the first robot to control the operation behavior of the first robot;

[0013] If not, determining a first vision reinforcement requirement of the first robot according to the first vision data and the first surrounding environment data, and determining a first edge intelligent terminal from the plurality of edge intelligent terminals according to the first vision reinforcement requirement;

[0014] The first edge intelligent terminal acquires second surrounding environment data and sends the second surrounding environment data to the first Internet of Things server;

[0015] The first Internet of Things server adjusts the first behavior control model according to the second surrounding environment data and the first visual reinforcement requirement to obtain a second behavior control model, and sends the second behavior control model to the first robot;

[0016] The first robot controls its own operation behavior according to the second behavior control model.

[0017] Optionally, it also includes:

[0018] The first Internet of Things server determines a second robot that meets a second preset condition;

[0019] Acquiring second visual data of the second robot;

[0020] Adjust the second behavior control model according to the second visual data to obtain a third behavior control model, and send the third behavior control model to the first robot;

[0021] The first robot controls its own operating behavior according to the third behavior control model.

[0022] Optionally, the step of the first Internet of Things server determining a second robot that meets the second preset condition includes:

[0023] The first IoT server collects reference robot attributes and reference robot status data of a plurality of reference robots;

[0024] Determine a second preset condition according to the first position information, the first robot attribute information, the first surrounding environment data, and the first task requirement of the first robot;

[0025] For each of the reference robots, judging whether its attribute meets the second preset condition according to the corresponding reference robot attribute and the reference robot state data;

[0026] Marking the reference robots that meet the second preset condition as candidate robots;

[0027] Scoring the candidate robots;

[0028] Selecting the robot with the highest score among the candidate robots as the second robot according to the scores from high to low;

[0029] The first IoT server sends the second robot ID to the first robot.

[0030] Optionally, the step of the first robot determining a first Internet of Things server from the Internet of Things servers according to the first position information and a first preset condition includes:

[0031] The first robot presets a first preset condition for selecting a corresponding server based on factors such as distance, performance, function, and resource utilization of multiple servers;

[0032] The first robot establishes connections with the plurality of IoT servers using the built-in communication module, and obtains corresponding server location information and server condition parameters from each IoT server;

[0033] The first robot calculates the distance with each of the IoT server locations according to the first location information and the server location information;

[0034] The first robot calculates a comprehensive score for each of the IoT servers using a weighted algorithm or a strategy optimization algorithm according to the distance calculation result and the server condition parameters;

[0035] The first robot selects the IoT server with the highest comprehensive score as the first IoT server to establish a long link for data communication.

[0036] Optionally, the step of the first IoT server determining a corresponding first behavior control model according to the first robot attribute information includes:

[0037] The first Internet of Things server receives a first data frame including attribute information of the first robot sent by the first robot;

[0038] The first Internet of Things server parses the header portion of the first data frame according to a predefined standard data transmission format;

[0039] Acquire the meta information of the first robot attribute information in the packet header part;

[0040] Decomposing the first data frame into domain fields according to the meta information;

[0041] Interpret each domain field in turn, restore its attribute content, and assemble all attribute contents into a key-value pair data structure;

[0042] Save the assembled attribute data structure to the local database;

[0043] The first IoT server retrieves the first robot attribute information sent by the first robot from the local database;

[0044] According to the first robot attribute information, matching and selecting a corresponding behavior control model from a predefined model library as the first behavior control model;

[0045] If the first robot attribute information cannot directly match the model, the first IoT server sends the query request to the corresponding cloud server;

[0046] After the cloud server passes the knowledge base matching, receiving the corresponding behavior control model definition returned by the cloud server;

[0047] The first Internet of Things server parses the behavior control model definition, establishes a behavior control model instance, and obtains the first behavior control model;

[0048] The first Internet of Things server sends a corresponding first control instruction to the first robot according to the rule constraints and algorithm settings of the first behavior control model.

[0049] Optionally, the first IoT server determines whether the visual recognition capability of the first robot meets current environmental conditions according to the first visual data and the first surrounding environment data, including:

[0050] The first IoT server performs object detection and recognition on the first visual data and the first surrounding environment data by using computer vision technology to obtain current environment feature data;

[0051] Query and obtain a first visual parameter of the visual module of the first robot;

[0052] Determining, according to the current environment feature data, whether the first visual parameter of the first robot can meet the current environment feature requirements;

[0053] If not, the first IoT server will notify the first robot through the network that the quality of the visual data provided cannot meet the task;

[0054] At the same time, a command is issued to adjust the first visual parameter of the first robot and collect a new round of data for re-evaluation.

[0055] Optionally, the step of determining a first visual reinforcement requirement of the first robot according to the first visual data and the first surrounding environment data, and determining a first edge intelligent terminal from the plurality of edge intelligent terminals according to the first visual reinforcement requirement includes:

[0056] By performing visual data analysis on the first visual data and the first surrounding environment data, determining environmental factors that hinder visual recognition, and determining a first visual reinforcement requirement;

[0057] Acquire corresponding device parameters and device functions from the plurality of edge intelligent terminals;

[0058] According to the device parameters and the device functions, evaluating whether each of the edge intelligent terminals can meet the first visual enhancement requirement, and calculating an evaluation score;

[0059] Calculating a first comprehensive score of the edge intelligent terminal according to a first distance between the edge intelligent terminal and the first robot in combination with the evaluation score;

[0060] The edge intelligent terminal corresponding to the highest score in the first comprehensive score is selected as the first edge intelligent terminal.

[0061] Optionally, the step of the first edge intelligent terminal acquiring the second surrounding environment data and sending the second surrounding environment data to the first Internet of Things server includes:

[0062] The first edge intelligent terminal obtains the surrounding second environment data in real time through various sensors configured on the first edge intelligent terminal;

[0063] The first edge intelligent terminal digitally processes the second environment data into a standard data format;

[0064] The first edge intelligent terminal establishes a long connection channel with the first Internet of Things server;

[0065] The first edge intelligent terminal sends the latest second environment data in the form of data packets to the first Internet of Things server at a preset time interval.

[0066] Optionally, the first IoT server adjusts the first behavior control model according to the second surrounding environment data and the first visual reinforcement requirement to obtain a second behavior control model, and sends the second behavior control model to the first robot, including:

[0067] identifying a second influencing factor from the second surrounding environment data;

[0068] Determining whether the first behavior control model needs to be adjusted according to the second influencing factor;

[0069] If necessary, the first IoT server loads the first behavior control model in a memory.

[0070] Adding a new rule or state to the first behavior control model according to the second influencing factor and the first visual reinforcement requirement;

[0071] Re-parameterizing and optimizing the adjusted model to obtain the second behavior control model;

[0072] The second behavior control model is serialized and sent to the first robot.

[0073] Another aspect of the present invention provides a robot control system based on computer vision, comprising: a first robot, a plurality of edge intelligent terminals, a plurality of Internet of Things servers and a cloud server; the first robot is equipped with a central processing unit, a vision module, a positioning module, an inertial measurement module, a laser radar module, a communication module, and an execution module; wherein,

[0074] The visual module is used to collect image data in real time;

[0075] The central processing unit is used to process the image data and detect targets using convolutional neural networks and computer vision technology to achieve environment modeling and recognition; and is also used to use data collected by the vision module and / or the positioning module and / or the inertial measurement module and / or the laser radar module to determine the posture and position of the first robot in real time through three-dimensional space modeling, visual positioning technology and navigation technology;

[0076] The communication module is used to realize data transmission and command communication between the first robot and the edge intelligent terminal and the Internet of Things server based on a 5G or WIFI network;

[0077] The execution module is used to control the movement of the first robot through various types of sensors and actuators;

[0078] The central processor is also used to model the environmental action value using a deep reinforcement learning algorithm and select actions in combination with multi-dimensional data obtained from each module;

[0079] The first robot is configured as:

[0080] Obtain the first location information of the user;

[0081] Determine a first Internet of Things server from the Internet of Things servers according to the first location information and a first preset condition;

[0082] Establishing a communication connection with the first Internet of Things server, and sending the first robot attribute information of the first robot to the first Internet of Things server;

[0083] Acquire first visual data and first surrounding environment data, and send the first visual data and the first surrounding environment data to the first Internet of Things server;

[0084] The first IoT server is configured as:

[0085] Determine a corresponding first behavior control model according to the first robot attribute information;

[0086] Determining whether the visual recognition capability of the first robot meets current environmental conditions according to the first visual data and the first surrounding environment data;

[0087] If satisfied, sending the first behavior control model to the first robot to control the operation behavior of the first robot;

[0088] If not, determining a first vision reinforcement requirement of the first robot according to the first vision data and the first surrounding environment data, and determining a first edge intelligent terminal from the plurality of edge intelligent terminals according to the first vision reinforcement requirement;

[0089] The first edge intelligent terminal is configured to: obtain second surrounding environment data and send the second surrounding environment data to the first Internet of Things server;

[0090] The first Internet of Things server is configured to: adjust the first behavior control model according to the second surrounding environment data and the first visual reinforcement requirement to obtain a second behavior control model, and send the second behavior control model to the first robot;

[0091] The first robot is configured to control its own operation behavior according to the second behavior control model.

[0092] According to the technical solution of the present invention, a robot control method based on computer vision includes: the first robot obtains first position information of itself; the first robot determines a first Internet of Things server from the Internet of Things server according to the first position information and a first preset condition; the first robot establishes a communication connection with the first Internet of Things server, and sends first robot attribute information of the first robot to the first Internet of Things server; the first robot obtains first visual data and first surrounding environment data, and sends the first visual data and the first surrounding environment data to the first Internet of Things server; the first Internet of Things server determines a corresponding first behavior control model according to the first robot attribute information; the first Internet of Things server determines the visual recognition of the first robot according to the first visual data and the first surrounding environment data. Whether the identification capability meets the current environmental conditions; if so, the first behavior control model is sent to the first robot to control the operation behavior of the first robot; if not, the first visual reinforcement requirement of the first robot is determined according to the first visual data and the first surrounding environment data, and the first edge intelligent terminal is determined from the plurality of edge intelligent terminals according to the first visual reinforcement requirement; the first edge intelligent terminal obtains the second surrounding environment data and sends the second surrounding environment data to the first Internet of Things server; the first Internet of Things server adjusts the first behavior control model according to the second surrounding environment data and the first visual reinforcement requirement to obtain the second behavior control model, and sends the second behavior control model to the first robot; the first robot controls its own operation behavior according to the second behavior control model. Through the scheme of the present invention, the robot's environmental perception range and level can be improved, the robot's adaptive ability can be enhanced, the task completion efficiency can be improved, the learning and collaboration ability can be improved, resource expenditure can be saved, scalability can be improved, task management can be optimized, operation efficiency can be improved, and operation cost can be reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0093] Figure 1 is a flow chart of a robot control method based on computer vision provided by one embodiment of the present invention;

[0094] Figure 2 It is a schematic block diagram of a robot control system based on computer vision provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0095] In order to more clearly understand the above-mentioned purpose, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0096] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the protection scope of the present invention is not limited to the specific embodiments disclosed below.

[0097] The terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices.

[0098] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0099] Refer to the following Figure 1 to Figure 2 A computer vision-based robot control method and system are described in detail according to some embodiments of the present invention.

[0100] like Figure 1 As shown, an embodiment of the present invention provides a robot control method based on computer vision, which is applied to a robot control system based on computer vision. The robot control system based on computer vision includes a first robot, multiple edge intelligent terminals, multiple Internet of Things servers and a cloud server; the first robot is configured with a central processing unit, a vision module, a positioning module, an inertial measurement module, a laser radar module, a communication module and an execution module; the robot control method based on computer vision includes:

[0101] The first robot obtains first position information of itself;

[0102] In this step, the central processor of the first robot determines the first position information of the first robot based on the data obtained from the vision module and / or the positioning module and / or the inertial measurement module and / or the lidar module. For example, the first robot obtains its own first position information, such as longitude and latitude coordinates, through a built-in positioning module (such as GPS).

[0103] The first robot determines a first Internet of Things server from the Internet of Things servers according to the first position information and a first preset condition;

[0104] The first robot establishes a communication connection with the first Internet of Things server, and sends first robot attribute information of the first robot to the first Internet of Things server;

[0105] In this step, the first robot maintains a continuous connection state through a long link established with the first IoT server; the first robot obtains its own first robot attribute information (such as robot model, hardware configuration, software version, function, performance, structural data, material, etc.) through the built-in communication module; the first robot packages the collected attribute information into a data frame in a standard protocol format (such as JSON format); the first robot sends the data frame to the first IoT server through the communication module; after receiving the data frame, the first IoT server unpacks and restores the first robot attribute information; the first IoT server stores the first robot attribute information in its internal database. Through this implementation process, the first robot establishes a two-way reliable data connection with the first IoT server, and the first IoT server learns the detailed information of the first robot, which will help the IoT server provide personalized services for the robot, such as customized software updates, task allocation for the best performance, and effectively improve the system collaboration capability.

[0106] The first robot acquires first visual data and first surrounding environment data, and sends the first visual data and the first surrounding environment data to the first Internet of Things server;

[0107] In this step, the first robot collects video streams in real time through a built-in visual module (such as a camera) to obtain first visual data; the first robot perceives the surrounding environment in real time through sensors such as an inertial measurement module and a laser radar module to obtain first surrounding environment data (such as position, distance, light data, data of objects in the environment, etc.); the first robot encodes and compresses the first visual data (such as H264 video encoding) and digitizes the first surrounding environment data; the encoded first visual data and the digitally processed first surrounding environment data are integrated into a data packet; the data packet is sent to the first Internet of Things server in a standard data transmission protocol format by using a communication connection with the first Internet of Things server; the first Internet of Things server receives and decodes the data packet to extract the first visual data and the first surrounding environment data. Through this implementation, the first Internet of Things server can obtain data support for the current operating status and surrounding conditions of the first robot in real time, and then perform monitoring management, task allocation and scheduling, etc., effectively improving the intelligence level of the Internet of Things system.

[0108] The first IoT server determines a corresponding first behavior control model according to the first robot attribute information (which may be determined by communicating with a cloud server);

[0109] The first IoT server determines whether the visual recognition capability of the first robot meets the current environmental conditions according to the first visual data and the first surrounding environment data;

[0110] If satisfied, sending the first behavior control model to the first robot to control the operation behavior of the first robot;

[0111] In this step, the first IoT server serializes the first behavior control model instance built in the memory into a byte stream; encapsulates the byte stream with a network data packet, and specifies it as a control model type in the packet header; sends the control model data packet to the first robot through a long connection; the first robot unpacks the control model byte stream according to the packet header; the first robot deserializes in a manner consistent with the server and rebuilds the model instance object; the first robot performs the operation behavior according to the rules and algorithms defined by the instance object; first performs the initialization behavior, then continuously reads the perception data and calls the control method; and implements the behavior control loop execution consistent with the server. Through this transmission and reconstruction mechanism, after the first task assignment, the robot can autonomously perform the behavior control task locally, reduce network dependence, and improve control efficiency.

[0112] If not, determining a first vision reinforcement requirement of the first robot according to the first vision data and the first surrounding environment data, and determining a first edge intelligent terminal from the plurality of edge intelligent terminals according to the first vision reinforcement requirement;

[0113] The first edge intelligent terminal acquires second surrounding environment data and sends the second surrounding environment data to the first Internet of Things server;

[0114] The first Internet of Things server adjusts the first behavior control model according to the second surrounding environment data and the first visual reinforcement requirement to obtain a second behavior control model, and sends the second behavior control model to the first robot;

[0115] The first robot controls its own operation behavior according to the second behavior control model.

[0116] In this embodiment, generally speaking, the behavior control model of the robot can be defined in the following manner:

[0117] Object-oriented definition: Use classes and objects to define the model body and its behavior. For example, define a Robot class, including the behavior method run().

[0118] State machine definition: Use finite state machines to describe each state and state transitions under trigger conditions;

[0119] Rule-driven definition: Use if-then rules to describe operation behaviors in different environments;

[0120] Definition of learning-driven approach: Using machine learning algorithms to learn behavioral relationship strategies on a large number of samples;

[0121] For the first robot, its behavior control model can be specifically defined as:

[0122] The Robot class defines robot objects and properties. The behavior methods include initialize(), detect(), plan(), and execute(). The states include ready, running, and stopped.

[0123] The state transition rule (such as trigger condition) is that the task is received and then enters the running state;

[0124] And it is realized by using mathematical model support methods such as kinematics and dynamics.

[0125] In this embodiment, the definition method takes into account the three aspects of object, state and rule at the same time, and can flexibly adjust the model behavior to meet the needs of different tasks.

[0126] According to the technical solution of this embodiment, the first robot obtains first position information of itself; the first robot determines a first Internet of Things server from the Internet of Things server according to the first position information and a first preset condition; the first robot establishes a communication connection with the first Internet of Things server, and sends first robot attribute information of the first robot to the first Internet of Things server; the first robot obtains first visual data and first surrounding environment data, and sends the first visual data and the first surrounding environment data to the first Internet of Things server; the first Internet of Things server determines a corresponding first behavior control model according to the first robot attribute information; the first Internet of Things server determines whether the visual recognition ability of the first robot meets the current requirements according to the first visual data and the first surrounding environment data. environmental conditions; if satisfied, the first behavior control model is sent to the first robot to control the operation behavior of the first robot; if not satisfied, the first visual reinforcement requirement of the first robot is determined according to the first visual data and the first surrounding environment data, and the first edge intelligent terminal is determined from the plurality of edge intelligent terminals according to the first visual reinforcement requirement; the first edge intelligent terminal obtains the second surrounding environment data and sends the second surrounding environment data to the first Internet of Things server; the first Internet of Things server adjusts the first behavior control model according to the second surrounding environment data and the first visual reinforcement requirement to obtain the second behavior control model, and sends the second behavior control model to the first robot; the first robot controls its own operation behavior according to the second behavior control model. Through the scheme of the present invention, the robot's environmental perception range and level can be improved, the robot's adaptive ability can be enhanced, the task completion efficiency can be improved, the learning and collaboration ability can be improved, resource expenditure can be saved, scalability can be improved, task management can be optimized, operation efficiency can be improved, and operation cost can be reduced.

[0127] In some possible implementations of the present invention, the following further aspects are included:

[0128] The first Internet of Things server determines a second robot that meets a second preset condition;

[0129] Acquiring second visual data of the second robot;

[0130] In this step, the first IoT server sends a request to the second robot, asking the second robot to provide visual data; the second robot that receives the request obtains real-time visual image data through its own visual sensor (such as a camera); the second robot encodes and compresses the collected visual data (such as a picture or video stream); the second robot transmits the compressed second visual data to the first IoT server through the network; the first IoT server receives this visual data packet and decodes it; the first IoT server extracts and restores the second visual data (visual image) from it. This process realizes the ability of the first IoT server to actively request certain data from other robots, providing technical support for subsequent complex tasks based on multi-machine collaboration, such as the first robot can refer to the existing visual samples of other robots to learn visual tasks.

[0131] Adjust the second behavior control model according to the second visual data to obtain a third behavior control model, and send the third behavior control model to the first robot;

[0132] In this step, the first IoT server extracts key information from the second visual data, such as the location of the identified target area, etc.; determines whether this information requires adjustment to the second behavior control model; if necessary, loads the second model into the memory; modifies the model according to the new information in the second visual data, such as adding new recognition and tracking rules; tests and optimizes to obtain the third behavior control model; sends the third model to the first robot; the first robot starts the third model to perform the task. This implementation enables the behavior model to continuously learn to detect new environmental details and optimize in real time after obtaining visual reference data, so that the first robot's task strategy continues to approach the reference robot, improves its own visual processing level, and has stronger learning enhancement capabilities than fixed models, reflecting the intelligent effect of knowledge dissemination between machines under the IoT.

[0133] The first robot controls its own operating behavior according to the third behavior control model.

[0134] In some possible implementations of the present invention, the step of the first IoT server determining a second robot that meets the second preset condition includes:

[0135] The first IoT server collects reference robot attributes and reference robot status data of a plurality of reference robots;

[0136] Determining a second preset condition (such as a model, performance, function, position range, etc. that matches the first position information, the first robot attribute information, the first surrounding environment data, and the first task requirement, or a difference is within a preset difference range) according to the first position information, the first robot attribute information, the first surrounding environment data, and the first task requirement of the first robot;

[0137] For each of the reference robots, judging whether its attribute meets the second preset condition according to the corresponding reference robot attribute and the reference robot state data;

[0138] Marking the reference robots that meet the second preset condition as candidate robots;

[0139] Scoring the candidate robots (e.g., scoring based on factors such as task relevance, location distance, performance / function matching, etc.);

[0140] Selecting the robot with the highest score among the candidate robots as the second robot according to the scores from high to low;

[0141] The first IoT server sends the second robot ID to the first robot.

[0142] In this embodiment, through this implementation, the system can assign the robot most relevant to the task as a reference, providing a more reasonable reference learning sample for the first robot; this is very beneficial for achieving collaborative learning and knowledge dissemination among robots, while avoiding unnecessary waste of resources and improving the working efficiency of the Internet of Things.

[0143] In some possible implementations of the present invention, the step of the first robot determining the first Internet of Things server from the Internet of Things servers according to the first position information and the first preset condition includes:

[0144] The first robot presets a first preset condition for selecting a corresponding server based on factors such as distance, performance, function, and resource utilization of multiple servers;

[0145] The first robot establishes connections with the plurality of IoT servers using the built-in communication module, and obtains corresponding server location information and server condition parameters from each IoT server;

[0146] The first robot calculates the distance with each of the IoT server locations according to the first location information and the server location information;

[0147] The first robot calculates a comprehensive score for each of the IoT servers using a weighted algorithm or a strategy optimization algorithm according to the distance calculation result and the server condition parameters (a server with a high score theoretically better meets the preset conditions);

[0148] In this step, the first robot normalizes various evaluation criteria such as distance, server performance, resource utilization, etc., so that the data range of each criterion becomes 0-1; for the distance criterion, 1 / distance is used for normalization; servers with closer distances have higher scores; for positive criteria such as performance, normalization is performed directly; for negative criteria such as resource occupancy, (1-occupancy) is used for normalization; different weights w are set for each criterion, such as distance weight 0.3, performance weight 0.5, etc.; and the comprehensive score of each server i is calculated:

[0149] score_i=w_distance*norm(1 / distance_i)+w_performance*norm(performance_i)+w_resource*norm(1-resource_i);

[0150] The comprehensive scores of all servers are sorted, and the server with the highest score is used as the target server; the algorithm takes various standards into consideration and gives a quantified priority result, realizing the first robot intelligent judgment method for different servers.

[0151] The first robot selects the IoT server with the highest comprehensive score as the first IoT server to establish a long link for data communication.

[0152] In this embodiment, the first robot can intelligently select an IoT server that is close and has low resource usage according to its own conditions, so as to achieve an effective matching connection between the machine and the server; this can avoid waste of resources, reduce communication delays, and improve task response efficiency.

[0153] In some possible implementations of the present invention, the step of the first IoT server determining the corresponding first behavior control model according to the first robot attribute information includes:

[0154] The first Internet of Things server receives a first data frame including attribute information of the first robot sent by the first robot;

[0155] The first IoT server parses the header portion of the first data frame according to a predefined standard data transmission format (such as a JSON data packet header format);

[0156] Obtaining meta information (such as data length and data type, etc.) of the first robot attribute information in the packet header part;

[0157] Decomposing the first data frame into domain fields according to the meta information;

[0158] Interpret each domain field in turn, restore its attribute content (for example, interpret the robot model field as the string "industrial robot type X"), and assemble all attribute contents into a key-value pair data structure (such as dict or class);

[0159] Save the assembled attribute data structure to the local database;

[0160] The first IoT server retrieves the first robot attribute information sent by the first robot from the local database;

[0161] According to the first robot attribute information (such as robot model, hardware configuration, software version, function, performance, structural data, material and other attributes), matching and selecting a corresponding behavior control model from a predefined model library as the first behavior control model;

[0162] If the first robot attribute information cannot directly match the model, the first IoT server sends the query request to the corresponding cloud server;

[0163] After the cloud server passes the knowledge base matching, receiving the corresponding behavior control model definition returned by the cloud server;

[0164] The first Internet of Things server parses the behavior control model definition, establishes a behavior control model instance, and obtains the first behavior control model;

[0165] In this step, the model type, version and other meta-information are obtained according to the model definition received from the cloud server; the model file format, such as JSON or XML, is identified according to the meta-information; the model file structure is parsed, and the content definitions such as classes, functions, and parameters are read; the class instance object of the class is generated in the server memory according to the class definition; the corresponding method is implemented in the class instance according to the function prototype; the class attribute member variables are assigned and initialized according to the model file configuration parameters; the instance object generates the next action by reading the sensor data and calling the control method; the action is sent to the first robot for execution; the sensor data is read repeatedly and the method is called, and the control loop is iteratively executed. Through such a repeated parsing and instantiation process, the model definition information is first extracted, and then the complete model behavior instance is restored in the server, which truly realizes the execution of the behavior control logic.

[0166] The first Internet of Things server sends a corresponding first control instruction to the first robot according to the rule constraints and algorithm settings of the first behavior control model.

[0167] In this embodiment, the first IoT server realizes automatic unpacking and attribute information restoration through predefined standard formats and step-by-step parsing. This can effectively meet the real-time processing requirements of a large number of robots reporting attributes; and can implement precisely customized control strategies for different types of robots, while solving the knowledge limitation of edge servers. Through cloud assistance, the success rate of intelligent selection models is increased, effectively improving the quality and success rate of IoT control.

[0168] In some possible implementations of the present invention, the first IoT server determines whether the visual recognition capability of the first robot meets the current environmental conditions according to the first visual data and the first surrounding environment data, including:

[0169] The first IoT server performs object detection and recognition on the first visual data and the first surrounding environment data by using computer vision technology to obtain current environment feature data;

[0170] Query and obtain first visual parameters (such as resolution, recognition algorithm, etc.) of the visual module of the first robot;

[0171] According to the current environment feature data, determining whether the first visual parameter of the first robot can meet the current environment feature requirements (such as object size, lighting conditions, etc.);

[0172] In this step, computer vision technology is used to identify key environmental feature objects from the first visual data and the first surrounding environment data; numerical parameters such as the size and chromaticity range of the identified key objects are counted; the resolution of the first robot visual device and the capability range of the recognition algorithm are obtained; the object parameters are compared with the capability range of the visual device: if the object size is larger than the device perception size, it is not satisfied; if the object chromaticity range is lower than the device chromaticity recognition range, it is not satisfied; if the object movement speed is higher than the device continuous observation capability, it is not satisfied; otherwise, the visual parameters meet the existing environmental feature requirements. If not satisfied, adjustment suggestions are given: improve the resolution to meet the size requirements, upgrade the recognition algorithm to meet the color or movement requirements, etc. In this step, the automation and standardization of judgment are achieved by quantitatively comparing the two parameter sets.

[0173] If not, the first IoT server will notify the first robot through the network that the quality of the visual data provided cannot meet the task;

[0174] At the same time, a command is issued to adjust the first visual parameter of the first robot and collect a new round of data for re-evaluation.

[0175] In this embodiment, the first IoT server can be responsible for evaluating whether the robot hardware matches the actual environment task requirements, effectively screening suitable robots, and avoiding task failures caused by improper use. This can improve the operating efficiency and success rate of the overall IoT system.

[0176] In some possible implementations of the present invention, the step of determining a first visual reinforcement requirement of the first robot according to the first visual data and the first surrounding environment data, and determining a first edge intelligent terminal from a plurality of edge intelligent terminals according to the first visual reinforcement requirement includes:

[0177] By performing visual data analysis on the first visual data and the first surrounding environment data, determining environmental factors that hinder visual recognition, and determining first visual reinforcement requirements (such as light brightness, acquisition method, resolution, etc.);

[0178] Obtain corresponding device parameters and device functions from a plurality of edge intelligent terminals (such as cameras, lighting equipment, etc.);

[0179] According to the device parameters and the device functions, evaluating whether each of the edge intelligent terminals can meet the first visual enhancement requirement, and calculating an evaluation score;

[0180] Calculating a first comprehensive score of the edge intelligent terminal according to a first distance between the edge intelligent terminal and the first robot in combination with the evaluation score;

[0181] In this step, each visual reinforcement requirement standard (such as light intensity, acquisition angle, resolution, etc.) is normalized to the range of 0-1; for each edge intelligent terminal, a standard score is given according to its support for each standard; each standard is given a weight, such as 0.5 for light intensity and 0.3 for resolution. Calculate the comprehensive score of each terminal j:

[0182] Comprehensive score j = weight 1 * standard score (light intensity) + weight 2 * standard score (resolution) + weight 3 * standard score (acquisition method);

[0183] At the same time, each edge intelligent terminal is assigned a distance weight according to the distance from the first robot, such as the inverse of the distance; the final first comprehensive score = comprehensive score * distance weight; sort by the final score, and select the edge intelligent terminal with the highest score or multiple edge intelligent terminals with scores exceeding the preset score. This step conducts quantitative evaluation of multiple factors, which can more accurately match the optimal visual aid device and help give full play to the synergy effect of the smart edge.

[0184] The edge intelligent terminal corresponding to the highest score in the first comprehensive score (or a preset number of scores from high to low) is selected as the first edge intelligent terminal.

[0185] In this embodiment, by adding support for appropriate edge vision devices, the deficiency of the first robot in meeting the task is effectively compensated, and the overall visual capacity of the system is improved; at the same time, the position factor is taken into consideration, the visual delay is effectively shortened, the real-time performance is enhanced, and the tasks with urgent visual requirements are better completed.

[0186] In some possible implementations of the present invention, the step of the first edge intelligent terminal acquiring the second surrounding environment data and sending the second surrounding environment data to the first Internet of Things server includes:

[0187] The first edge intelligent terminal obtains the surrounding second environment data in real time through various sensors (such as cameras, laser radars, etc.) configured by itself;

[0188] The first edge intelligent terminal digitally processes the second environment data into a standard data format (such as JSON, etc.);

[0189] The first edge intelligent terminal establishes a long connection channel with the first Internet of Things server;

[0190] The first edge intelligent terminal sends the latest second environment data in the form of a data packet to the first Internet of Things server at a preset time interval (such as every 1 second).

[0191] In some possible implementations of the present invention, the method further includes: the first IoT server receives a data packet and extracts the second environment data; performs format analysis on the data and restores the data to recognizable environment parameters; synchronizes the second environment data to a database and adds the data to an environment model; and determines whether the second environment data affects a robot task according to the position of the first robot;

[0192] If there is any impact, the first IoT server performs the following processing: when it is detected that the second environmental data contains obstacle information, it sends an obstacle avoidance instruction to the first robot; determines whether a new target object enters, and notifies the first robot to change its task; determines whether the environmental conditions have changed (such as weak light), and selects a new visual aid for the first robot; applies the second environmental data to the behavior control model, and modifies the operation strategy of the first robot;

[0193] Otherwise, the first IoT server directly imports the second environment data, and updates the environment model but does not adjust the first robot.

[0194] In this embodiment, the first IoT server can obtain the environmental perception of the area where the first edge terminal is located in real time, thereby expanding the environmental perception range of the system. This should play an important role in achieving more extensive environmental monitoring and task scheduling and debugging. At the same time, the edge load can also effectively alleviate the processing pressure of the central server. By comprehensively utilizing multi-source environmental data, the system can respond to changes in the task environment in real time and improve the robot's adaptability in complex environments.

[0195] In some possible implementations of the present invention, the first IoT server adjusts the first behavior control model according to the second surrounding environment data and the first visual reinforcement requirement to obtain a second behavior control model, and sends the second behavior control model to the first robot, including:

[0196] Identifying a second influencing factor (such as a newly appeared obstacle position) from the second surrounding environment data;

[0197] Determining whether the first behavior control model needs to be adjusted according to the second influencing factor;

[0198] If necessary, the first IoT server loads the first behavior control model in a memory.

[0199] According to the second influencing factor and the first visual enhancement requirement, adding new rules or states (such as actions to avoid new obstacles, fill light rules, resolution adjustment rules, shooting frequency control rules, camera movement rules, etc.) to the first behavior control model;

[0200] Re-parameterizing and optimizing the adjusted model to obtain the second behavior control model;

[0201] serializing the second behavior control model and sending it to the first robot;

[0202] In this embodiment, the first robot receives and deserializes the reconstructed second model instance (ie, the second behavior control model) and operates according to the new model.

[0203] In this embodiment, the first behavior control model can be adjusted and optimized in real time after monitoring environmental changes, so that the operation strategy of the first robot can better adapt to the actual environment and improve the quality and efficiency of task completion; compared with manual adjustment, it realizes the automatic model adaptation capability driven by environmental perception.

[0204] See also Figure 2Another embodiment of the present invention provides a robot control system based on computer vision, comprising: a first robot, a plurality of edge intelligent terminals, a plurality of Internet of Things servers and a cloud server; the first robot is configured with a central processing unit, a vision module, a positioning module, an inertial measurement module, a laser radar module, a communication module, and an execution module; wherein,

[0205] The visual module (such as a smart camera) is used to collect image data in real time;

[0206] The central processing unit is used to process the image data and detect targets using convolutional neural networks and computer vision technology to achieve environment modeling and recognition; it is also used to use the data collected by the vision module and / or the positioning module and / or the inertial measurement module and / or the laser radar module to determine the posture and position of the first robot in real time through three-dimensional space modeling, visual positioning technology (such as SLAM algorithm) and navigation technology;

[0207] The communication module is used to realize data transmission and command communication between the first robot and the edge intelligent terminal and the Internet of Things server based on a 5G or WIFI network;

[0208] The execution module is used to control the movement of the first robot through various types of sensors and actuators (such as a robotic arm);

[0209] The central processor is also used to model the environmental action value using a deep reinforcement learning algorithm (such as DDPG or A3C, etc.), and select actions based on the multi-dimensional data obtained from each module;

[0210] The first robot is configured as:

[0211] Obtain the first location information of the user;

[0212] Determine a first Internet of Things server from the Internet of Things servers according to the first location information and a first preset condition;

[0213] Establishing a communication connection with the first Internet of Things server, and sending the first robot attribute information of the first robot to the first Internet of Things server;

[0214] Acquire first visual data and first surrounding environment data, and send the first visual data and the first surrounding environment data to the first Internet of Things server;

[0215] The first IoT server is configured as:

[0216] Determine a corresponding first behavior control model according to the first robot attribute information (which may be determined by communicating with a cloud server);

[0217] Determining whether the visual recognition capability of the first robot meets current environmental conditions according to the first visual data and the first surrounding environment data;

[0218] If satisfied, sending the first behavior control model to the first robot to control the operation behavior of the first robot;

[0219] If not, determining a first vision reinforcement requirement of the first robot according to the first vision data and the first surrounding environment data, and determining a first edge intelligent terminal from the plurality of edge intelligent terminals according to the first vision reinforcement requirement;

[0220] The first edge intelligent terminal is configured to: obtain second surrounding environment data and send the second surrounding environment data to the first Internet of Things server;

[0221] The first Internet of Things server is configured to: adjust the first behavior control model according to the second surrounding environment data and the first visual reinforcement requirement to obtain a second behavior control model, and send the second behavior control model to the first robot;

[0222] The first robot is configured to control its own operation behavior according to the second behavior control model.

[0223] It should be known that Figure 2 The block diagram of the computer vision-based robot control system is for illustration only, and the number of modules shown therein does not limit the protection scope of the present invention.

[0224] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0225] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0226] In the several embodiments provided in the present application, it should be understood that the disclosed device can be implemented in other ways. For example, the device embodiments described above are only schematic, such as the division of the above-mentioned units, which is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

[0227] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0228] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0229] If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a memory, including a number of instructions to enable a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the above-mentioned methods of each embodiment of the present application. The aforementioned memory includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or CD-ROM and other media that can store program codes.

[0230] A person skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable memory, and the memory can include: a flash drive, a read-only memory (English: Read-Only Memory, abbreviated as: ROM), a random access memory (English: Random Access Memory, abbreviated as: RAM), a magnetic disk or an optical disk, etc.

[0231] The embodiments of the present application are introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method and core idea of ​​the present application. At the same time, for general technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

[0232] Although the present invention is disclosed as above, the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions without departing from the spirit and scope of the present invention, and can make various changes and modifications, including the combination of the above-mentioned different functions and implementation steps, including software and hardware implementation methods, all of which are within the scope of protection of the present invention.

Claims

1. A robot control method based on computer vision, characterized in that: The invention is applied to a robot control system based on computer vision, wherein the robot control system based on computer vision includes a first robot, a plurality of edge intelligent terminals, a plurality of Internet of Things servers and a cloud server; the first robot is configured with a central processing unit, a vision module, a positioning module, an inertial measurement module, a laser radar module, a communication module and an execution module; the robot control method based on computer vision includes: The first robot obtains first position information of itself; The first robot determines a first Internet of Things server from the Internet of Things servers according to the first position information and a first preset condition; The first robot establishes a communication connection with the first Internet of Things server, and sends first robot attribute information of the first robot to the first Internet of Things server; The first robot acquires first visual data and first surrounding environment data, and sends the first visual data and the first surrounding environment data to the first Internet of Things server; The first Internet of Things server determines a corresponding first behavior control model according to the first robot attribute information; The first IoT server determines whether the visual recognition capability of the first robot meets the current environmental conditions according to the first visual data and the first surrounding environment data; If satisfied, sending the first behavior control model to the first robot to control the operation behavior of the first robot; If not, determining a first vision reinforcement requirement of the first robot according to the first vision data and the first surrounding environment data, and determining a first edge intelligent terminal from the plurality of edge intelligent terminals according to the first vision reinforcement requirement; The first edge intelligent terminal acquires second surrounding environment data and sends the second surrounding environment data to the first Internet of Things server; The first Internet of Things server adjusts the first behavior control model according to the second surrounding environment data and the first visual reinforcement requirement to obtain a second behavior control model, and sends the second behavior control model to the first robot; The first robot controls its own operation behavior according to the second behavior control model.

2. The robot control method based on computer vision according to claim 1, characterized in that: Also includes: The first Internet of Things server determines a second robot that meets a second preset condition; Acquiring second visual data of the second robot; Adjust the second behavior control model according to the second visual data to obtain a third behavior control model, and send the third behavior control model to the first robot; The first robot controls its own operating behavior according to the third behavior control model.

3. The robot control method based on computer vision according to claim 2, characterized in that: The step of the first Internet of Things server determining a second robot that meets the second preset condition includes: The first IoT server collects reference robot attributes and reference robot status data of a plurality of reference robots; Determine a second preset condition according to the first position information, the first robot attribute information, the first surrounding environment data, and the first task requirement of the first robot; For each of the reference robots, judging whether its attribute meets the second preset condition according to the corresponding reference robot attribute and the reference robot state data; Marking the reference robots that meet the second preset condition as candidate robots; Scoring the candidate robots; Selecting the robot with the highest score among the candidate robots as the second robot according to the scores from high to low; The first IoT server sends the second robot ID to the first robot.

4. The robot control method based on computer vision according to claim 3, characterized in that: The step of the first robot determining the first Internet of Things server from the Internet of Things servers according to the first position information and the first preset condition includes: The first robot presets a first preset condition for selecting a corresponding server based on factors such as distance, performance, function, and resource utilization of multiple servers; The first robot establishes connections with the plurality of IoT servers using the built-in communication module, and obtains corresponding server location information and server condition parameters from each IoT server; The first robot calculates the distance with each of the IoT server locations according to the first location information and the server location information; The first robot calculates a comprehensive score for each of the IoT servers using a weighted algorithm or a strategy optimization algorithm according to the distance calculation result and the server condition parameters; The first robot selects the IoT server with the highest comprehensive score as the first IoT server to establish a long link for data communication.

5. The robot control method based on computer vision according to claim 4, characterized in that: The step of the first IoT server determining a corresponding first behavior control model according to the first robot attribute information includes: The first Internet of Things server receives a first data frame including attribute information of the first robot sent by the first robot; The first Internet of Things server parses the header portion of the first data frame according to a predefined standard data transmission format; Acquire the meta information of the first robot attribute information in the packet header part; Decomposing the first data frame into domain fields according to the meta information; Interpret each domain field in turn, restore its attribute content, and assemble all attribute contents into a key-value pair data structure; Save the assembled attribute data structure to the local database; The first IoT server retrieves the first robot attribute information sent by the first robot from the local database; According to the first robot attribute information, matching and selecting a corresponding behavior control model from a predefined model library as the first behavior control model; If the first robot attribute information cannot directly match the model, the first IoT server sends the query request to the corresponding cloud server; After the cloud server passes the knowledge base matching, receiving the corresponding behavior control model definition returned by the cloud server; The first Internet of Things server parses the behavior control model definition, establishes a behavior control model instance, and obtains the first behavior control model; The first Internet of Things server sends a corresponding first control instruction to the first robot according to the rule constraints and algorithm settings of the first behavior control model.

6. The robot control method based on computer vision according to claim 5, characterized in that: The first IoT server determines whether the visual recognition capability of the first robot meets the current environmental conditions according to the first visual data and the first surrounding environment data, including: The first IoT server performs object detection and recognition on the first visual data and the first surrounding environment data by using computer vision technology to obtain current environment feature data; Query and obtain a first visual parameter of the visual module of the first robot; Determining, according to the current environment feature data, whether the first visual parameter of the first robot can meet the current environment feature requirements; If not, the first IoT server will notify the first robot through the network that the quality of the visual data provided cannot meet the task; At the same time, a command is issued to adjust the first visual parameter of the first robot and collect a new round of data for re-evaluation.

7. The robot control method based on computer vision according to claim 6, characterized in that: The step of determining a first visual reinforcement requirement of the first robot according to the first visual data and the first surrounding environment data, and determining a first edge intelligent terminal from the plurality of edge intelligent terminals according to the first visual reinforcement requirement includes: By performing visual data analysis on the first visual data and the first surrounding environment data, determining environmental factors that hinder visual recognition, and determining a first visual reinforcement requirement; Acquire corresponding device parameters and device functions from the plurality of edge intelligent terminals; According to the device parameters and the device functions, evaluating whether each of the edge intelligent terminals can meet the first visual enhancement requirement, and calculating an evaluation score; Calculating a first comprehensive score of the edge intelligent terminal according to a first distance between the edge intelligent terminal and the first robot in combination with the evaluation score; The edge intelligent terminal corresponding to the highest score in the first comprehensive score is selected as the first edge intelligent terminal.

8. The robot control method based on computer vision according to claim 7, characterized in that: The step of the first edge intelligent terminal acquiring the second surrounding environment data and sending the second surrounding environment data to the first Internet of Things server includes: The first edge intelligent terminal obtains the surrounding second surrounding environment data in real time through various sensors configured on the first edge intelligent terminal; The first edge intelligent terminal digitally processes the second surrounding environment data into a standard data format; The first edge intelligent terminal establishes a long connection channel with the first Internet of Things server; The first edge intelligent terminal sends the latest second ambient environment data in the form of data packets to the first Internet of Things server at a preset time interval.

9. The robot control method based on computer vision according to claim 8, characterized in that: The first IoT server adjusts the first behavior control model according to the second surrounding environment data and the first visual reinforcement requirement to obtain a second behavior control model, and sends the second behavior control model to the first robot, including: identifying a second influencing factor from the second surrounding environment data; Determining whether the first behavior control model needs to be adjusted according to the second influencing factor; If necessary, the first IoT server loads the first behavior control model in a memory; Adding a new rule or state to the first behavior control model according to the second influencing factor and the first visual reinforcement requirement; Re-parameterizing and optimizing the adjusted model to obtain the second behavior control model; The second behavior control model is serialized and sent to the first robot.

10. A robot control system based on computer vision, characterized in that: include: A first robot, a plurality of edge intelligent terminals, a plurality of IoT servers and a cloud server; the first robot is equipped with a central processing unit, a visual module, a positioning module, an inertial measurement module, a laser radar module, a communication module, and an execution module; wherein, The visual module is used to collect image data in real time; The central processing unit is used to process the image data and detect targets using convolutional neural networks and computer vision technology to achieve environment modeling and recognition; and is also used to use data collected by the vision module and / or the positioning module and / or the inertial measurement module and / or the laser radar module to determine the posture and position of the first robot in real time through three-dimensional space modeling, visual positioning technology and navigation technology; The communication module is used to realize data transmission and command communication between the first robot and the edge intelligent terminal and the Internet of Things server based on a 5G or WIFI network; The execution module is used to control the movement of the first robot through various types of sensors and actuators; The central processor is also used to model the environmental action value using a deep reinforcement learning algorithm and select actions in combination with multi-dimensional data obtained from each module; The first robot is configured as: Obtain the first location information of the user; Determine a first Internet of Things server from the Internet of Things servers according to the first location information and a first preset condition; Establishing a communication connection with the first Internet of Things server, and sending the first robot attribute information of the first robot to the first Internet of Things server; Acquire first visual data and first surrounding environment data, and send the first visual data and the first surrounding environment data to the first Internet of Things server; The first IoT server is configured as: Determine a corresponding first behavior control model according to the first robot attribute information; Determining whether the visual recognition capability of the first robot meets current environmental conditions according to the first visual data and the first surrounding environment data; If satisfied, sending the first behavior control model to the first robot to control the operation behavior of the first robot; If not, determining a first vision reinforcement requirement of the first robot according to the first vision data and the first surrounding environment data, and determining a first edge intelligent terminal from the plurality of edge intelligent terminals according to the first vision reinforcement requirement; The first edge intelligent terminal is configured to: obtain second surrounding environment data and send the second surrounding environment data to the first Internet of Things server; The first Internet of Things server is configured to: adjust the first behavior control model according to the second surrounding environment data and the first visual reinforcement requirement to obtain a second behavior control model, and send the second behavior control model to the first robot; The first robot is configured to control its own operation behavior according to the second behavior control model.